Frame Rate April Tags

Discussion related to "under the hood" OpenMV topics.
tobias1234
Posts: 1
Joined: Thu Oct 11, 2018 3:52 pm

Frame Rate April Tags

Postby tobias1234 » Thu Oct 11, 2018 3:58 pm

Hi,

what kind of frame rates are there to expect for april tag pose estimation at VGA resolution with the new H7?

Cheers
Tobias
User avatar
kwagyeman
Posts: 3046
Joined: Sun May 24, 2015 2:10 pm

Re: Frame Rate April Tags

Postby kwagyeman » Thu Oct 11, 2018 6:39 pm

April Tags runs at 240x240 max on the H7 at about 20 FPS.

The AprilTag algorithm has a rather large memory overhead that scales exponentially with resolution. Anyway, in order to get more RAM for the algorithm I locked the res to 256x256 at max. This allowed me to cut the size of the union find table by half allowing for higher reses than 160x120. But, this also limits the max res too.
Nyamekye,
ScottM
Posts: 9
Joined: Thu May 30, 2019 2:12 pm
Location: Ottawa, Canada

Re: Frame Rate April Tags

Postby ScottM » Tue Jun 11, 2019 11:23 am

I'm trying to maximize the frame size for detecting April tags. How do I set it for 256x256 when the sensor.set_framesize command doesn't have that as an option?

Alternatively, can I go with a larger frame size if I switch to Datamatrix or QR Codes?

Thanks.
User avatar
kwagyeman
Posts: 3046
Joined: Sun May 24, 2015 2:10 pm

Re: Frame Rate April Tags

Postby kwagyeman » Tue Jun 11, 2019 12:41 pm

Don't run at 256x256, stay at 240x240. Use the set_windowing method with a resolution of QVGA or higher. set the windowing to 240x240.

As for QrCodes, I believe that takes less RAM. Remember resolution is in 2-dimensions so going up to the next res is 4x more RAM.
Nyamekye,
ScottM
Posts: 9
Joined: Thu May 30, 2019 2:12 pm
Location: Ottawa, Canada

Re: Frame Rate April Tags

Postby ScottM » Thu Jun 13, 2019 12:38 pm

With respect to windowing, I understand, but I am getting an error. I entered the following lines;

sensor.set_framesize(sensor.VGA)
sensor.set_windowing(240,240)

to create a 240x240 window centered inside a VGA frame. I get the following error.

TypeError: function takes 1 positional arguments but 2 were given
ScottM
Posts: 9
Joined: Thu May 30, 2019 2:12 pm
Location: Ottawa, Canada

Re: Frame Rate April Tags

Postby ScottM » Thu Jun 13, 2019 12:47 pm

I did a search and discovered a syntax error. I needed 2 sets of brackets. Why, I have no idea.

sensor.set_windowing((240,240))
ScottM
Posts: 9
Joined: Thu May 30, 2019 2:12 pm
Location: Ottawa, Canada

Re: Frame Rate April Tags

Postby ScottM » Thu Jun 13, 2019 1:09 pm

I found something disturbing. If I use windowing, I cannot decode April Tags. Why is this?

sensor.set_framesize(sensor.VGA)
sensor.set_windowing((240,240))

Scott
User avatar
kwagyeman
Posts: 3046
Joined: Sun May 24, 2015 2:10 pm

Re: Frame Rate April Tags

Postby kwagyeman » Thu Jun 13, 2019 1:19 pm

Hi, it requires brackets because the Python code expects a tuple of 2 or 4 length.

When you increase the AprilTag resolution the algorithm adjusts to the larger memory requirement by reducing a temporary heap size used to store edge point matches. Because this memory pool is smaller the algorithm runs out of RAM and reallocs fail. When this happens point structures it was iterating on are lost. I choose the algorithm to have this behavior versus throwing an error when this happens. The algorithm can start finding new April structures so it's not fatal for all tags in the image. Just large ones that are built from a long list of edge points.

In the H7 240x240 can be achieved. However, I recommend lowering the res to as low as you can get by with. The algorithm is exponentially memory hungry and our RAM did not grow by an exponential amount for the H7.
Nyamekye,
User avatar
kwagyeman
Posts: 3046
Joined: Sun May 24, 2015 2:10 pm

Re: Frame Rate April Tags

Postby kwagyeman » Thu Jun 13, 2019 1:21 pm

Um, try 200x200. That works well.

Note... we might be making an H7 version with RAM where this limit would be removed. The AprilTag algorithm doesn't actually need a lot of memory bandwidth so the external RAM and resolution would not necessary reduce performance.
Nyamekye,
ScottM
Posts: 9
Joined: Thu May 30, 2019 2:12 pm
Location: Ottawa, Canada

Re: Frame Rate April Tags

Postby ScottM » Thu Jun 13, 2019 4:46 pm

I think it would be great to have more RAM available. How about if we could use the micro SD card as memory with the caveat that we'd have to reduce the frame rate? I'd be fine with 5-10 fps if I could use the entire 640 x 480 sensor resolution. I'd use a 1 or 2 GB card.
User avatar
kwagyeman
Posts: 3046
Joined: Sun May 24, 2015 2:10 pm

Re: Frame Rate April Tags

Postby kwagyeman » Thu Jun 13, 2019 7:04 pm

SD Cards can't be used for RAM. Running at 200x200 is about the max the H7 can do with AprilTags. I'm sorry this isn't enough res.

If we have an H7 with RAM working soon I will send you a model. Then this limit will be removed.
Nyamekye,

Return to “Technical Discussion”

Who is online

Users browsing this forum: No registered users and 4 guests