Hi, i’m trying out apriltag detection using the RT1062 with a framesize sensor.VGA and a window of 240x240 and there is this issue where the video feed on the IDE can be seen to be jumping around and does not seem to settle. I’ve tried this on higher resolutions with no issues at all.
May i know what may be causing this issue?
Hi, please avoid using the sensor.set_windowing() and sensor.set_framerate() for right now if possible. The camera sensor driver code is missing some features right now.
If you need the cropping just specify an roi to find_apriltags()
Ok got it!
just to clarify regarding the specification of an roi, do the x and y parameters refer to the mid point? Also is there a way to display the roi only on the IDE?
Just do crop() on the image to get rid of parts you don’t want to see. It’s more CPU than set_windowing() but works for right now. Using ROI in apriltags() directly uses 0 extra CPU.
Its a rect. So, x/y are the top left of the rect. Then w/h are the width/height.
Got it, thanks!
Hi @kwagyeman, may i know if it’s possible to place smaller apriltag markers inside of larger ones? is there a particular tag family that should be used for this if it is possible?
It’s technically possible as along as the larger tag decodes. Tags have a certain number of hamming bits. The H* part is the number of its that can be in error. So, it’s possible to corrupt a bit with a tag inside of that. Maybe 2-4 bits. Tag families don’t have more than 5-bits of hamming distance typically though… So, just be aware this will make it very hard for the algorithm to match tags.
I see. I was looking up the apriltag github and i realised they have a tag family 48H12 that seems to faciliate for the placing of markers within a larger one. May i know if there is support for this tag family currently? i looked up the apriltag object documentation and currently the family is not listed there.
Hi @kwagyeman.
I was testing out using the ROI directly in find_apriltags() and i realised that the fps is quite significantly lesser (about half) than if i were to use crop() on the image or set_windowing(). May i know if this is supposed to be expected?
Mmm, interesting. It’s probably an issue with RAM allocation. We cache things in SRAM to go faster. It must be when you crop that frees up RAM to use the stuff on chip versus the external SDRAM.
We don’t have support for that larger tag family yet. Our library is based on AprilTag 2, not 3 which is more flexible.
I’d like to upgrade to 3… but, I can’t focus on that right now.
Hi @kwagyeman
I’ve ran a couple of simple tests where i varied the resolutions used (SXGA, SVGA, HD) and i noticed that the fps drops quite substantially, by around 40%, when i specify an ROI inside find_apriltags() as compared to if i were to use set_windowing() instead.
The way i am specifying the ROI of 240x240 inside find_apriltags for a resolution of HD (1280x720) is as follows:
img.find_apriltags(roi=(520,240,240,240), families=tag_families, fx=f_x, fy=f_y, cx=c_x, cy=c_y)
You’ve also mentioned that using ROI in find_apriltags() uses 0 extra CPU, so i’m not too sure if what i am seeing now corresponds to that.
It’s because the RAM is already used by the image in the frame buffer.
ROI in this case copies the image into a new frame buffer which fits into internal SRAM, however, this doesn’t leave space for other data structures that need to be allocated in internal SRAM, so, they get moved to SDRAM which is slower. This slows the algorithm down.
It sounds like I need to get windowing working completely ASAP on the RT1062. I will schedule that work.
Given the difference between SRAM and SDRAM in speed, it looks like you get a weird slow down here when our code tries to put the image in SRAM versus internal structures allocated later.
A way around this. Just crop the image using the crop operation. This will bypass the SRAM usage issue when running AprilTags.
Ok, will work with this, thank you!
Hi @kwagyeman,
May i know what is the highest roi resolution that can be used for apriltag detection before the RT1062 runs out of memory? Is it still 240x240?
It shouldn’t have a limit. 640x480 should work fine.
The last time I check it was okay on the H7 Plus and this should be exactly the same on the RT1060.
I can look at this today and see if there’s some type of issue.
I was testing it yesterday and it appeared to run out of gas around 480x400 or so.
This is a burning question for me, as my application really needs the extra resolution at the edges of the image. (I knocked it down to QVGA to at least get something, and tag detection fails at the edges and especially at the corners, because (I believe) the physical tag size I’m using just doesn’t occupy enough pixels there.) If there’s a chance of getting Apriltag detection at full VGA resolution, that would solve all my problems.
Thank you!
Give me a bit. I should be able to fix whatever the issue is pretty quickly today.