I fixed the issue. The UMM heap size was too small. So, it was running out of blocks.
firmware.zip (1.3 MB)
I’ve attached a fixed firmware. VGA should work now.
Worked for me! Thanks so much.
So it looks like AprilTag detection framerate is more or less linear with the number of pixels in the ROI. (I get about 2 fps for full VGA, and about 3fps when I restrict the area by about 1/3.) Are there any other tricks to squeeze every last bit of speed out of the detection process?
Thank you!
We previously hired a performance optimizer to speed the algorithm up and this is as fast as he got it. It’s about 2x faster than baseline already.
There may be more things that can be done. But, I don’t want to spend time on that now until I update us from AprilTag2 to AprilTag3. We should get another 2x performance boost once we do that and then I may be able to improve other things.
Note that the baseline code only gets 30 FPS at 640x480 on an i7 processor from 2016. The algorithm is pretty heavy weight.
Understood! I’ll just work on my end on things like doing everyting I can to minimize needed ROI. (My application involves outlining tagged cards with a projector in realtime, so I’m greedy for every last bit of available framerate!)
Hi @kwagyeman ,
Under what circumstances should i use set_auto_gain(False) and set_auto_whitebal(False)?
You want to turn them off to prevent the camera from adjusting the color gains and general illumination gains in response to what the system sees.
For AprilTags, the algorithm is robust to them changing. Color tracking though is affected by them.
I see.
I am currently trying to make use of apriltag markers to do precision landing outdoors for a drone and i noticed (through a tv shield) that above a certain altitude, the landing markers appear to be over-exposed and are not detected anymore. I’ve looked through the documentation on the camera sensor and it is stated that the exposure algorithms do not change the exposure values much, rather the gain values are changed more instead.
Would it hence be better to set the gain ceilings instead of turning the auto-exposure off? If so, may i know how the gain values affect the exposure of an image? I.e. does higher gain mean higher exposure?
Yeah, if you lower the gain ceiling then the camera will be forced to increase the exposure instead of increasing the gain to deal with changing lighting. This of course will lower the frame rate. But, that shouldn’t be the problem.
Ok. Is there a way to find out what the current gain ceiling is? I noticed that there is a sensor.set_gainceiling but no sensor.get_gainceiling.
Also, is there a difference between setting the gain ceiling using sensor.set_gainceiling(32) vs sensor.set_auto_gain(True, gain_db_ceiling=32)?
That’s an API limitation. However, you can do direct register access to get the gain ceiling. openmv/src/omv/sensors/ov5640.c at master · openmv/openmv (github.com)
As for your second question.They appear to do the same thing in this camera driver for the OV5640.
Ok thank you!
Just curious, but would you happen to have used an ND filter with the openmv cameras before? I was thinking if that could be a way to deal with the very exposed images that i am seeing currently
Yeah, that would help. The sunlight typically causes issues for these types of cameras. We’re going to launch HDR sensors later this year, which will solve your problem. But, until then we just have the OV5640.
1 Like
Does anyone know if inverting the apriltag marker colours will still allow the apriltag detection to work?
That’s a no, it’s looking for a specific pattern.
Ok thanks!
Hi @kwagyeman, i did some testing on the automatic exposure correction for the detection of apriltag markers and i noticed that if i were to do set_auto_gain(True) and then vary the gain_db_ceiling values, the gain_db values don’t really change much and there isn’t any noticeable exposure differences, as compared to if i were to do set_auto_gain(False) and then vary the gain_db values instead.
Should there be an observable change in the gain_db values? How should the gain_db_ceiling values affect the gain_db values?
Is it also advisable to vary the gain_db values in order to work with highly exposed images under bright sunlight? i.e. reduce gain_db values under brighter conditions
My apologies for asking so many questions, i am a complete beginner in this area and i would like to say that i really appreciate all your replies!
So, we’re just setting the gain ceiling register on the OV5640. However, I don’t have documentation for what the right bits to set in the register are. Our driver kinda guesses. OmniVision doesn’t provide support. So, our driver quality is kinda a reverse engineer. On OnSemi cameras we have complete datasheet access and can make the driver better.
Anyway, yeah, you may wish to just control the gain directly based on what you see. This will prevent a loss of frame rate as you can keep the exposure fixed.
Use get_stats() on the whole image or a piece of the image to determine the brightness. This is what the camera is doing internally and what the gain ceiling should control.
…
Also, the questions are great. Since the forum is searchable by google and chatbots it helps build a database of help and support.
I see. May i know where i can find get_stats()? I think it would be really useful to be able to determine the brightness of the image.
Thank you. May i know which method would be best suited for getting the brightness of a grayscale image? Also, is there a maximum/minimum value that can be used for the gain_db argument in set_auto_gain?