Optimizing line follower that also read april tags

Hi!

We are a group of electrical engineering students from Argentina, currently doing our thesis. It’s about a line follower vehicle that follows lines with merges/forks and sees april tags, controlled by other microprocessor, but we use the OpenMV to do the image processing part. We have now a code that works fine at 10 FPS.

In our code we call a function every 0.1 secs that calculates the distance to the center of the line AND calls find_apriltags. For our project, high FPS for find_apriltags is not required, whereas the distance to the center of the line is desired to be sampled as fast as possible. Given the fact that we are working with a QQVGA resolution in grayscale, it’s possible to store a copy of an image. So our question is, is there any way that we can run both routines (calculating line error and finding april tags) so that we could increment our line following sampling rate? Is there any way of calling find_apriltags that is not blocking? Could it work with interruptions or multithreading?

Our code is basically just like this:

sensor_init()
While(true):
if(t_elapsed()>0.1):
img=get_img()
e=compute_line_error(img)
tag_found,tag_nmbr = find_tags(img)
communicateUART(e,tag_found,tag_nmbr)

Thanks for your time :smiley:

AprilTags takes the time it takes. If you are calling that then your FPS is limited by the max it runs at. If you want to update the line tracking method at the same time but at a higher rate you have to then cut how long apriltags takes to run.

Yes, but my question is if it is possible to run the line error function while the april tags find is running. For example, detecting april tags at a sampling frequency of 1 Hz and calculating the error to the line at 60Hz. Of course the structure of the code must change, but is there something similar done in other project?

I see, that would require modifying the camera’s firmware to allow two threads to run at the same time (and allow two processes to use the memory management system).

This isn’t supported by our system. Even if you were doing this in C, it would have to be done in an interrupt driven way. You’d have to overcome memory allocation issues during interrupts along with using floats during interrupts as floating point registers are not saved during interrupts.

Additionally, doing any sort of long time processing during an interrupt will cause other parts of the system to crash nicely as you’ll break the timing return requirements for things.

If we can get thread support for MicroPython working then this would also be possible. As long as the framebuffer stack and heap were locked per thread then it could be done.

In the mean-time, I’d use two cameras or make apriltags run on a lower resolution.