AprilTag3 Support

OpenMV Products
1

Thank you for your great work! I look forward to the release of the 5MP + SDRAM version of OpenMV!

I am interested in AprilTag3 support for OpenMV. But, from my reading of this forum it looks like it was quite a task to port the AprilTag algorithm to OpenMV due to heap allocations and limited SRAM.

According to the AprilTag github page, AprilTag3 includes a 2x speed improvement and improved detection rate. AprilTag3 also includes a tag type that has a hole in the middle that can be used for nested/recursive tags. These are all desirable features for the project I am working on.

Would you consider evaluating and possibly adding support for the AprilTag3 algorithm to OpenMV?

Also, can you point me to the version/commit of the AprilTag implementation that was referenced when porting the algorithm to OpenMV? I am curious to see how much the latest AprilTag release has changed from the version in OpenMV and I’d also to understand how much work it was for you to implement.

Thanks!

2

Yeah… it’s a ton of work to update that. Probably 80 hours of effort. That said, if not everything changed it’s not impossible to do it. I haven’t scheduled myself to do this however.

That’s the tracking repo. You can diff that with the new code they have.

Given apriltags is a huge feature of our system I guess that could be the thing to work on next year. It would be like mid next year however. We kinda need more folks helping out with code contributions.

3

Thanks for the quick response.

I did a quick review of the AprilTag3 changes. In case it is helpful here were my finding:

  1. Almost every variable length array (VLA) was changed to dynamically allocated arrays (malloc/calloc/free). I’m not sure why they made that change but it would be easy to change back if VLAs are preferred.
  2. The only major changes were to apriltag.c and apriltag_quad_thresh.c including a variety of algorithmic optimization. I didn’t dig into reviewing the major algorithm changes but it didn’t look intimidating to me (granted I don’t know all you had to do for the original port).
  3. There was one new file that included some functions for optimization, homography, and pose estimation.
  4. Other than that there were some well contained algorithmic optimizations and other insignificant changes.

If I can find time I’ll try to do some further investigation to understand what you had to do to port the algorithm. And if it isn’t too intimidating I will try to start experimenting with a port.

4

Yeah, so, all the code for AprilTag support is in one file. So, you can’t diff changes in there: https://github.com/openmv/openmv/blob/master/src/omv/img/apriltag.c

What you’d have to do is lookup what changes were made for apriltags v3 versus v2, then find the file section in the merged code in the OpenMV repo, and then port the changes over. This process will be quite tedious. That said, not impossible.

Of note, debugging that code was… challenging… if you want to do this… you need to be at ultra pro level.

…
Generally, you’d find what files changed and then read the code line by line looking for diffs. I basically had to learn how all the lines of the apriltag code worked to do the port. That said, if you accomplish it you’ll be quite a lot smarter about how computer vision algorithms work. I definitely learned many new tricks reading that code.

5

Yeah, so, all the code for AprilTag support is in one file. So, you can’t diff changes in there.

What you’d have to do is lookup what changes were made for apriltags v3 versus v2, then find the file section in the merged code in the OpenMV repo, and then port the changes over. This process will be quite tedious. That said, not impossible.

Of note, debugging that code was… challenging… if you want to do this… you need to be at ultra pro level.