Minimal detect AprilTag (don't need tag ID) with better range?

Here is the problem statement: there will be at most one AprilTag in view at all times and I need to find its approximate horizontal position in frame. The approximate size in frame of the tag might be useful but is not necessary. I don’t need the tag ID, its pose, or any other information.

I have tried the find_apriltags function and it can get the job done, but the distance the tag can be seen from is pretty limited unsurprisingly. I’m wondering if there is a way to modify the algorithm or create a new one to only detect a tag rather than read it, and hopefully bump up the range a bit. I’m thinking of using something like a blob finding algorithm to locate the tag most of the time since the background should be somewhat regular. What sort of algorithm is recommended for this purpose? I’ve done some testing and the best I’ve done so far is comparable range to find_apriltags_w_lens_zoom.py, though the current background is more chaotic than the intended background.

If it’s too far maybe you can use the telephoto lens ?

There’s an example script which shows off how to us find_blobs() to do what you want at 640x480. it’s under the AprilTag examples.

We will have a DRAM camera coming out soon which will have no limits on this.

I am thinking about the telephoto lenses, yes. I’m going to order them soon. I’m just worried that the smaller FOV might get in the way of other aspects of the project so I’m looking at other solutions too.

I did also test the blob AprilTag example but with the current chaotic setting its performance wasn’t any better and it ran less than half the framerate. I’ll test it out in the better environment too when I get a chance and hopefully it will do better.

When you say the DRAM camera is coming out soon, how soon? The project I’m working on will be at the point of no return in a couple weeks, but if the camera comes out in the next week or so I would gladly get one and try it out.

Um, I just have dev units. If I get one working I could send it to you. Don’t expect this to happen in a timely manner however.

Oh, if they’re dev units they’d be better used by someone else so no worries. The H7 was a lucky enough find for us and I don’t think we could do our project in the time we have without it. Thanks for doing the work so we don’t have to!

Is this a DRAM camera like this:

Or is a DRAM module between the camera sensor and the mcu ?

If it is between the sensor and the mcu will it slot between either the rolling or global shutter modules or is it on a board with its own sensor ?

Best Wishes

Ed

It’s a new camera with SDRAM for the MCU.

“We will have a DRAM camera coming out soon which will have no limits on this.”

I’m really struggling trying to design my robot to detect April Tags on the floor and to navigate from one to another. The problem is the limited field of view because of the 64K memory limit. I want to place the tags no less than 4" apart and the camera needs to see several in the FOV so that it can see the next one. I bought a brand new printer, a Zebra ZD420 along with their 1" x 1" Poly 3000T plastic labels. I used your program to create 100 April tags, numbered 1 - 100. I then used another utility to crop the bottom 240 rows off because the text is too small. I then used Bartender software to create the labels, essentially 100 labels each with a different image on them, plus a serialized text field below to be the human readable text. The labels look great, with the April tag on top of a number label that shows what is encoded. I have been playing with both the stock lens and the ultra wide angle lens to come up with a FOV and distance that works. I don’t want to have to move the camera too high up to get a bigger FOV. If I could get April Tag decoding with 640 x 480, I think that it would work.

I tried to use Code 39 barcodes instead with VGA and GRAYSCALE, but the decoder isn’t as omnidirectional as it needs to be. If quality dips to 1 - 2, it misdecodes. It does this often when the barcode is at an angle other than 0 or 90, but that is a topic for another post.

If you have any sample cameras with more memory that would give me April Tag decoding with full VGA, I’d love to test it out and report my findings back to you.

I’m working on the H7 with DRAM right now. I need to finish JPEG support for the OV2640 and OV5640. The dram camera will ship with the OV5640 5MP camera.

Sounds exciting to go from VGA to 2MP and 5MP cameras. How long before you can field test them?

I finished the OV2640 driver with jpeg support last night. I will start on the DRAM camera support now. We have the hardware in hand. I just have to work through things now. The OV5640 driver is also almost already done.

I want my robot to navigate across a room between two spots. I’m thinking of mounting a camera at ceiling height (9’) and having it identify two locations on the floor, plus the robot itself. I haven’t tested it yet, so I’m not sure if using April tags or color spots will be the best option. I’m pretty sure however, that I will need a higher resolution camera. I’d like to test your new 2 or 5 MP cameras.

Hi, we just got this working. High res apriltag support is now all done.

The frame rate is a problem however. The SDRAM is currently about 24x slower than internal memory. So, you get frame rates at UXGA that are like 0.2 Hz. This is to be expected however. At QQVGA the frame rate is still bad at like 1.5fps however because of the speed difference (versus 20 fps with internal sram). There may be a way to cache certain data structures in sram to improve performance. But, generally, large random access heap like data structures don’t use the H7s cache well so performance doesn’t look better than the sdram buses speed.

That said, the detection is great though. The algorithm picks out all tags at very high distances.

I’m not a systems designer, but shouldn’t SDRAM be able to run at the processor speed, which is 480MHz? It sounds like you are saying it runs at 20MHz.

The H7 SDRAM bus runs at 80MHz with 16bits of data. Compared to a 480MHz bus with 64-bits of data. We have some knobs we can turn to get this up to 32-bits at 120 MHz but that will be the max. Please remember that we are not using an application processor with a DDR3 RAM interface and L1/L2 cache subsystem.

The STM32 line of processors all support SDRAM only. Not DDR RAM, nor DDR2, nor DDR3, nor DDR4.

… From the get go I haven’t been to big on SDRAM for this reason since the performance is so low compared to the internal SRAM. Anyway, we will release the product no matter what. The cache on the M7 makes some algorithms work quite well. Picture taking capability works great and you can use a 5Mp camera. Lots of filtering algorithms work fine too. QRcode support also works great. Apriltags do a heavy amount of heap random access however so you need a memory subsystem for them that really has a high memory bandwidth. E.g. the cache on the Cortex-M7 processor gets destroyed by the random access pattern which causes cache misses all the time lowering the memory access speed to the SDRAM speed versus the cache being hit a lot which would heavily accelerate things.

We could cache some stuff in the unused SRAM on chip, but, we have to do that in code versus the processor memory subsystem doing it for us for free. So, any performance improvements where we do this will have to be explicit.

I’m going to try out a fix on Saturday that should massively improve performance. I can make fballoc which we use to alloc memory target the sram for things that fit in sram and sdram for things that are bigger. This should give the best of both worlds, speed and space. I should be able to get apriltags performance up by more than 10x. Note, on large resolutions this will be moot. But, QVGA/VGA AprilTag should hit a decent performance sweet spot. Larger resolutions will be super slow still.

It has been awhile since I posted, so I would like an update. I am building my robot and I need to know if I can get a camera to detect an April tag with a bigger window than is currently allowed. Do your 2MP and 5MP cameras have more memory for processing April tags?

Yes, they can do VGA AprilTags at 1fps. 4fps for QVGA apriltags. We are placing the order this week for 1k units.

There’s actually no res limit… But, above VGA your looking at less than 1 fps.

At vga res your looking at 4x more distance.