Minimal detect AprilTag (don't need tag ID) with better range?

OpenMV related project discussion.
qschebdrap
Posts: 3
Joined: Mon Jul 08, 2019 5:03 pm

Minimal detect AprilTag (don't need tag ID) with better range?

Postby qschebdrap » Tue Jul 09, 2019 8:27 am

Here is the problem statement: there will be at most one AprilTag in view at all times and I need to find its approximate horizontal position in frame. The approximate size in frame of the tag might be useful but is not necessary. I don't need the tag ID, its pose, or any other information.

I have tried the find_apriltags function and it can get the job done, but the distance the tag can be seen from is pretty limited unsurprisingly. I'm wondering if there is a way to modify the algorithm or create a new one to only detect a tag rather than read it, and hopefully bump up the range a bit. I'm thinking of using something like a blob finding algorithm to locate the tag most of the time since the background should be somewhat regular. What sort of algorithm is recommended for this purpose? I've done some testing and the best I've done so far is comparable range to find_apriltags_w_lens_zoom.py, though the current background is more chaotic than the intended background.
User avatar
kwagyeman
Posts: 3513
Joined: Sun May 24, 2015 2:10 pm

Re: Minimal detect AprilTag (don't need tag ID) with better range?

Postby kwagyeman » Tue Jul 09, 2019 2:45 pm

There's an example script which shows off how to us find_blobs() to do what you want at 640x480. it's under the AprilTag examples.

We will have a DRAM camera coming out soon which will have no limits on this.
Nyamekye,
qschebdrap
Posts: 3
Joined: Mon Jul 08, 2019 5:03 pm

Re: Minimal detect AprilTag (don't need tag ID) with better range?

Postby qschebdrap » Tue Jul 09, 2019 2:58 pm

I am thinking about the telephoto lenses, yes. I'm going to order them soon. I'm just worried that the smaller FOV might get in the way of other aspects of the project so I'm looking at other solutions too.

I did also test the blob AprilTag example but with the current chaotic setting its performance wasn't any better and it ran less than half the framerate. I'll test it out in the better environment too when I get a chance and hopefully it will do better.

When you say the DRAM camera is coming out soon, how soon? The project I'm working on will be at the point of no return in a couple weeks, but if the camera comes out in the next week or so I would gladly get one and try it out.
User avatar
kwagyeman
Posts: 3513
Joined: Sun May 24, 2015 2:10 pm

Re: Minimal detect AprilTag (don't need tag ID) with better range?

Postby kwagyeman » Tue Jul 09, 2019 3:58 pm

Um, I just have dev units. If I get one working I could send it to you. Don't expect this to happen in a timely manner however.
Nyamekye,
qschebdrap
Posts: 3
Joined: Mon Jul 08, 2019 5:03 pm

Re: Minimal detect AprilTag (don't need tag ID) with better range?

Postby qschebdrap » Tue Jul 09, 2019 4:13 pm

Oh, if they're dev units they'd be better used by someone else so no worries. The H7 was a lucky enough find for us and I don't think we could do our project in the time we have without it. Thanks for doing the work so we don't have to!
ecdm2
Posts: 11
Joined: Sun Sep 09, 2018 6:15 am

Re: Minimal detect AprilTag (don't need tag ID) with better range?

Postby ecdm2 » Fri Jul 12, 2019 7:51 am

Is this a DRAM camera like this:

https://hackaday.com/2014/04/05/taking- ... dram-chip/

Or is a DRAM module between the camera sensor and the mcu ?

If it is between the sensor and the mcu will it slot between either the rolling or global shutter modules or is it on a board with its own sensor ?

Best Wishes

Ed
User avatar
iabdalkader
Posts: 1039
Joined: Sun May 24, 2015 3:53 pm

Re: Minimal detect AprilTag (don't need tag ID) with better range?

Postby iabdalkader » Sat Jul 13, 2019 3:44 pm

ecdm2 wrote:
Fri Jul 12, 2019 7:51 am
Is this a DRAM camera like this:

https://hackaday.com/2014/04/05/taking- ... dram-chip/

Or is a DRAM module between the camera sensor and the mcu ?

If it is between the sensor and the mcu will it slot between either the rolling or global shutter modules or is it on a board with its own sensor ?

Best Wishes

Ed
It's a new camera with SDRAM for the MCU.
ScottM
Posts: 62
Joined: Thu May 30, 2019 2:12 pm
Location: Ottawa, Canada

Re: Minimal detect AprilTag (don't need tag ID) with better range?

Postby ScottM » Fri Aug 16, 2019 7:59 pm

"We will have a DRAM camera coming out soon which will have no limits on this."

I'm really struggling trying to design my robot to detect April Tags on the floor and to navigate from one to another. The problem is the limited field of view because of the 64K memory limit. I want to place the tags no less than 4" apart and the camera needs to see several in the FOV so that it can see the next one. I bought a brand new printer, a Zebra ZD420 along with their 1" x 1" Poly 3000T plastic labels. I used your program to create 100 April tags, numbered 1 - 100. I then used another utility to crop the bottom 240 rows off because the text is too small. I then used Bartender software to create the labels, essentially 100 labels each with a different image on them, plus a serialized text field below to be the human readable text. The labels look great, with the April tag on top of a number label that shows what is encoded. I have been playing with both the stock lens and the ultra wide angle lens to come up with a FOV and distance that works. I don't want to have to move the camera too high up to get a bigger FOV. If I could get April Tag decoding with 640 x 480, I think that it would work.

I tried to use Code 39 barcodes instead with VGA and GRAYSCALE, but the decoder isn't as omnidirectional as it needs to be. If quality dips to 1 - 2, it misdecodes. It does this often when the barcode is at an angle other than 0 or 90, but that is a topic for another post.

If you have any sample cameras with more memory that would give me April Tag decoding with full VGA, I'd love to test it out and report my findings back to you.
User avatar
kwagyeman
Posts: 3513
Joined: Sun May 24, 2015 2:10 pm

Re: Minimal detect AprilTag (don't need tag ID) with better range?

Postby kwagyeman » Fri Aug 16, 2019 8:11 pm

I'm working on the H7 with DRAM right now. I need to finish JPEG support for the OV2640 and OV5640. The dram camera will ship with the OV5640 5MP camera.
Nyamekye,
ScottM
Posts: 62
Joined: Thu May 30, 2019 2:12 pm
Location: Ottawa, Canada

Re: Minimal detect AprilTag (don't need tag ID) with better range?

Postby ScottM » Sat Aug 17, 2019 9:45 pm

Sounds exciting to go from VGA to 2MP and 5MP cameras. How long before you can field test them?
User avatar
kwagyeman
Posts: 3513
Joined: Sun May 24, 2015 2:10 pm

Re: Minimal detect AprilTag (don't need tag ID) with better range?

Postby kwagyeman » Sun Aug 18, 2019 10:34 am

I finished the OV2640 driver with jpeg support last night. I will start on the DRAM camera support now. We have the hardware in hand. I just have to work through things now. The OV5640 driver is also almost already done.
Nyamekye,
ScottM
Posts: 62
Joined: Thu May 30, 2019 2:12 pm
Location: Ottawa, Canada

Re: Minimal detect AprilTag (don't need tag ID) with better range?

Postby ScottM » Tue Sep 03, 2019 10:26 am

I want my robot to navigate across a room between two spots. I'm thinking of mounting a camera at ceiling height (9') and having it identify two locations on the floor, plus the robot itself. I haven't tested it yet, so I'm not sure if using April tags or color spots will be the best option. I'm pretty sure however, that I will need a higher resolution camera. I'd like to test your new 2 or 5 MP cameras.
User avatar
kwagyeman
Posts: 3513
Joined: Sun May 24, 2015 2:10 pm

Re: Minimal detect AprilTag (don't need tag ID) with better range?

Postby kwagyeman » Tue Sep 03, 2019 10:43 am

Hi, we just got this working. High res apriltag support is now all done.

The frame rate is a problem however. The SDRAM is currently about 24x slower than internal memory. So, you get frame rates at UXGA that are like 0.2 Hz. This is to be expected however. At QQVGA the frame rate is still bad at like 1.5fps however because of the speed difference (versus 20 fps with internal sram). There may be a way to cache certain data structures in sram to improve performance. But, generally, large random access heap like data structures don't use the H7s cache well so performance doesn't look better than the sdram buses speed.

That said, the detection is great though. The algorithm picks out all tags at very high distances.
Nyamekye,
ScottM
Posts: 62
Joined: Thu May 30, 2019 2:12 pm
Location: Ottawa, Canada

Re: Minimal detect AprilTag (don't need tag ID) with better range?

Postby ScottM » Tue Sep 03, 2019 2:55 pm

I'm not a systems designer, but shouldn't SDRAM be able to run at the processor speed, which is 480MHz? It sounds like you are saying it runs at 20MHz.
User avatar
kwagyeman
Posts: 3513
Joined: Sun May 24, 2015 2:10 pm

Re: Minimal detect AprilTag (don't need tag ID) with better range?

Postby kwagyeman » Tue Sep 03, 2019 4:50 pm

The H7 SDRAM bus runs at 80MHz with 16bits of data. Compared to a 480MHz bus with 64-bits of data. We have some knobs we can turn to get this up to 32-bits at 120 MHz but that will be the max. Please remember that we are not using an application processor with a DDR3 RAM interface and L1/L2 cache subsystem.

The STM32 line of processors all support SDRAM only. Not DDR RAM, nor DDR2, nor DDR3, nor DDR4.

... From the get go I haven't been to big on SDRAM for this reason since the performance is so low compared to the internal SRAM. Anyway, we will release the product no matter what. The cache on the M7 makes some algorithms work quite well. Picture taking capability works great and you can use a 5Mp camera. Lots of filtering algorithms work fine too. QRcode support also works great. Apriltags do a heavy amount of heap random access however so you need a memory subsystem for them that really has a high memory bandwidth. E.g. the cache on the Cortex-M7 processor gets destroyed by the random access pattern which causes cache misses all the time lowering the memory access speed to the SDRAM speed versus the cache being hit a lot which would heavily accelerate things.

We could cache some stuff in the unused SRAM on chip, but, we have to do that in code versus the processor memory subsystem doing it for us for free. So, any performance improvements where we do this will have to be explicit.
Nyamekye,
User avatar
kwagyeman
Posts: 3513
Joined: Sun May 24, 2015 2:10 pm

Re: Minimal detect AprilTag (don't need tag ID) with better range?

Postby kwagyeman » Fri Sep 06, 2019 3:25 pm

I'm going to try out a fix on Saturday that should massively improve performance. I can make fballoc which we use to alloc memory target the sram for things that fit in sram and sdram for things that are bigger. This should give the best of both worlds, speed and space. I should be able to get apriltags performance up by more than 10x. Note, on large resolutions this will be moot. But, QVGA/VGA AprilTag should hit a decent performance sweet spot. Larger resolutions will be super slow still.
Nyamekye,
ScottM
Posts: 62
Joined: Thu May 30, 2019 2:12 pm
Location: Ottawa, Canada

Re: Minimal detect AprilTag (don't need tag ID) with better range?

Postby ScottM » Wed Oct 30, 2019 1:02 pm

It has been awhile since I posted, so I would like an update. I am building my robot and I need to know if I can get a camera to detect an April tag with a bigger window than is currently allowed. Do your 2MP and 5MP cameras have more memory for processing April tags?
User avatar
kwagyeman
Posts: 3513
Joined: Sun May 24, 2015 2:10 pm

Re: Minimal detect AprilTag (don't need tag ID) with better range?

Postby kwagyeman » Wed Oct 30, 2019 1:22 pm

Yes, they can do VGA AprilTags at 1fps. 4fps for QVGA apriltags. We are placing the order this week for 1k units.
Nyamekye,
User avatar
kwagyeman
Posts: 3513
Joined: Sun May 24, 2015 2:10 pm

Re: Minimal detect AprilTag (don't need tag ID) with better range?

Postby kwagyeman » Wed Oct 30, 2019 1:23 pm

There's actually no res limit... But, above VGA your looking at less than 1 fps.

At vga res your looking at 4x more distance.
Nyamekye,
ScottM
Posts: 62
Joined: Thu May 30, 2019 2:12 pm
Location: Ottawa, Canada

Re: Minimal detect AprilTag (don't need tag ID) with better range?

Postby ScottM » Fri Nov 01, 2019 10:05 am

This is great news. Just yesterday, I cut a 80 x 80 mm hole in my Superdroids robot base and designed and printed a mount for the OpenMV camera. The camera lens is 42 mm above the base and approximately 125 mm off the floor. It looks through the hole to April tags on the floor. You can see what it looks like in the attached picture. Even with the wide angle lens, at this distance, I'm not getting a very large FOV. I'm going to use a UWB indoor positioning system to tell the robot to go to certain marks on the floor, but the accuracy is stated as within 10 cm. At the marks, I will put an April tag. Ideally, I'd like the OpenMV camera FOV to be larger than this. Otherwise, I plan to put more April tags around the target one to guide the robot the last few cm.

If I could get one of the new cameras, I could enlarge the hole and get a bigger FOV. When do you think they will be available?

Scott
Attachments
OpenMV mount.PNG
User avatar
kwagyeman
Posts: 3513
Joined: Sun May 24, 2015 2:10 pm

Re: Minimal detect AprilTag (don't need tag ID) with better range?

Postby kwagyeman » Fri Nov 01, 2019 12:19 pm

I can send you like one model that lacks the Qspi flash. It will work as long as an SD card is inserted. Email me about this and I'll make a PO for you.

Note, I don't have extra OV5640 cameras. So, it will be just the base.
Nyamekye,

Return to “Project Discussion”

Who is online

Users browsing this forum: No registered users and 1 guest