Pointers for Info on Symbol Recognition

Hi all,

I just received my M7 together with a robotic arm from another crowd funding project.

As I am new to image processing, recognition, etc. I do need some pointers to information on how to go ahead.

In my project, the M7 should recognize (no tracking required) about 20 different symbol types (eg. runes) on an image
with may contain up to 100 of these symbols. I am not sure on what needs to be done to “teach” the M7 these symbols and make it recognize them.

I was able to run the facetracking examples, but could not derive information on how to use my own
symbols/image templates to teach the M7 or what the options are.
The tutorial was not helpful here, because it only covers intro and basic topics. I am familiar with python, electronics and programming.

If someone could just write down a few lines as an orientation indicating the typical steps to do this kind of project, I could work my way from there.
I would also be willing to document my proceedings and provide my findings in the form of a tutorial back to this community.

Any help is greatly appreciated.

Thank you.

Kind regards

Frank

Hi, thanks for buying M7. So, just a note. We’re rapidly expanding the feature set of the OpenMV Cam. If there’s something it doesn’t do and it’s not too hard to add we’ll try to implement it. The product is still new and being built up.

Anyway, we don’t have good support really for what you want to do except via template matching. Template matching works by having the camera look a like 32x32 image to find out of a larger image (e.g. 320x240). That said, template matching doesn’t handle roation or scale changes at all. It just slides around the image you want to find on the large image and then returns the point of best match. This means its very fragile.

For higher level matching, we do have a Haar Cascade system but that difficult to use other than with the face tracking and eyetracking examples.

Anyway, so, is it possible to replace your runes with AprilTags? This works magically well on the camera. The 36h11 AprilTag family allows for up to about 500 unique tags which can be seen from about 8ft away in bad lighting and it’s rotation, scale, and shear robust. Additionally, if you don’t like the tags looking binary like you could round the pixels to make it look more natural and it would still be found by the OpenMV Cam.

Hi Nyamekye,

Template matching seems a good fit since I do not need scaling and rotation.
I tweaked the template matching example and it works for two runes now.

Memorywise I need to develop a feeling where the limits are.
With QVGA and higher there seems to be a memory issue, I got a “MemoryError: FB Alloc Collision!!!” message
running the template matching example from OpenMV IDE with one 32x32 template and QVGA mode.
For my purpose, QQVGA is sufficient.

This is a cool product with a lot of features to be discovered.
Keep up the good work!

Kind regards,

Frank

Template matching using something called an integral image to work. This requires 4 bytes per pixel in the big image. So, 307200 bytes… plus the size of the image itself which is 320x240x2 = 153600 bytes. So, that’s 460800 in total. The M7 has about 350KB to play with.

We can improve template matching in the future using a rolling integral image window which will fix this problem. However, doing template matching in general requires a lot of CPU. I looked into doing scale and rotation invariant template matching. While possible on the M7 there’s no way it will be real time. So, we’ll probably ever only implement standard template matching since anything above that will take seconds to process which is better left for a desktop CPU. Our focus is only on real-time stuff you want working in the field.