Use OpenMV with UARM Swift to recognize numbers/characters and tap on them

I’m going to attach OpenMV module in UARM swift pro. I’d like to know if there is a way to recognize numbers/characters using OpenMV? Eventually I’d like to use UARM to tap on the letters/numbers which I think I can make it work if I could find a way to recognize the numbers/character.

We actually implement the lenet6 network on the MCU. So, it can detect characters 0 to 9 with better than human like accuracy in a 28x28 pixel window. The feature is still in “demo” mode right now as not a lot of work has been put in to make it easy but it works. It gets above 10 FPS.

Anyway, this new feature will be available in the next firmware release which is coming very soon with OpenMV IDE v1.7.0.

Once I cleared a lot of the lower level machine vision work I plan to try to port a Google mobile net to the system. They have a CCN that can fit in our flash but it has a very low accuracy of about 60 percent.

Thanks a lot. Pls let me know if I can contribute to this project in anyway. Thanks!

Do you know if there is any other way to detect the character as of now? Like taking a picture of the character and have OpenMV to match the image and recognize it? I know this is not smart and hardcoding the data. But I’d like to try a few stuff until we have the real support for character recognition.

I tried template matching. It’s not really reliable. It doesn’t find the 32x32 image all the time. I even tried SEARCH_DS but it crashes the openMV.

I can post the lastest binary with number finding want to try it. We haven’t yet built all this stuff out though so it’s going to be somewhat disappointing now. That said, we do listen to customer request and build out features folks need.

Let me know if you want the binary and example script for just number detection.

That would be very helpful. If the POC works then we need to order more Uarm and OpenMV for our company.

Hi, I’ve attached the new firmware with “find_numbers”.

Please note that I’m probably going to edit the method to make it better for searching the image.

If you want to manually work on portions of the image use the .copy() method which you can pass an roi to create a 28x28 image for find_numbers to work on. The example script just works on the center 28x28 pixels of the camera’s field of view. (1010 KB)

Thanks Nyamekye. Can I flash the firmware from Uarm Studio?

Um, no. You need to download and install OpenMV IDE. Under tools you’ll find a run bootloader option. You can then run the script for OpenMV IDE.

I meant OpenMV IDE. Sorry. It was a typo. :slight_smile:

@ Nyamekye

I tried to “Run Bootloader” and and flash the bin you gave me. But it says “Connecting” and never ends.

Mmm, is this an OpenMV Cam M7, we fixed that issue with the bootloader on it… Okay, we can use DFU mode to get around this but I’ll have to updload a new image file for DFU programming. Will do tonight.

Here’s the DFU image. Use the same run bootloader tool. Follow the instructions. (1020 KB)

Sorry for the late response. I tried with the firmware that you sent me. It’s recognizing the numbers but it’s 60% accurate. I tried with different sizes and different environmental conditions.

Yeah, it’s an experimental feature.

Hi… did you get a chance to have this work out of experimental mode?

I’ll give you a firmware image tonight that can do this. It works but is very basic right now.