Use OpenMV with UARM Swift to recognize numbers/characters and tap on them

Discussion related to "under the hood" OpenMV topics.
prvns
Posts: 11
Joined: Tue Jul 25, 2017 2:17 pm

Use OpenMV with UARM Swift to recognize numbers/characters and tap on them

Postby prvns » Tue Jul 25, 2017 9:18 pm

I'm going to attach OpenMV module in UARM swift pro. I'd like to know if there is a way to recognize numbers/characters using OpenMV? Eventually I'd like to use UARM to tap on the letters/numbers which I think I can make it work if I could find a way to recognize the numbers/character.
User avatar
kwagyeman
Posts: 1175
Joined: Sun May 24, 2015 2:10 pm

Re: Use OpenMV with UARM Swift to recognize numbers/characters and tap on them

Postby kwagyeman » Wed Jul 26, 2017 2:20 am

We actually implement the lenet6 network on the MCU. So, it can detect characters 0 to 9 with better than human like accuracy in a 28x28 pixel window. The feature is still in "demo" mode right now as not a lot of work has been put in to make it easy but it works. It gets above 10 FPS.

Anyway, this new feature will be available in the next firmware release which is coming very soon with OpenMV IDE v1.7.0.

...

Once I cleared a lot of the lower level machine vision work I plan to try to port a Google mobile net to the system. They have a CCN that can fit in our flash but it has a very low accuracy of about 60 percent.
Nyamekye,
prvns
Posts: 11
Joined: Tue Jul 25, 2017 2:17 pm

Re: Use OpenMV with UARM Swift to recognize numbers/characters and tap on them

Postby prvns » Wed Jul 26, 2017 6:13 am

Thanks a lot. Pls let me know if I can contribute to this project in anyway. Thanks!
prvns
Posts: 11
Joined: Tue Jul 25, 2017 2:17 pm

Re: Use OpenMV with UARM Swift to recognize numbers/characters and tap on them

Postby prvns » Thu Jul 27, 2017 6:00 am

Do you know if there is any other way to detect the character as of now? Like taking a picture of the character and have OpenMV to match the image and recognize it? I know this is not smart and hardcoding the data. But I'd like to try a few stuff until we have the real support for character recognition.
prvns
Posts: 11
Joined: Tue Jul 25, 2017 2:17 pm

Re: Use OpenMV with UARM Swift to recognize numbers/characters and tap on them

Postby prvns » Thu Jul 27, 2017 11:58 am

I tried template matching. It's not really reliable. It doesn't find the 32x32 image all the time. I even tried SEARCH_DS but it crashes the openMV.
User avatar
kwagyeman
Posts: 1175
Joined: Sun May 24, 2015 2:10 pm

Re: Use OpenMV with UARM Swift to recognize numbers/characters and tap on them

Postby kwagyeman » Thu Jul 27, 2017 1:10 pm

I can post the lastest binary with number finding if.you want to try it. We haven't yet built all this stuff out though so it's going to be somewhat disappointing now. That said, we do listen to customer request and build out features folks need.

Let me know if you want the binary and example script for just number detection.
Nyamekye,
prvns
Posts: 11
Joined: Tue Jul 25, 2017 2:17 pm

Re: Use OpenMV with UARM Swift to recognize numbers/characters and tap on them

Postby prvns » Thu Jul 27, 2017 3:51 pm

That would be very helpful. If the POC works then we need to order more Uarm and OpenMV for our company.
User avatar
kwagyeman
Posts: 1175
Joined: Sun May 24, 2015 2:10 pm

Re: Use OpenMV with UARM Swift to recognize numbers/characters and tap on them

Postby kwagyeman » Fri Jul 28, 2017 1:21 am

Hi, I've attached the new firmware with "find_numbers".

Please note that I'm probably going to edit the method to make it better for searching the image.

If you want to manually work on portions of the image use the .copy() method which you can pass an roi to create a 28x28 image for find_numbers to work on. The example script just works on the center 28x28 pixels of the camera's field of view.
Attachments
find_numbers.zip
(1006.6 KiB) Downloaded 36 times
Nyamekye,
prvns
Posts: 11
Joined: Tue Jul 25, 2017 2:17 pm

Re: Use OpenMV with UARM Swift to recognize numbers/characters and tap on them

Postby prvns » Fri Jul 28, 2017 7:22 am

Thanks Nyamekye. Can I flash the firmware from Uarm Studio?
User avatar
kwagyeman
Posts: 1175
Joined: Sun May 24, 2015 2:10 pm

Re: Use OpenMV with UARM Swift to recognize numbers/characters and tap on them

Postby kwagyeman » Fri Jul 28, 2017 11:04 am

Um, no. You need to download and install OpenMV IDE. Under tools you'll find a run bootloader option. You can then run the script for OpenMV IDE.
Nyamekye,
prvns
Posts: 11
Joined: Tue Jul 25, 2017 2:17 pm

Re: Use OpenMV with UARM Swift to recognize numbers/characters and tap on them

Postby prvns » Fri Jul 28, 2017 11:24 am

I meant OpenMV IDE. Sorry. It was a typo. :)
prvns
Posts: 11
Joined: Tue Jul 25, 2017 2:17 pm

Re: Use OpenMV with UARM Swift to recognize numbers/characters and tap on them

Postby prvns » Mon Jul 31, 2017 10:41 am

@ Nyamekye

I tried to "Run Bootloader" and and flash the bin you gave me. But it says "Connecting" and never ends.
User avatar
kwagyeman
Posts: 1175
Joined: Sun May 24, 2015 2:10 pm

Re: Use OpenMV with UARM Swift to recognize numbers/characters and tap on them

Postby kwagyeman » Mon Jul 31, 2017 12:40 pm

Mmm, is this an OpenMV Cam M7, we fixed that issue with the bootloader on it... Okay, we can use DFU mode to get around this but I'll have to updload a new image file for DFU programming. Will do tonight.
Nyamekye,
User avatar
kwagyeman
Posts: 1175
Joined: Sun May 24, 2015 2:10 pm

Re: Use OpenMV with UARM Swift to recognize numbers/characters and tap on them

Postby kwagyeman » Mon Jul 31, 2017 11:53 pm

Here's the DFU image. Use the same run bootloader tool. Follow the instructions.
Attachments
openmv.zip
(1016.41 KiB) Downloaded 26 times
Nyamekye,
prvns
Posts: 11
Joined: Tue Jul 25, 2017 2:17 pm

Re: Use OpenMV with UARM Swift to recognize numbers/characters and tap on them

Postby prvns » Mon Aug 14, 2017 10:10 am

Sorry for the late response. I tried with the firmware that you sent me. It's recognizing the numbers but it's 60% accurate. I tried with different sizes and different environmental conditions.
User avatar
kwagyeman
Posts: 1175
Joined: Sun May 24, 2015 2:10 pm

Re: Use OpenMV with UARM Swift to recognize numbers/characters and tap on them

Postby kwagyeman » Mon Aug 14, 2017 1:29 pm

Yeah, it's an experimental feature.
Nyamekye,

Return to “Technical Discussion”

Who is online

Users browsing this forum: No registered users and 1 guest