Use the OpenMV cam M7 with CMSIS-NN

Discussion related to "under the hood" OpenMV topics.
baptistea
Posts: 6
Joined: Fri Jul 06, 2018 6:06 am

Use the OpenMV cam M7 with CMSIS-NN

Postby baptistea » Fri Jul 06, 2018 6:11 am

Hi guys,

I received a couple a days a brand new OpenMV Cam M7 in order to study its capabilities with Machine Learning algorithms. I saw the last blog post about how to convert a Caffe network and run it on the camera: https://openmv.io/blogs/news
However despite being able to compile and convert the cifar-10 provided example I am completely unable to run it on the cam.

It seems the

Code: Select all

nn
MicroPython package is unknown as I got an

Code: Select all

ImportError
on it. I tried to update the firmware from the OpenMV github https://github.com/openmv/openmv.git but it didn't change anything (I have done it using the Tools -> Bootloader utility, I might be wrong here).
Have I updated it wrongly?
Should I install CMSIS-NN on the camera to make it run correctly?

All help appreciated :p
User avatar
kwagyeman
Posts: 2407
Joined: Sun May 24, 2015 2:10 pm

Re: Use the OpenMV cam M7 with CMSIS-NN

Postby kwagyeman » Fri Jul 06, 2018 1:07 pm

Hi, what version does the IDE display for the firmware? It should display version 3.0.0. We will have the final release of everything soon with the updated IDE.

Please let me know what the firmware version displayed is.
Nyamekye,
baptistea
Posts: 6
Joined: Fri Jul 06, 2018 6:06 am

Re: Use the OpenMV cam M7 with CMSIS-NN

Postby baptistea » Mon Jul 09, 2018 4:17 am

Hi, sorry for the delay, it says I got the 2.9.0 version.

And the https://github.com/openmv/ repo I have is set to the master branch. If I have the 3.0.0 firmware I think my issue is the update of the camera. What I do for that is :

1) In the IDE, Tools -> Run Bootloader
2) Select the firmware.bin file from the https://github.com/openmv/ master branch and Run
3) The bootloader successfully erase everything on the camera and program it
4) I got the message:
Done upgrading your OpenMV Cam's firmware!

Click the Ok button after your OpenMV Cam has enumerated and finished running its built-in self test (blue led blinking - this takes a while).
As I undestand I need to have a blinking blue led and I need to wait for it stop before continuing. But here I got nothing, the blue led is not blinking... It might be the real issue here.
User avatar
kwagyeman
Posts: 2407
Joined: Sun May 24, 2015 2:10 pm

Re: Use the OpenMV cam M7 with CMSIS-NN

Postby kwagyeman » Mon Jul 09, 2018 1:30 pm

Um, okay, so, you can recover by using DFU. Use the run bootloader method with the OpenMV.dfu file and follow up the prompts given to you by the IDE.
Nyamekye,
User avatar
kwagyeman
Posts: 2407
Joined: Sun May 24, 2015 2:10 pm

Re: Use the OpenMV cam M7 with CMSIS-NN

Postby kwagyeman » Mon Jul 09, 2018 1:30 pm

I'll have the next release done by the end of the week.
Nyamekye,
baptistea
Posts: 6
Joined: Fri Jul 06, 2018 6:06 am

Re: Use the OpenMV cam M7 with CMSIS-NN

Postby baptistea » Tue Jul 10, 2018 4:16 am

Yes the IDE always managed to recover the camera to a sane state. That's good to here. I'll wait for the new IDE release then.
User avatar
kwagyeman
Posts: 2407
Joined: Sun May 24, 2015 2:10 pm

Re: Use the OpenMV cam M7 with CMSIS-NN

Postby kwagyeman » Wed Jul 11, 2018 1:52 am

The IDE release is done. v2.0.0 has been released. I'll do the update email soon.
Nyamekye,
baptistea
Posts: 6
Joined: Fri Jul 06, 2018 6:06 am

Re: Use the OpenMV cam M7 with CMSIS-NN

Postby baptistea » Mon Jul 16, 2018 5:36 am

I just quickly tested the new IDE with the new firmware version and I am happy to be able to load some neural networks you provided as examples. I actually managed to run only 1 network: the lenet.network file. For all the other networks, the cifar.network, cifar_fast.network and smile.network I had the same error from the examples:

Code: Select all

MemoryError: Out of normal MicroPython Heap Memory! Please reduce the resolution of the image you are running this algorithm on to bypass this issue!


Even if I ask:

Code: Select all

sensor.set_framesize(sensor.QQQQVGA)
sensor.set_windowing((40, 30))
which is, I guess, the minimum frame size, I still got this issue.

I tried Is this something other people here have when running some network? I am using an 32 Go SDCard containing all the networks I might want to load.
User avatar
kwagyeman
Posts: 2407
Joined: Sun May 24, 2015 2:10 pm

Re: Use the OpenMV cam M7 with CMSIS-NN

Postby kwagyeman » Mon Jul 16, 2018 1:43 pm

Yeah, the network is to big to run the M7. The CNN stuff runs much better on the H7. The MP heap on the M7 is only 128KB and that's shared with everything else going on. On the H7 it increases to 256KB. That said, lenet is usually the network that doesn't fit. the Smile network must not fit because Haar is in the memory. As for the CIFAR one that one I thought fit... Ibrahim?
Nyamekye,
baptistea
Posts: 6
Joined: Fri Jul 06, 2018 6:06 am

Re: Use the OpenMV cam M7 with CMSIS-NN

Postby baptistea » Tue Jul 17, 2018 5:00 am

Yeah seeing the different network sizes I was surprised to see the lenet one only working.
baptistea
Posts: 6
Joined: Fri Jul 06, 2018 6:06 am

Re: Use the OpenMV cam M7 with CMSIS-NN

Postby baptistea » Thu Jul 19, 2018 3:47 am

Quick update. I don't know what I have done earlier but it seems I wrongly stated the NN I could use on the M7. It seems after a quick retest to be more logical as the ones I could load were the smallest in size. The logic is respected. So I don't think it's an issue anymore as I understand the memory is very limited on the device.

I still have several questions that you could help me with:
- Have you made the examples to be run on the H7? Do you which example networks should be runnable on the M7?
- Is there somewhere on the Internet where the M7 shared memory is detailed? I understand it contains at least the OS, maybe the frame buffer if not on the camera chip ? Do we know how much memory it remains for the application?
User avatar
kwagyeman
Posts: 2407
Joined: Sun May 24, 2015 2:10 pm

Re: Use the OpenMV cam M7 with CMSIS-NN

Postby kwagyeman » Fri Jul 20, 2018 1:01 am

This used to be more readable... but, now it's not really.

https://github.com/openmv/openmv/blob/m ... 2fxxx.ld.S
https://github.com/openmv/openmv/blob/m ... nfig.h#L71

So, the CCM size is 128KB. This stores both the .data and .bss along with the stack and heap. Additionally, we have to put a lot of the DMA buffers for various things in there too.

Anyway, the networks are generally as large as their binary file. The format is pretty efficient.
Nyamekye,
User avatar
iabdalkader
Posts: 596
Joined: Sun May 24, 2015 3:53 pm

Re: Use the OpenMV cam M7 with CMSIS-NN

Postby iabdalkader » Mon Jul 23, 2018 12:49 pm

Hi,

The smile.network and cifar10_fast.network work on the F7 (maybe we should rename cifar10_fast to cirfar10_f7). Note the F7 heap is ~52KBs.

Return to “Technical Discussion”

Who is online

Users browsing this forum: No registered users and 331 guests