Use the OpenMV cam M7 with CMSIS-NN

Hi guys,

I received a couple a days a brand new OpenMV Cam M7 in order to study its capabilities with Machine Learning algorithms. I saw the last blog post about how to convert a Caffe network and run it on the camera: News | OpenMV
However despite being able to compile and convert the cifar-10 provided example I am completely unable to run it on the cam.

It seems the

nn

MicroPython package is unknown as I got an

ImportError

on it. I tried to update the firmware from the OpenMV github https://github.com/openmv/openmv.git but it didn’t change anything (I have done it using the Tools → Bootloader utility, I might be wrong here).
Have I updated it wrongly?
Should I install CMSIS-NN on the camera to make it run correctly?

All help appreciated :stuck_out_tongue:

Hi, what version does the IDE display for the firmware? It should display version 3.0.0. We will have the final release of everything soon with the updated IDE.

Please let me know what the firmware version displayed is.

Hi, sorry for the delay, it says I got the 2.9.0 version.

And the OpenMV · GitHub repo I have is set to the master branch. If I have the 3.0.0 firmware I think my issue is the update of the camera. What I do for that is :

  1. In the IDE, Tools → Run Bootloader
  2. Select the firmware.bin file from the OpenMV · GitHub master branch and Run
  3. The bootloader successfully erase everything on the camera and program it
  4. I got the message:

Done upgrading your OpenMV Cam’s firmware!

Click the Ok button after your OpenMV Cam has enumerated and finished running its built-in self test (blue led blinking - this takes a while).

As I undestand I need to have a blinking blue led and I need to wait for it stop before continuing. But here I got nothing, the blue led is not blinking… It might be the real issue here.

Um, okay, so, you can recover by using DFU. Use the run bootloader method with the OpenMV.dfu file and follow up the prompts given to you by the IDE.

I’ll have the next release done by the end of the week.

Yes the IDE always managed to recover the camera to a sane state. That’s good to here. I’ll wait for the new IDE release then.

The IDE release is done. v2.0.0 has been released. I’ll do the update email soon.

I just quickly tested the new IDE with the new firmware version and I am happy to be able to load some neural networks you provided as examples. I actually managed to run only 1 network: the lenet.network file. For all the other networks, the cifar.network, cifar_fast.network and smile.network I had the same error from the examples:

MemoryError: Out of normal MicroPython Heap Memory! Please reduce the resolution of the image you are running this algorithm on to bypass this issue!

Even if I ask:

sensor.set_framesize(sensor.QQQQVGA)
sensor.set_windowing((40, 30))

which is, I guess, the minimum frame size, I still got this issue.

I tried Is this something other people here have when running some network? I am using an 32 Go SDCard containing all the networks I might want to load.

Yeah, the network is to big to run the M7. The CNN stuff runs much better on the H7. The MP heap on the M7 is only 128KB and that’s shared with everything else going on. On the H7 it increases to 256KB. That said, lenet is usually the network that doesn’t fit. the Smile network must not fit because Haar is in the memory. As for the CIFAR one that one I thought fit… Ibrahim?

Yeah seeing the different network sizes I was surprised to see the lenet one only working.

Quick update. I don’t know what I have done earlier but it seems I wrongly stated the NN I could use on the M7. It seems after a quick retest to be more logical as the ones I could load were the smallest in size. The logic is respected. So I don’t think it’s an issue anymore as I understand the memory is very limited on the device.

I still have several questions that you could help me with:

  • Have you made the examples to be run on the H7? Do you which example networks should be runnable on the M7?
  • Is there somewhere on the Internet where the M7 shared memory is detailed? I understand it contains at least the OS, maybe the frame buffer if not on the camera chip ? Do we know how much memory it remains for the application?

This used to be more readable… but, now it’s not really.

https://github.com/openmv/openmv/blob/master/src/omv/stm32fxxx.ld.S

So, the CCM size is 128KB. This stores both the .data and .bss along with the stack and heap. Additionally, we have to put a lot of the DMA buffers for various things in there too.

Anyway, the networks are generally as large as their binary file. The format is pretty efficient.

Hi,

The smile.network and cifar10_fast.network work on the F7 (maybe we should rename cifar10_fast to cirfar10_f7). Note the F7 heap is ~52KBs.