Help on Background Subtraction Methods

OpenMV Boards
1

Any help would be much appreciated.
I need an algorithm to provide a rough foreground mask, for a moving camera and a dynamic scene.

  • I have gyroscopic stabilization of the image.

  • For my purposes, It’s safe to assume every moving thing in the frame is a person.

  • I looked into Spatio-temporal LBP’s [ https://core.ac.uk/download/pdf/194697801.pdf ], but sadly, the implementation of image.find_lbp() does not reveal the necessary histogram to construct an STLBP (although it does calculate it- I did not succeed in obtaining it on the micropython level. My C++ is as bad as my English).

  • I looked into frame differencing, and I could not figure out a way to construct a somewhat accurate foreground mask; it only seems to detect the edges of the moving object. I tried both blending and straight differencing.

2

Hi, we’re about to release a feature that will allow you to run bounding box detection models on the H7 Plus, will this work for you?

3

Bounding box detection of people? That’s the dream!
a) Will it be able to run on the portenta h7?
b) What would I need to provide? A trained model?

4

a) Yes
b) EdgeImpulse has a training system for this. You can manually label data. I’ll start working on integration once we get a release out that boosts the system FPS by 2x.

5

Thank you. If I may ask, when do you expect will the bounding box feature to be released?
Also: Roughly, how many times a second under normal load would a person bounding box detection model run?

6

Maybe next month or the month after. Chip shortage is going to redirect time right now.

7

Okay. Will the integration be absolutely necessary for me to get a model going? Wouldn’t I be able to flash the Edge Impulse firmware, train a model, then flash the openmv firmware, take the tf lite file from Edge Impulse and use it with the OpenMV IDE?

8

Uh, yeah, it’s just a lot of super technical work…

9

I have a problem with this. When I try to use my person detection model (by detection I mean a bounding box) in the OpenMV environment, I get Errors:

  • When I use the .tflite file with tf.load(), I get:
    image

But I’m using the Portenta h7, which has 8MB of SDRAM and my model is only 3.7 MB.

  • When I try to use tf.classify(), I get:
    image

What should I do to get my model working?
Regards,
Ido

10

You have to pass the load_to_fb=True flag to load the model to frame buffer ram which is the 8MB. Otherwise it tries to load to the heap. This is documented.

As for the second error… our library doesn’t support an op in your model.

11

Can you maybe add an uncompiled version of libtf.a on github? I would love to try and implement the support.

12

It’s on github… GitHub - openmv/tensorflow-lib: TensorFlow package for Cortex-M4 and Cortex-M7 CPUs with hardware floating point.

You are on your own building it.