Higher FPS

Hi, I’ve been working on the sensor code lately, and it seems the sensor actually supports a much higher FPS, without going into too many details, the higher FPS enables the cam to capture frames when needed and run faster (unless it’s running a slow algorithm). So, I’m attaching a firmware image if anyone’s interested in testing. Please let me know if you find any problems (but keep in mind this is a development firmware image) you should see at least 42FPS when running helloworld.py (if you disable the FB it goes up to 84FPS :open_mouth: and same for color blobs using a single blob)

FPS.gif
firmware.zip (815 KB)

I just flashed my OpenMV M7 with your firmware, but don’t any changes in FPS. It’s still around 16FPS with FB enabled and around 28-29FPS with FB disabled.

What I did to flash the device:

  1. Connected OpenMV to Windows laptop
  2. Opened OpenMV IDE
  3. Without connecting to the device: Tools → Run Bootloader
  4. Selected extracted firmware.bin file
  5. Pressed Run with “erase internal FS” checked.

What I’m doing wrong?

1.png
2.png

Can you try a single color tracking script? The multi-blob one goes over the image 3 times for each color. I tested with the single color script and got 56 FPS or so.

I tuned it a bit. I’ve switchted to grayscale and decreased framesize. I also decresed size of roi.
So now it’s around 80FPS.

Thank you very much for your work, guys.

Hello gentlemen,

I tried to follow sergeyzwezdin install instructions, but it never makes it past the “if this takes more than 5 seconds please cancel it” message.

After I cancel it says it can’t contact the boot-loader. The current firmware is at 2.7.0. This looks rather new, does it work, I mean it should erase the current firmware in the Arm.

I can see the green led doing it’s boot-loader thing, but then it just hangs. Is there a work around for this?
Do I need to jump a pin to ground someplace?

Thanks for the help you can provide,

Robert

Hmm, please wait for the latest IDE. It will come out this weekend.

We did an update of the bootloader and the old IDE can’t handle it. If you updated your board’s firmware using DFU recently then you’d have this issue.

That said, you may also use DFU to update your firmware. Please see the OpenMV.dfu file in the firmware folder.

Yes, I did update the firmware from the IDE. And if so I have this problem. If the new IDE come out this weekend, then I don’t need to worry about it…

In the worst case, I do have The Easy STM32 Pro development board and the debugger/programmer is built into the board. I have not looked at the pin-out of the openMV M7 cam, But I’m going to
guess, that the Programming pins can be accessed there.

If so I can just update the firmware with that too. But I think your boot loader would get erased by doing this. I’ll just wait for the new IDE. :smiley:

Alas I did look in that firmware folder that I downloaded from this topic, It only contains firmware.bin, there is no other file contained in it that I can find.

I did download openMV.dfu zip file from 2016 from this site. But I’m not sure if it’s the most current, so I did not use it to update the firmware.

What are you trying to do exactly?

I’m building a robot, and it is moving a 1 meter per second, it needs to read a data matrix (10x10) at that speed.
The frames per second are around 6 to 7 FPS. I’m worrying, that it will not be able to read the data matrix at that speed.

So i’m trying to get the FPS as high as possible…

Has that IDE been released as of yet? Note, I’m doing this in a windows environment, can also do this in Linux. if it okay to use the DFU that was created in 2016 to place this firmware on the M7?

Hi, I’m almost done with the release. I’ve just finishing up releasing for the Pi right now and will post it in a few hours.

Um, so, data matrix speed is limited by the CPU. Messing around with the camera sensor will not improve it. That said, you can change the effort level the data matrix algorithm uses. This might help.

Hello there,

I just tried this out on my laptop with windows 10, the IDE placed the firmware on the M7 without any issues. However the windows 7 desktop seems to have an issue, I will make sure that both IDE’s are the same version.

Once I did place this firmware on it and tried to run it did complain about my program, I will try it with the hello world one…

Thanks for your hard work,

Robert

hello there, I’m using the 1.9.0 version now. Works perfect!

By adjusting the frame size to QQQVGA I can double the FPS count. However if I try the QQQQVGA it has an issue with FB How do I disable FB?

I’m not sure where this is located

Robert

P.S never mind the FB I found it from a search here…

Not really sure QQQQVGA works correctly. That’s a 40x30 image right? Mmm, will fix.

Um, for a higher FPS play around with the sensor exposure control. You can go way higher if you lower it. But, to adjust you have to make the gain go way up which amplifies the noise. See the camera exposure control scripts.

Yes, that is the right image size for QQQQVGA, I can turn off the Frame buffer in the ide but when I save the script to the OpenMV M7 it still
gets frame buffer error in a serial application.

It does work in the IDE with it disabled, just can’t seem to save it to the M7. I tried to do it manually in the code, but never got it to disable,
do to my newness of this ide. (not sure of how to write it in the code thus far.)

I will have a look at exposure time also, thanks for the tip,

Robert

Hi, you can’t disable the frame buffer in the code. When you click the button in the IDE it just tells the camera it need not try to read a JPEG image from the camera. The camera will not produce another JPEG image if the current one has not been read.

Um, anyway, what’s the error otherwise? Can you take a screen shot?

Hi there,
when using QQQQVGA I get a frame buffer collision error in the serial terminal. I’ll list the exact error in a few mins.

Exact error message in terminal window, also in the IDE:

Traceback (most recent call last):
File “/main.py”, line 22, in
MemoryError: FB Alloc Collision!!!
MicroPython v1.9.3-4944-gd5396e0a4 on 2018-05-06; OPENMV3 with STM32F765
Type “help()” for more information.

And the code is:

# OpenMV 10x10 data matrix decoder
# Added the Infrared LED'S to the project
# Removed the IR blocker on the camera lens, it can now read matrix codes in the dark
# Pi 3.14159265359

import sensor, image, time, math, pyb

from pyb import LED
red_led   = LED(1)
green_led = LED(2)
blue_led  = LED(3)
ir_led    = LED(4)

def led_control(x):

#    if (x&8)==8: ir_led.on()
    if (x&1)==1: red_led.on()
#    if (x&2)==2: green_led.on()
#    if (x&4)==4: blue_led.on()



sensor.reset()
sensor.set_pixformat(sensor.GRAYSCALE) # GrayScale runs faster
#sensor.set_pixformat(sensor.RGB565)
sensor.set_framesize(sensor.QQQQVGA)
sensor.skip_frames(time = 2000)
sensor.set_auto_gain(False)  # must turn this off to prevent image washout...
sensor.set_auto_whitebal(False)  # must turn this off to prevent image washout...
clock = time.clock()

while(True):
    clock.tick()
    img = sensor.snapshot()
    img.lens_corr(1.8) # strength of 1.8 is good for the 2.8mm lens.
    for i in range(16):
        led_control(i)

    matrices = img.find_datamatrices()
    for matrix in matrices:
        img.draw_rectangle(matrix.rect(), color = (255, 0, 0))
#        print_args = (matrix.rows(), matrix.columns(), matrix.payload(), (180 * matrix.rotation()) / math.pi, clock.fps())
#        print("Matrix [%d:%d], Payload \"%s\", rotation %f (degrees), FPS %f" % print_args)
#        print("N"+ matrix.payload()+"X"+ str(matrix.x())+"Y"+ str(matrix.y())+"R"+ str(matrix.rotation()))
        Cx = matrix.x() + (matrix.w()/2) # get the center of the matrix "X" value
        Cy = matrix.y() + (matrix.h()/2) # get the center of the matrix "Y" value
        Xoff = (img.width()/2) - Cx # Determine how much if any error is present for the X axis
        Yoff = (img.height()/2) - Cy # Determine how much if any error is present for the Y axis

        print("N"+ str(matrix.payload())+"X"+ str(int(math.ceil(Xoff)))+"Y"+ str(int(math.ceil(Yoff)))+"R"+ str(int(math.ceil((180 * matrix.rotation() / math.pi)))))

Hmm, that’s means you’re out of RAM. Need to make that error more verbose.

Um, the resolution is really small however. This shouldn’t happen.

I’ll try your code out and see what’s up. You should have no memory issues.

Hi there,

Any new information yet?