Border effect after cropping a binary image

OpenMV Boards
1

Here my code:

import sensor, image, time

thresholdNG = [(30, 255)]
thresholdBIN = [(1,1)]
roi = (234,82,152,152) #VGA
#roi = (117,41,76,76) #QVGA

sensor.reset()
sensor.set_pixformat(sensor.GRAYSCALE)
sensor.set_framesize(sensor.VGA)
sensor.set_windowing(roi)
sensor.set_auto_gain(False)
sensor.set_auto_exposure(False, exposure_us=100)
sensor.skip_frames(time = 1000)
clock = time.clock()

while(True):
    clock.tick()
    img = sensor.snapshot()
    img.binary(thresholdNG,to_bitmap=True)
    for blob in img.find_blobs(thresholdBIN, pixels_threshold=500, merge=True):
        img.draw_cross(blob.cx(), blob.cy(), color=(0,0,0))
        blobx, bloby, blobw, blobh = blob.rect()
        blobroi = (blobx-2,bloby-2,blobw+4,blobh+4)
        break
    img = img.crop(blobroi)
    print("FPS: " + str(clock.fps()))

Ball-binary-03.png
Ball-grayscale.png : snapshot
Ball-binary-00.png
Ball-binary-00.png : image without cropping
Ball-binary-01.png
Ball-binary-01.png : cropping on the blob.rect() rectangle
Ball-binary-02.png
Ball-binary-02.png : cropping on the blob.rect() rectangle - 1 pixel
Ball-grayscale.png
Ball-binary-03.png : cropping on the blob.rect() rectangle - 2 pixels

It seems there is transformation issues on the right of the image (Ball-binary-01.png and Ball-binary-02.png).

And I wonder why there is gray dots on the binary image in the IDE. Is the binary image a jpeg one then in the IDE? Is it the same in the frame buffer?

2

Everything is jpeg compressed to be sent to the IDE. That is done in 8-bit Y. So, grayscale levels can appear.

We will have a new firmware out shortly that should fix this. Crop now uses the draw image pipeline so this issue should go away.

3

I see 3.9.2 firmware is out. Don’t know if it solves the issue.
But I didn’t manage to upgrade the firmware from a zip file. What I done:

  • Jumper between RST and BOOT.


  • From IDE, run bootloader and select OPENMV4P\openmv.dfu


  • Disconnect camera and reconnect. I see “SMT device in DFU mode” in windows devices.

DFU Util says “Waiting for device, exit with ctrl-C”
Then I tried the following:
I copied files from firmware_v3.9.2.zip in C:\Program Files (x86)\OpenMV IDE\share\qtcreator\firmware change text in firmware.txt to 3.9.2 . Nevertheless IDE says 3.8 is still the latest.

4

Hi, don’t use DFU to update the device. That super doesn’t work anymore with the latest ST silicon, they broke their DFU bootloader in hardware. You have to use our firmware.bin file and in non-DFU update mode.

DFU loading ONLY works for loading the bootloader.dfu file onto the OpenMV Cam H7 Plus. From there, you can then use our firmware.bin bootloader to get the main image onboard the camera.

…

If your device bricked? You generally don’t want to use the DFU bootloader as we have our own bootloader which is better.

5

Thank you for your quick answer.
I tried the 3.9.2 firmware by loading through IDE \firmware_v3.9.2\OPENMV4P\firmware.bin
But I got no croping now. img = img.crop(blobroi) seems doing nothing:
Ball-binary-3.9.2.png
I tried again 3.8.0 to be sure by loading \firmware_v3.8.0\OPENMV4P\firmware.bin
Crop function works:
Ball-binary-3.8.0.png

6

The arguments changed. Please pass:

roi=[x, y, w, h]

7

I done:
blobroi = [blobx-2,bloby-2,blobw+4,blobh+4]
but nothing changed

8

Just “roi”. image — machine vision — MicroPython 1.15 documentation

9

Sorry, didn’t understand well, I was changing the variable assignment in the for loop, not the argument in the crop function.
My crop code is now working: img.crop(roi=blobroi)
But I still have the same artefact that for 3.8.0.
Ball-binary-3.9.2-2.png

10

Okay, I can check tonight, can you give me the full code and files the generate the error and I will fix it.

Thanks,

11

Um, did you try cropping in the latest firmware?

12

It is the latest firmware as reported here:
MicroPython: v1.13-r63 OpenMV: v3.9.2 HAL: v1.9.0 BOARD: OPENMV4P-STM32H743

Here the code:

import sensor, image, time

thresholdNG = [(30, 255)]
thresholdBIN = [(1,1)]
roi = (234,82,152,152) #VGA
#roi = (117,41,76,76) #QVGA

sensor.reset()
sensor.set_pixformat(sensor.GRAYSCALE)
sensor.set_framesize(sensor.VGA)
sensor.set_windowing(roi)
sensor.set_auto_gain(False)
sensor.set_auto_exposure(False, exposure_us=100)
sensor.skip_frames(time = 1000)
clock = time.clock()

while(True):
    clock.tick()
    img = sensor.snapshot()
    img.binary(thresholdNG,to_bitmap=True)
    for blob in img.find_blobs(thresholdBIN, pixels_threshold=500, merge=True):
        img.draw_cross(blob.cx(), blob.cy(), color=(0,0,0))
        blobx, bloby, blobw, blobh = blob.rect()
        blobroi = (blobx-2,bloby-2,blobw+4,blobh+4)
        break
    img = img.crop(roi=blobroi)
    print("FPS: " + str(clock.fps()))

Sometime the artefact is subtle, it doesn’t occur all the time in a same way. The right of the circle should be straight + 1 pixel like here (without cropping):
Ball-binary-3.9.2-1.png
Sometimes it is like that:
Ball-binary-3.9.2-2.png
Or like that:
Ball-binary-3.9.2-3.png
I wish I could post here shapshots from the camera for you to check. Is the only solution is to do some img_writer.add_frame(img) on an sdcard?

13

Here a 10s stream of snapshots done with image.ImageIO(“/stream.bin”, “w”)
stream.zip (3.47 MB)

14

Taking a look now.

15

Okay, I see the issue. It looks like the binary image is getting corrupted in the last few pixels.

16

Found it. It was a jpeg compressor bug. 1 line of code fix. What camera do you have? I can make a binary to fix it.

17

Nice. :slight_smile:
When I saw you published a new IDE/firmware release, I imagined you were quite busy.
If it is only an artefact on the IDE display, I can live with it for the moment till the next firmware release, saving you some time.
My cam is a H7plus if you already compiled the firmware.
I have 3 H7plus for the moment. Figuring out if they fit my project. And I hope so as I found your products amazing.

18

Fix is in this firmware.
firmware.zip (1.13 MB)

19

Thanx.
stream-c1.png