Collect ML sample images through serial

Hi,

I am playing with the machine learning capabilities of the camera, therefore, I’d like to collect positive and negative sample images. Saving them on the SD card seemed to be unstable since if the camera was reset or disconnected, the captured data was lost sometimes. I was looking for a way to save the raw image through serial port. img.save(‘img.bmp’) works fine, but sending it over the serial port and saving on a host machine does not. At the end, the file is different than the one saved on the SD card.

The code on the cam:

...
usb = USB_VCP()
...
img = sensor.snapshot() # Take a picture and return the image.
usb.send(img)

The code on the host:

ser = serial.Serial('/dev/ttyACM0')  # open serial port
ser.baudrate = 115200

# This is just a simplified version, I am sure the correct section of the data is captured
# since data separators are used.
data = ser.read(38400) # Read data

f = open("img.bmp", 'wb')  # open in binary
f.write(data)
f.close()

Do you have any idea what’s wrong here?

Thanks,
Peter

Hi, just use the ImageWriter method. The latest OpenMV IDE release can turn the ImageWriter files to a set of PNG/BMP/JPG image file outputs. You just have to say “*.jpg” as the target filename and FFMPEG will convert the video to a set of images.

As for your code. The data sent by the OpenMV Cam is not a BMP file but instead the raw image data. Do this instead:

...
usb = USB_VCP()
...
img = sensor.snapshot().compress() # Take a picture and return the image.
usb.send(struct.pack("<L", img.size()))
usb.send(img)



ser = serial.Serial('/dev/ttyACM0')  # open serial port
ser.baudrate = 115200

# This is just a simplified version, I am sure the correct section of the data is captured
# since data separators are used.
size = struct.unpack("<L", ser.read(4))[0]
data = ser.read(size) # Read data

f = open("img.jpg", 'wb')  # open in binary
f.write(data)
f.close()

Ok, a bit more detail: my goal is to record many images through serial from many cameras. Positive and negative ones. It’s already done btw. I was about to record raw images, but you are right, it’s far better to save compressed images and decompress them before using them in ML.

I do not know yet if I will be able to train a small enough network for the camera to work with. If the CNN network turns out to be too complex for the cam, I will be using the versatile built-in tools of the OpenMV cam manually (no ML involved), but I would like to backtest my code with the same images so they need to be reloaded into the camera.

I am able to save the compressed images via serial, but reusing them with OpenMV IDE is a partial success. I am converting the compressed image back into bitmap, and loading the image. The image shows up in the IDE, but I have some issues with the colors. :slight_smile: Blue turns out to be orange for some reason.

The compressed image recorded:
img_cake.jpg
The decompressed image: not able to attach in the forum, but generated with the code below.

The image showing up in the IDE:
img_cake_ide.jpg
The way I am converting it back to bitmap:

from PIL import Image

img = Image.open('images/img_cake.jpg')
im_conv = img.convert('RGB')
im_conv.save('images/img_cake.bmp')

Mmm, um, it’s likely the BMP byte order being wrong. My BMP code is old and needs to be updated. Just haven’t got to it yet. Um, save as a PPM file and you won’t have any issues. PPM/PGM are much simpler file formats and we can parse all variants.

Perfect, it works! Thank you for the great support!