Collect ML sample images through serial

Discussion related to "under the hood" OpenMV topics.
pvadam
Posts: 9
Joined: Tue May 01, 2018 3:25 am

Collect ML sample images through serial

Postby pvadam » Mon Aug 06, 2018 6:44 pm

Hi,

I am playing with the machine learning capabilities of the camera, therefore, I'd like to collect positive and negative sample images. Saving them on the SD card seemed to be unstable since if the camera was reset or disconnected, the captured data was lost sometimes. I was looking for a way to save the raw image through serial port. img.save('img.bmp') works fine, but sending it over the serial port and saving on a host machine does not. At the end, the file is different than the one saved on the SD card.

The code on the cam:

Code: Select all

...
usb = USB_VCP()
...
img = sensor.snapshot() # Take a picture and return the image.
usb.send(img)
The code on the host:

Code: Select all

ser = serial.Serial('/dev/ttyACM0')  # open serial port
ser.baudrate = 115200

# This is just a simplified version, I am sure the correct section of the data is captured
# since data separators are used.
data = ser.read(38400) # Read data

f = open("img.bmp", 'wb')  # open in binary
f.write(data)
f.close()
Do you have any idea what's wrong here?

Thanks,
Peter
User avatar
kwagyeman
Posts: 2116
Joined: Sun May 24, 2015 2:10 pm

Re: Collect ML sample images through serial

Postby kwagyeman » Mon Aug 06, 2018 10:29 pm

Hi, just use the ImageWriter method. The latest OpenMV IDE release can turn the ImageWriter files to a set of PNG/BMP/JPG image file outputs. You just have to say "*.jpg" as the target filename and FFMPEG will convert the video to a set of images.
Nyamekye,
User avatar
kwagyeman
Posts: 2116
Joined: Sun May 24, 2015 2:10 pm

Re: Collect ML sample images through serial

Postby kwagyeman » Mon Aug 06, 2018 10:31 pm

As for your code. The data sent by the OpenMV Cam is not a BMP file but instead the raw image data. Do this instead:

Code: Select all

...
usb = USB_VCP()
...
img = sensor.snapshot().compress() # Take a picture and return the image.
usb.send(struct.pack("<L", img.size()))
usb.send(img)

Code: Select all

ser = serial.Serial('/dev/ttyACM0')  # open serial port
ser.baudrate = 115200

# This is just a simplified version, I am sure the correct section of the data is captured
# since data separators are used.
size = struct.unpack("<L", ser.read(4))[0]
data = ser.read(size) # Read data

f = open("img.jpg", 'wb')  # open in binary
f.write(data)
f.close()
Nyamekye,
pvadam
Posts: 9
Joined: Tue May 01, 2018 3:25 am

Re: Collect ML sample images through serial

Postby pvadam » Tue Aug 07, 2018 10:39 am

Ok, a bit more detail: my goal is to record many images through serial from many cameras. Positive and negative ones. It's already done btw. I was about to record raw images, but you are right, it's far better to save compressed images and decompress them before using them in ML.

I do not know yet if I will be able to train a small enough network for the camera to work with. If the CNN network turns out to be too complex for the cam, I will be using the versatile built-in tools of the OpenMV cam manually (no ML involved), but I would like to backtest my code with the same images so they need to be reloaded into the camera.

I am able to save the compressed images via serial, but reusing them with OpenMV IDE is a partial success. I am converting the compressed image back into bitmap, and loading the image. The image shows up in the IDE, but I have some issues with the colors. :) Blue turns out to be orange for some reason.

The compressed image recorded:
img_cake.jpg
img_cake.jpg (10.8 KiB) Viewed 57 times
The decompressed image: not able to attach in the forum, but generated with the code below.

The image showing up in the IDE:
img_cake_ide.jpg
img_cake_ide.jpg (9.27 KiB) Viewed 57 times
The way I am converting it back to bitmap:

Code: Select all

from PIL import Image

img = Image.open('images/img_cake.jpg')
im_conv = img.convert('RGB')
im_conv.save('images/img_cake.bmp')
User avatar
kwagyeman
Posts: 2116
Joined: Sun May 24, 2015 2:10 pm

Re: Collect ML sample images through serial

Postby kwagyeman » Tue Aug 07, 2018 12:37 pm

Mmm, um, it's likely the BMP byte order being wrong. My BMP code is old and needs to be updated. Just haven't got to it yet. Um, save as a PPM file and you won't have any issues. PPM/PGM are much simpler file formats and we can parse all variants.
Nyamekye,
pvadam
Posts: 9
Joined: Tue May 01, 2018 3:25 am

Re: Collect ML sample images through serial

Postby pvadam » Wed Aug 08, 2018 2:23 am

Perfect, it works! Thank you for the great support!

Return to “Technical Discussion”

Who is online

Users browsing this forum: No registered users and 3 guests