Wifi Streaming of RGB565 Images

OpenMV related project discussion.
Jhc999
Posts: 6
Joined: Fri Nov 06, 2020 4:31 pm

Wifi Streaming of RGB565 Images

Postby Jhc999 » Fri Nov 06, 2020 4:46 pm

For my project, I need to stream RGB565 images (160x120, QQVGA) using the wifi shield. I've looked at the mjpegstreamer and rtsp_video_server examples. I've also looked at the mqtt and http examples. However, my project requires uncompressed RGB565 images.

I've written client/socket code for the OpenMV. It works fine when I set the sensor format to JPEG. Here are the relevant snippets.

OpenMV Client

Code: Select all

...
sensor.set_pixformat(sensor.JPEG)
... 
s = usocket.socket(usocket.AF_INET, usocket.SOCK_STREAM)
s.connect((HOST, PORT))
s.settimeout(2.0)  
...
frame = sensor.snapshot()
frame = frame.bytearray()
sent = s.write(frame)
... 
PC Server

Code: Select all

import io
from PIL import Image, ImageFile
...
while True:
    data = conn.recv(4000)
    if not data:
        break
    if msglen < imLen:
        msglen += len(data)       
        img += data
    if msglen >= imLen:
        break
image = Image.open(io.BytesIO(img))
...
The above code streams well when I set the format to JPEG. However, when I change the format to RGB565 using sensor.set_pixformat(sensor.RGB565), the bytearray can no longer be decoded.
OSError: cannot identify image file <_io.BytesIO object at 0x10ce322f0>
Any help is appreciated! If there is a better way to stream RGB565 images over wifi, please let me know!
Jhc999
Posts: 6
Joined: Fri Nov 06, 2020 4:31 pm

Re: Wifi Streaming of RGB565 Images

Postby Jhc999 » Fri Nov 06, 2020 5:01 pm

The Python struct module is probably a more efficient way of packing and unpacking the byte data. However, my main problem is decoding the RGB565 image after it is unpacked. I'm unable to convert it back to an RGB image on my PC.
User avatar
kwagyeman
Posts: 4658
Joined: Sun May 24, 2015 2:10 pm

Re: Wifi Streaming of RGB565 Images

Postby kwagyeman » Fri Nov 06, 2020 5:02 pm

Um, I would just send a JPG at a really high quality and then convert back into an RGB565 image. There's no issues sending an RGB565 image, but, it's more data than normal. Any reason you need it raw?
Nyamekye,
User avatar
kwagyeman
Posts: 4658
Joined: Sun May 24, 2015 2:10 pm

Re: Wifi Streaming of RGB565 Images

Postby kwagyeman » Fri Nov 06, 2020 5:02 pm

Yeah, um, I don't think PC libraries easily can decode RGB565. JPG transfer is kinda the only universal way.
Nyamekye,
Jhc999
Posts: 6
Joined: Fri Nov 06, 2020 4:31 pm

Re: Wifi Streaming of RGB565 Images

Postby Jhc999 » Fri Nov 06, 2020 5:13 pm

The project is contactless vital signs monitoring inside a hospital. The current literature provides methods for calculating heart rate, respiratory rate, etc. but it requires uncompressed RGB pixels over 10 seconds to make accurate calculations. If possible, the stream needs to be with full RGB images.

On the subject of JPEG compression, is it possible to have JPEG compression at the highest quality of 100? In the helloworld example, I can only set the quality to 98.

Code: Select all

sensor.set_pixformat(sensor.RGB565) # Set pixel format to RGB565 (or GRAYSCALE)
#sensor.set_quality(98)
At 99 and 100 the images glitch and the camera disconnects.
User avatar
kwagyeman
Posts: 4658
Joined: Sun May 24, 2015 2:10 pm

Re: Wifi Streaming of RGB565 Images

Postby kwagyeman » Fri Nov 06, 2020 7:01 pm

That's probably a bug if you can't set 100. That said, you won't be able to see a difference above 90.

To note, all that code is doing is setting: https://github.com/openmv/openmv/blob/m ... 640.c#L740
Nyamekye,
RickBullotta
Posts: 3
Joined: Thu Nov 12, 2020 2:12 pm

Re: Wifi Streaming of RGB565 Images

Postby RickBullotta » Thu Nov 12, 2020 3:24 pm

Does this help? Just uses PIL and Numpy. You could possibly use this on the consumer side to decode it to visualize it.

https://github.com/humbertokramm/RGB565 ... G_-python-

Return to “Project Discussion”

Who is online

Users browsing this forum: No registered users and 0 guests