send frames to PC

OpenMV Products
#1

Hi - very impressed with the camera and software. Hope this is a simple question:

I have been doing some image processing using openCV. I bought an M7 camera because I need IR sensitivity, adjustable frame rate and frame size, synchronization with a blinking LED illuminator,etc. But for now I would like to dump the frames to my PC and continue to process them in openCV. (May switch to processing in openMV later.)

What is the most efficient way to send frames over USB/serial to a Python program running on a PC? On the PC I want each monochrome frame to be a simple Numpy integer array. I don’t need any compression or a standard video/image format. I think that transmission would be fast enough with ASCII text, although I suppose binary would be faster overall.
I am reasonably familiar with Python but new to Micropython. Maybe I am just not understanding the documentation.

Thanks for any suggestions!

MW

#2

Hi, we have an openmv.py script in our openmv/usr directory that allows you to get frames as numpy arrays. The script is a little old but it should still work.

If you want the most up to date implementation of the protocol in C++ see OpenMV IDE. The code for the IDE is a lot more complex though since it uses threads and buffers commands and etc.

#3

Note, you’ll be able to capture an image to within 1/120th of a frame. The camera itself is a data source that is just constantly spitting out video.

#4

Thanks for the very fast reply! I looked at the openmv.py code. Not sure I understand it all but will give it a try. It looks like the code converts monochrome images (HxW) in the frame buffer to RGB format (HxWx3). I suppose that could be simplified if all I need is an array of monochrome values.

Also, I don’t understand what you meant by “within 1/120th of a frame”. Please explain.

I greatly appreciate the work you have done on this product.

MW

#5

So, the camera chip generates a stream of video data. One frame every 1/120th of a second. The MCU captures images in this stream when told to via snapshot(). Thus you are always 1/120th of a frame away in uncertainty at least when you tell it to take a picture. Since that’s a small time I’m assuming it should be fine for you.

#6

Note things have changed, the camera now sends JPEG images only (not the best thing for image processing).

@kwabena maybe print image to serial ?

#7

Hello,
I’d need to send frames to the PC. I work with Windows 7, where can I find the openmv.py script you suggested?
Thanks in advance.

#8
#9

Thank you Nyamekye for your reply.
As preamble, I am programming using Spyder IDE.
I tried to use the pyopenmv.py script you suggested. I first tried it in Windows 7, but I had an error.
Then, I tried it in Ubuntu, but I still had the same error.
The error is the following:

File “/home/ronnye/.local/lib/python2.7/site-packages/spyder_kernels/console/start.py”, line 261, in main
locals().pop(‘file’)

KeyError: ‘file’

Do you have any ideas to make things work?
Thank you.

Ronnye

#10

Sorry, don’t consider my previous post, I made a mistake about the script I launched.

Actually my error, in Linux Ubuntu, is the following:


File “/usr/local/lib/python2.7/dist-packages/serial/serialposix.py”, line 37, in device
return ‘/dev/ttyS%d’ % port

TypeError: %d format: a number is required, not Serial

My OpenMV serial device seems to be something like
(’/dev/ttyACM0’, ‘ttyACM0’, ‘USB VID:PID=1209:abd1 SNR=000000000011’)

I successfully installed the OpenMV IDE and I was able to run the hello world script, so I can imagine the drivers have been correctly installed.

Ronnye

#11

I ran into the same error that was previously mentioned by Ronnye. I tried renaming the serial port from /dev/openmvcam to /dev/ttyACM0 but I still ended up not establishing a connection. I had to hard code the port variable to get it to work.

But I am confused about what this script pyopenmv.py actually does and how it works. Is it supposed to allow you to upload scripts to the camera (ie., helloworld.py) from the command line like so: python pyopenmv.py helloworld.py? Can you use this tool to save a frame from this camera to your local computer (this is what I really want to do)?

I did try using this script with the pygame script posted here: openmv/pyopenmv_fb.py at master · openmv/openmv · GitHub. This works for me but the problem is that I do not want to bog my computer CPU with a stream like this. Just want to save a frame when I need it … if possible.

thanks

#12

Hi sorry for not getting back to you about this quickly.

Um, so, if you read the code for:

openmv/pyopenmv_fb.py at master · openmv/openmv · GitHub → This script shows off how to use the script below.

You’ll see exactly how you can in python control the OpenMV Cam using the:

openmv/pyopenmv.py at master · openmv/openmv · GitHub → This script provides the methods to pull text / data from the OpenMV Cam and send scripts for it to execute.

Script which provides the library calls to pull text from the camera and such.

That said… not to be mean… but… you’re goal is to use the camera through a script you hope to write code around. But, from the questions you’ve asked you’ve hinted that you’re not really a master at python on a desktop. So, are you able to pull off what you want to do?

Note that while we have the scripts above and that I understand the need for better documentation about interfacing to the camera… I am unable to focus on massively improving this right now as we’re focused on OpenMV Cam H7 driver bring up.

Anyway, if you don’t want to stream frames just call the fb_dump() method until it returns a frame. Note that the camera may or may not have a frame loaded at a particular time.

Finally, note that you don’t need to use any of these scripts. These scripts show off how the OpenMV Cam’s debug protocol works. You can however roll you own serial protocol in python using the VCP class.

#13

Not to say that I am a master of python, but I do spend most of my day python coding in linux, macs and windows systems and have been doing that for many years. But you are right I am scratching my head on the documentation at the moment sorry but I want to understand how to make this camera useful. I see you want to increase sales to make a better product but helping your current customers is important too.

So I get it just take the out of the while loop then you should be able to call the script from command line to grab a frame. But what about script variable in pyopenmv_fb.py I guess this gets uploaded to the camera … correct? So if I want to run some image processing algorithm and save some data in a file or in sometype of variable how can I put that into the pygame world? Does this make sense. Its great that I can download the image but now I need to get the some statistical data. Can you show me how to do that?

#14

Can you post the full traceback ?

#15

Hi,

I just added a new VCP example, it shows how to send images with USB VCP on demand (host side script examples in the comments)

https://github.com/openmv/openmv/blob/master/scripts/examples/02-Board-Control/usb_vcp.py

Re the error, I think newer pyserial don’t accept port numbers any more, will look into it.

#16

Just to clarify. If on line 34 of the script you reference the “.compress” is omitted then an uncompressed image will be sent?

#17

Yes.

#18

Uncompressed images are large. Note the FPS will drop considerably.