send frames to PC

General discussion about topics related to OpenMV.
mkwoll
Posts: 7
Joined: Wed Jul 12, 2017 12:49 pm

send frames to PC

Postby mkwoll » Wed Jul 12, 2017 1:06 pm

Hi - very impressed with the camera and software. Hope this is a simple question:

I have been doing some image processing using openCV. I bought an M7 camera because I need IR sensitivity, adjustable frame rate and frame size, synchronization with a blinking LED illuminator,etc. But for now I would like to dump the frames to my PC and continue to process them in openCV. (May switch to processing in openMV later.)

What is the most efficient way to send frames over USB/serial to a Python program running on a PC? On the PC I want each monochrome frame to be a simple Numpy integer array. I don't need any compression or a standard video/image format. I think that transmission would be fast enough with ASCII text, although I suppose binary would be faster overall.
I am reasonably familiar with Python but new to Micropython. Maybe I am just not understanding the documentation.

Thanks for any suggestions!

MW
User avatar
kwagyeman
Posts: 2401
Joined: Sun May 24, 2015 2:10 pm

Re: send frames to PC

Postby kwagyeman » Wed Jul 12, 2017 3:14 pm

Hi, we have an openmv.py script in our openmv/usr directory that allows you to get frames as numpy arrays. The script is a little old but it should still work.

If you want the most up to date implementation of the protocol in C++ see OpenMV IDE. The code for the IDE is a lot more complex though since it uses threads and buffers commands and etc.
Nyamekye,
User avatar
kwagyeman
Posts: 2401
Joined: Sun May 24, 2015 2:10 pm

Re: send frames to PC

Postby kwagyeman » Wed Jul 12, 2017 3:16 pm

Note, you'll be able to capture an image to within 1/120th of a frame. The camera itself is a data source that is just constantly spitting out video.
Nyamekye,
mkwoll
Posts: 7
Joined: Wed Jul 12, 2017 12:49 pm

Re: send frames to PC

Postby mkwoll » Wed Jul 12, 2017 8:59 pm

Thanks for the very fast reply! I looked at the openmv.py code. Not sure I understand it all but will give it a try. It looks like the code converts monochrome images (HxW) in the frame buffer to RGB format (HxWx3). I suppose that could be simplified if all I need is an array of monochrome values.

Also, I don't understand what you meant by "within 1/120th of a frame". Please explain.

I greatly appreciate the work you have done on this product.

MW
User avatar
kwagyeman
Posts: 2401
Joined: Sun May 24, 2015 2:10 pm

Re: send frames to PC

Postby kwagyeman » Wed Jul 12, 2017 11:20 pm

So, the camera chip generates a stream of video data. One frame every 1/120th of a second. The MCU captures images in this stream when told to via snapshot(). Thus you are always 1/120th of a frame away in uncertainty at least when you tell it to take a picture. Since that's a small time I'm assuming it should be fine for you.
Nyamekye,
User avatar
iabdalkader
Posts: 595
Joined: Sun May 24, 2015 3:53 pm

Re: send frames to PC

Postby iabdalkader » Fri Jul 14, 2017 7:15 pm

mkwoll wrote:
Wed Jul 12, 2017 8:59 pm
Thanks for the very fast reply! I looked at the openmv.py code. Not sure I understand it all but will give it a try. It looks like the code converts monochrome images (HxW) in the frame buffer to RGB format (HxWx3). I suppose that could be simplified if all I need is an array of monochrome values.

Also, I don't understand what you meant by "within 1/120th of a frame". Please explain.

I greatly appreciate the work you have done on this product.

MW
Note things have changed, the camera now sends JPEG images only (not the best thing for image processing).

@kwabena maybe print image to serial ?
Ronnye
Posts: 3
Joined: Mon Oct 22, 2018 2:48 am

Re: send frames to PC

Postby Ronnye » Mon Oct 22, 2018 2:54 am

Hello,
I'd need to send frames to the PC. I work with Windows 7, where can I find the openmv.py script you suggested?
Thanks in advance.
User avatar
kwagyeman
Posts: 2401
Joined: Sun May 24, 2015 2:10 pm

Re: send frames to PC

Postby kwagyeman » Mon Oct 22, 2018 12:09 pm

Nyamekye,
Ronnye
Posts: 3
Joined: Mon Oct 22, 2018 2:48 am

Re: send frames to PC

Postby Ronnye » Wed Oct 24, 2018 6:06 am

Thank you Nyamekye for your reply.
As preamble, I am programming using Spyder IDE.
I tried to use the pyopenmv.py script you suggested. I first tried it in Windows 7, but I had an error.
Then, I tried it in Ubuntu, but I still had the same error.
The error is the following:

File "/home/ronnye/.local/lib/python2.7/site-packages/spyder_kernels/console/start.py", line 261, in main
locals().pop('__file__')

KeyError: '__file__'

Do you have any ideas to make things work?
Thank you.

Ronnye
Ronnye
Posts: 3
Joined: Mon Oct 22, 2018 2:48 am

Re: send frames to PC

Postby Ronnye » Wed Oct 24, 2018 8:52 am

Sorry, don't consider my previous post, I made a mistake about the script I launched.

Actually my error, in Linux Ubuntu, is the following:


File "/usr/local/lib/python2.7/dist-packages/serial/serialposix.py", line 37, in device
return '/dev/ttyS%d' % port

TypeError: %d format: a number is required, not Serial

My OpenMV serial device seems to be something like
('/dev/ttyACM0', 'ttyACM0', 'USB VID:PID=1209:abd1 SNR=000000000011')

I successfully installed the OpenMV IDE and I was able to run the hello world script, so I can imagine the drivers have been correctly installed.

Ronnye
rister
Posts: 14
Joined: Wed Oct 24, 2018 3:23 pm

Re: send frames to PC

Postby rister » Wed Oct 24, 2018 10:59 pm

I ran into the same error that was previously mentioned by Ronnye. I tried renaming the serial port from /dev/openmvcam to /dev/ttyACM0 but I still ended up not establishing a connection. I had to hard code the port variable to get it to work.

But I am confused about what this script pyopenmv.py actually does and how it works. Is it supposed to allow you to upload scripts to the camera (ie., helloworld.py) from the command line like so: python pyopenmv.py helloworld.py? Can you use this tool to save a frame from this camera to your local computer (this is what I really want to do)?

I did try using this script with the pygame script posted here: https://github.com/openmv/openmv/blob/m ... enmv_fb.py. This works for me but the problem is that I do not want to bog my computer CPU with a stream like this. Just want to save a frame when I need it ... if possible.

thanks
User avatar
kwagyeman
Posts: 2401
Joined: Sun May 24, 2015 2:10 pm

Re: send frames to PC

Postby kwagyeman » Thu Oct 25, 2018 4:36 pm

Hi sorry for not getting back to you about this quickly.

Um, so, if you read the code for:

https://github.com/openmv/openmv/blob/m ... enmv_fb.py -> This script shows off how to use the script below.

You'll see exactly how you can in python control the OpenMV Cam using the:

https://github.com/openmv/openmv/blob/m ... yopenmv.py -> This script provides the methods to pull text / data from the OpenMV Cam and send scripts for it to execute.

Script which provides the library calls to pull text from the camera and such.

That said... not to be mean... but... you're goal is to use the camera through a script you hope to write code around. But, from the questions you've asked you've hinted that you're not really a master at python on a desktop. So, are you able to pull off what you want to do?

Note that while we have the scripts above and that I understand the need for better documentation about interfacing to the camera... I am unable to focus on massively improving this right now as we're focused on OpenMV Cam H7 driver bring up.

Anyway, if you don't want to stream frames just call the fb_dump() method until it returns a frame. Note that the camera may or may not have a frame loaded at a particular time.

Finally, note that you don't need to use any of these scripts. These scripts show off how the OpenMV Cam's debug protocol works. You can however roll you own serial protocol in python using the VCP class.
Nyamekye,
rister
Posts: 14
Joined: Wed Oct 24, 2018 3:23 pm

Re: send frames to PC

Postby rister » Thu Oct 25, 2018 10:42 pm

Not to say that I am a master of python, but I do spend most of my day python coding in linux, macs and windows systems and have been doing that for many years. But you are right I am scratching my head on the documentation at the moment sorry but I want to understand how to make this camera useful. I see you want to increase sales to make a better product but helping your current customers is important too.

So I get it just take the out of the while loop then you should be able to call the script from command line to grab a frame. But what about script variable in pyopenmv_fb.py I guess this gets uploaded to the camera ... correct? So if I want to run some image processing algorithm and save some data in a file or in sometype of variable how can I put that into the pygame world? Does this make sense. Its great that I can download the image but now I need to get the some statistical data. Can you show me how to do that?
User avatar
iabdalkader
Posts: 595
Joined: Sun May 24, 2015 3:53 pm

Re: send frames to PC

Postby iabdalkader » Fri Oct 26, 2018 7:13 am

Ronnye wrote:
Wed Oct 24, 2018 8:52 am
Actually my error, in Linux Ubuntu, is the following:

File "/usr/local/lib/python2.7/dist-packages/serial/serialposix.py", line 37, in device
return '/dev/ttyS%d' % port

TypeError: %d format: a number is required, not Serial
Can you post the full traceback ?
User avatar
iabdalkader
Posts: 595
Joined: Sun May 24, 2015 3:53 pm

Re: send frames to PC

Postby iabdalkader » Mon Oct 29, 2018 8:15 am

Hi,

I just added a new VCP example, it shows how to send images with USB VCP on demand (host side script examples in the comments)

https://github.com/openmv/openmv/blob/m ... usb_vcp.py

Re the error, I think newer pyserial don't accept port numbers any more, will look into it.
bogflap
Posts: 35
Joined: Mon Apr 16, 2018 10:03 am

Re: send frames to PC

Postby bogflap » Mon Oct 29, 2018 10:27 am

Just to clarify. If on line 34 of the script you reference the ".compress" is omitted then an uncompressed image will be sent?
User avatar
iabdalkader
Posts: 595
Joined: Sun May 24, 2015 3:53 pm

Re: send frames to PC

Postby iabdalkader » Mon Oct 29, 2018 10:37 am

bogflap wrote:
Mon Oct 29, 2018 10:27 am
Just to clarify. If on line 34 of the script you reference the ".compress" is omitted then an uncompressed image will be sent?
Yes.
User avatar
kwagyeman
Posts: 2401
Joined: Sun May 24, 2015 2:10 pm

Re: send frames to PC

Postby kwagyeman » Mon Oct 29, 2018 11:57 am

Uncompressed images are large. Note the FPS will drop considerably.
Nyamekye,

Return to “General Topics”

Who is online

Users browsing this forum: No registered users and 4 guests