Controlling OpenMV 7 camera pins during remote image streaming to pc

Hello,

I am using the OpenMV camera to stream jpg images to my computer.

I would like to be able to change some GPIO pin states while the stream buffer is on and streaming to the pc.

For example, I would like to switch GPIO pins connected to relays of different lighting sources while streaming the images (to get better lighting, different viewpoints, etc.).

I attempted to pass it as an additional str element in the following function call that passes data to the camera’s jpeg_image_stream function,
result = interface.call(“jpeg_image_stream”, “sensor.GRAYSCALE,sensor.QVGA,”+str(controlGPIO))
but it does not seem to work.

Do you support this feature or have code that could be used to control GPIOs while streaming from the camera?

Thank you in advance.

Hi, you have to edit the script on the PC side and on the camera side. This is fully controllably by you. You can literally do whatever you like.

Yes, I see.

I was just asking if there was any example using the RPC library to send commands from the computer side to the camera in order to change the camera’s pin status?

Hi, what you did is fine. You just need to edit the camera side code to receive that additional argument. If you look at the camera code you’ll notice that it receives the string. Splits it into two items, and the exec()s the two strings in order to set the frame size and pixformat. So, you just need to extend the camera side code to do what you want to do.

There isn’t an example for this… but, if you know how to change the camera pin state regularly then this should be very straight forward.

Thank you for the prompt responses!

Yes, I have modified the camera code exactly as you said. I split the data into 3 items right now(since I have added one more).

Then use the info to trigger the pin states, but my code is not working. I guess I am missing something else.

Couldn’t the same functionality be performed by adding a new callback?

Just trying to think of alternative ways to do it.

Add a print() statement to the callbacks. You can print(“a string”) in them without side affects.

Tried the print() statements too. I am still not sure what the issue might be.

On the PC side, when I call the following 2 lines of code the pygame window does not open.
interface.call(“relay_control”, “sensor.GRAYSCALE,sensor.QVGA,”+str(relayFlag))
interface.stream_reader(jpg_frame_buffer_cb, queue_depth=8)

If I remove the interface.call to the relay_control callback, and restart the camera the code works. Do you have any idea what might be causing this behavior? Thanks!

The relay_control callback code (on the camera side) is the following and just flips the pin values based on their current status:
def relay_control(data):
global relayFlag, pin7, pin8
# pixformat, framesize, temp = bytes(data).decode().split(",")
#
# sensor.set_pixformat(eval(pixformat))
# sensor.set_framesize(eval(framesize))
# relayFlag += eval(temp)

print("The relay will be flipped...")

if pin7.value() == 1:
    pin7.value(0)
    pin8.value(1)
else:
    pin7.value(1)
    pin8.value(0)

print("The relay has been set.")
return(0)

Can you post your code on the PC and camera side and use the code formatting option. It’s hard to tell what you are doing from here.

Thank you for looking into it.

I have attached the code from the camera and the PC. Essentially what I would like to do is, while the jpeg streaming is happening, to be able to trigger the light relays on/off on the camera side.

I have tried calling the “relay_control” callback (in the while loop) both before and after the “jpeg_image_stream”, but once the “relay_control” call is performed the streaming stops and the camera needs to be restarted to send image frames against.

If I comment out the “relay_control” call the code works fine and the camera streams normally.

It would be great if you could help!

Thanks in advance.

main.py (4.8 KB)
rpc_image_transfer_jpg_streaming_as_the_controller_device.py (3.6 KB)

Ah, I see…

Um, don’t put a delay in the stream_generator_cb() method. As mentioned in the RPC library documentation the stream_generator_cb() is a streaming method. There’s only one-way communication in this mode.

Mmm, so, stream mode is just one way. The camera to the PC. If you want to toggle the relay then you have to abort stream mode, issue a command, and then resetup stream mode.

Here’s what you should do. Start with the default code first. Add another callback like you did to the MicroPython code to toggle the relay.

Then the main script should call that to toggle to the relay. Then it should setup the stream. If the user changes the relay then you want to generate an exception in the main script stream reader method: openmv/tools/rpc at master · openmv/openmv · GitHub to cause the stream to shutdown. Then you can toggle the relay via a regular RPC command, then setup stream mode again.

Thank you for the reply.

Is there any chance to provide us with the necessary additions to the code (e.g., the exception handling part)? How could the frame buffer get started again? Does it happen in the while(true) block of the controller device (PC)?

I believe it would be very useful to have an example where:

  1. The stream buffer is initiated and we get jpeg streaming.
  2. Then it is stopped, some operations are performed, e.g. we might want to change the camera pixformat, set some GPIOs to control another device, etc.
  3. After that we restart the stream buffer and take images based on the new setup and the changes performed in the previous step.

How would the code on the controller device (PC) look like?

Thank you again

Hi, I am fully throttled with work. I cannot provide this level of help.

As mentioned in my comment above… you can read the RPC python code. You can see directly what you need to do… it’s not a hidden library.

As mentioned in my comment, just raise an exception and then catch that exception in a try, except block around the start streaming callback. After you do that the other side will timeout in 1 second. Then you can try the normal RPC call again.