I am trying to use RPC image streaming along with some other functionality.
I saw RPC has setup_loop_callback that appears to be what I need to schedule some other actions, but while trying to test it, I notice the recv_timeout/send_timeout parameters of the loop function are used only when streaming is not ongoing, otherwise, the callback is never called. I verified this by setting a led toggle call in the setup_loop_callback’s function.
Is it possible to do other actions while RCP is streaming, using RPC’s callbacks, or do I need to setup a timer for this purpose?
The code needs to be re-written with uasyncio. However, this is not a priority.
If you’d like to modify the loop, you can just call the functions the loop is calling directly and modify its behavior. You do not have to use the function.
Just inherit from the RPC interface object you are using, and then you can customize the loop behavior.
The led indication was misleading. I thought it was stuck, but it was actually blinking very fast. For some reason, when trying to send a processed image, instead of snapshot, I can only see the first processed image, which lead me to think also that loop is not doing it’s job.
Image processing did worked in a simple loop within IDE, without any freeze. I am still looking what the real problem is…
I am getting an exception while trying to transfer processed image that needs to send further thru rpc.
The exception is “OSError, Images not equal!”, and this happen in my processing class. First iteration I am able to get the image ok, then, on second iteration, this exception appears and to my end it appears that image is frozen.
If I reduce the image resolution from QVGA to QQVGA, the exception moves from my image processing to my caller function, a bit later on, but the exception is the same.
Do I have a memory management problem?
[LE] Actually the exception is Expected a mutable image
I dunno, maybe print() on the image and see if it’s what you expect?
I don’t really have an idea of what you are doing exactly now…
Can I use print with rcp over VCP? I got my exception thru a log file to the storage, but it is usually breaking the storage space and sometimes I have dirs/files lost, other time corrupted files or even file system, while unplug the USB and active logging.
[LE] add some code
- path Image/motion.py
import image, sensor
class Motion:
def __init__(self, size, image):
#keep input params
self.size = size
self.image = image
def ProcessNewImage(self, image):
#keep input params
self.image = image
self.image.to_grayscale()
self.image.to_rgb565()
def GetImage(self):
return self.image
- path Image/init.py
__all__ = ["Motion"]
- path main.py
import sensor, image, omv, rpc, micropython
from Image.motion import Motion
from pyb import LED
logFilename = 'log.txt'
#logFilename = None
red_led = LED(1)
sensor.reset()
sensor.set_pixformat(sensor.RGB565)
sensor.set_framesize(sensor.QVGA)
sensor.skip_frames(time = 2000)
omv.disable_fb(True)
motion = Motion((sensor.width(), sensor.height()), sensor.snapshot())
waitStream = False
def logInfo(info):
try:
if None == logFilename:
#print(info)
return
logFile = open(logFilename, 'a')
logFile.write(info)
logFile.write('\r\n')
logFile.close()
except:
pass
def stream_generator_cb():
keepImg.to_jpeg(quality=90)
return keepImg.bytearray()
def jpeg_image_stream_cb():
global waitStream
logInfo('jpeg_image_stream_cb')
if not waitStream:
return
logInfo('jpeg_image_stream_cb streaming...')
interface.stream_writer(stream_generator_cb)
waitStream = False
logInfo('jpeg_image_stream_cb waitStream reset.')
def jpeg_image_stream(data):
pixformat, framesize = bytes(data).decode().split(",")
sensor.set_pixformat(eval(pixformat))
sensor.set_framesize(eval(framesize))
interface.schedule_callback(jpeg_image_stream_cb)
return bytes()
def schedule_app_loop(_):
app_loop()
def schedule_app_loop_no_arg():
micropython.schedule(schedule_app_loop_ref, 0)
def app_loop():
global waitStream, motion
red_led.toggle()
logInfo('app_loop')
if waitStream:
return
logInfo('app_loop processing...')
try:
motion.ProcessNewImage(sensor.snapshot())
keepImg.replace(motion.GetImage())
except OSError as osError:
info = 'app_loop OSError during processing: '
info += str(osError.errno)
logInfo(info)
except Exception as error:
info = 'app_loop exception during processing: '
info += type(error).__name__
info += '–'
info += str(error)
logInfo(info)
waitStream = True
logInfo('app_loop waitStream set.')
keepImg = sensor.alloc_extra_fb(sensor.width(), sensor.height(), sensor.RGB565)
#keepImg[:] = (0)
#keepImg.replace(sensor.snapshot())
schedule_app_loop_ref = schedule_app_loop
interface = rpc.rpc_usb_vcp_slave()
interface.register_callback(jpeg_image_stream)
interface.setup_loop_callback(schedule_app_loop_no_arg)
interface.loop(recv_timeout=300, send_timeout=300)
This is part of the log file I get:
app_loop
app_loop
app_loop
jpeg_image_stream_cb
jpeg_image_stream_cb streaming...
jpeg_image_stream_cb waitStream reset.
app_loop
app_loop processing...
app_loop exception during processing: ValueError–Expected a mutable image
app_loop waitStream set.
app_loop
app_loop
app_loop
Running it from my host machine with following test.py, the image is ok, no freezed image, but ir is freezing after first frame with main.py
import sensor, image, time
from Image.motion import Motion
sensor.reset()
sensor.set_pixformat(sensor.RGB565)
sensor.set_framesize(sensor.QVGA)
sensor.skip_frames(time = 2000)
clock = time.clock()
motion = Motion((sensor.width(), sensor.height()), sensor.snapshot())
keepImg = sensor.alloc_extra_fb(sensor.width(), sensor.height(), sensor.RGB565)
while(True):
clock.tick()
img = sensor.snapshot()
motion.ProcessNewImage(img)
keepImg.replace(motion.GetImage())
img.replace(keepImg)
print(clock.fps())
Yeah, I would recommend fully debugging your application first before adding the RPC library via the IDE. You cannot use it and print at the same time.
Note that you can use the RPC library via the UART pins on the OpenMV Cam and a USB to serial converter. This gives you a sideband channel to have all the code working and debug it.
Since there’s only one VCP port for the camera it can’t be used for debug and the RPC lib at the same time. Note, you may wish to use our debug protocol instead to control the camera from the PC:
openmv/tools/pyopenmv.py at master · openmv/openmv (github.com)
Given all the people having issues with trying to use the RPC library via USB I’m thinking it may be best to delete that interface. I just exposed it because I could at the time, but, it’s not necessary the best way to do things.
I already tested the “Motion” class, which is a dummy grayscale conversion, in my example above, with a test file, test.py from above, which does not involve rpc, and the “video” has no issues.
Anyway, we decided to switch from Raspberry PI to a Nuc, so we can use IDE remotely, to observe outdoor conditions for our functionality, instead of using RPC.
The reason for using RPC with VCP was that we didn’t have to have additional wires, since the USB connection that provide power was already there, beside VCP connection is faster than serial for image capture.
Okay, if you just need data out of the camera just use our debug protocol that the pyopenmv.py script exposes. It’s the same as the IDE. You can remotely execute scripts on the camera via it and pull the frame buffer/text.