I have a question about my project and I hope someone has some experience on this.
I have put the OpenMV cam + WiFi shield on a Elecfreaks CuteBot (powered by a Micro:bit) in order to make a remote controlled robot (with autopilot based on images). My goal is first to control the robot via my browser while seeing everything it does. That requires the board to do two things:
- Show video over the network
- Accept commands from the network
- Conttinuously control the robot
My first attempt was creating a HTTP server that would have a GET command for getting images and a POST for commands but this was not satisfactory as the HTTP server would need to (blocking) listen on a port for connections and hence couldn’t do anything else in between. This meant that (3) was not possible (and also HTTP was not always reliable).
So I decided MQTT might be a better choice here as the publish/subscribe model seems to be make more sense. However, I run into performance/reliability problems and I’m not sure if what I want is out of the ordinary or if I’m doing something wrong.
My setup is now: MQTT server (mosquitto) hosted locally, my laptop with webapp connecting via websocket to mqtt server, openMV that has two connections to the server via the usual port. I have pasted some of the code below.
First, the reason I am using two separate client connections is that I use one for sending images and one for receiving commands. I tried doing it over one connection but I quickly ran into problems that check_msg would fail when sending multiple (large) messages. Having two connections seems to fix the problem.
The problem I run into now is that the cam misses messages sent by the browser (client2.check_msg) if I send messages too quickly from the cam (client.publish). I have used a loop variable to artificially tone down on the FPS that is sent to the browser which seems to work around the problem a bit. If I disable the sending of image (cam = False) everything works perfectly.
The images that are sent are only about 12kb in size and I get a decent framerate, meaning that the check_msg should be called on a very regular basis.
What should I do to minimise the chance of missing commands while maximizing the FPS sent to the browser?
from mqtt import MQTTClient <sensor setup> client = MQTTClient("openmv-publish", <server>, port=1883) client2 = MQTTClient("openmv-listen", <server>, port=1883) client.connect() client2.connect() def callback(topic, msg): global command command = msg.decode('utf-8') print(topic, msg.decode('utf-8')); # must set callback first client2.set_callback(callback) client2.subscribe("commands") loop = 0 while True: try: loop = loop + 1 # Check for messages client2.check_msg() # Handle command if command: # handle commands i got # Do video stuff objects, frame = do_video() if camon and loop % 5 == 0: msg = ubinascii.b2a_base64(frame.compress(quality=35)).decode('utf-8').strip() client.publish("videostream", msg); except Exception as e: print(e) raise