I am trying to start and stop streaming images from a portenta h7 with camera shield. I used the mpeg streamer example provided with openmv ide. First time I start the streaming works fine, than when I am trying again camera freezes. I tried unsuccessfully to close the server socket and open it again. Below is the code I am using. Any help would be appreciated.
from pyb import UART
import sensor, image, time, network, socket, sys
SSID='some_net' # Network SSID
KEY='some_pass' # Network key
HOST ='' # Use first available interface
PORT = 8080 # Arbitrary non-privileged port
# init sensor
sensor.reset()
sensor.set_framesize(sensor.QVGA)
sensor.set_pixformat(sensor.GRAYSCALE)
# initialize com port
uart = UART(1, 19200)
# init wlan module and connect to network
wlan = network.WLAN(network.STA_IF)
wlan.deinit()
wlan.active(True)
wlan.connect(SSID, KEY, timeout=30000)
# get the cam IP
cam_ip = wlan.ifconfig()[0]
# confirm that vision subsytem is up
buf = bytes([0xAB,0xFD])
uart.write(buf)
# create server socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, True)
# bind and listen
s.bind([HOST, PORT])
s.listen(1)
# set server socket to blocking
s.setblocking(True)
def stream_video(s):
client, addr = s.accept()
# set client socket timeout to 5s
client.settimeout(5.0)
# read request from client
data = client.recv(1024)
# send multipart header
client.sendall("HTTP/1.1 200 OK\r\n" \
"Server: OpenMV\r\n" \
"Content-Type: multipart/x-mixed-replace;boundary=openmv\r\n" \
"Cache-Control: no-cache\r\n" \
"Pragma: no-cache\r\n\r\n")
# start streaming images
while (True):
frame = sensor.snapshot()
cframe = frame.compressed(quality=35)
header = "\r\n--openmv\r\n" \
"Content-Type: image/jpeg\r\n"\
"Content-Length:"+str(cframe.size())+"\r\n\r\n"
client.sendall(header)
client.sendall(cframe)
def process_request(cmd,s):
if cmd == "cmd00": # streamer
# send IP address of the cam
ip_bytes = bytes(map(int, cam_ip.split('.')))
uart.write(ip_bytes)
time.sleep_ms(500)
# start streaming
try:
stream_video(s)
except OSError as e:
pass
while(True):
while (uart.any() < 5):
time.sleep_ms(200)
cmd_bytes = bytearray(5)
uart.readinto(cmd_bytes)
cmd = cmd_bytes.decode('utf-8')
process_request(cmd,s)
How do you stop/start the streaming ? The code above only runs in a loop. Please post an example that shows the actual issue, and use code formatting.
Initially I was using inside stream_video function a 10 sec chunk of time and then stop. It did not make any difference.
# start streaming images
start = time.ticks_ms()
while (time.ticks_diff(time.ticks_ms() - start) < 10000):
frame = sensor.snapshot()
cframe = frame.compressed(quality=35)
header = "\r\n--openmv\r\n" \
"Content-Type: image/jpeg\r\n"\
"Content-Length:"+str(cframe.size())+"\r\n\r\n"
client.sendall(header)
client.sendall(cframe)
The streaming is triggered from a browser. When a button is pushed, an ajax call is made to a web server and then the cmd00 is sent through a chain of microcontrollers to the camera UART1 port. Then the IP of the server socket is sent back through the same chain to the awaiting ajax call in the browser
$.ajax({
url: '/cgi-bin/proc_cmd.py',
dataType: 'JSON',
type: 'POST',
contentType: 'application/json',
data: dataJSON,
processData: false,
success: function(result) {
const obj = JSON.parse(JSON.stringify(result));
if (obj.status == "OK") {
if (cmd == "stream_video") {
// alert(obj.streamer_ip);
$('#sec_div').show();
$('#sec_div').html('<img src=' + '"http://' + obj.streamer_ip +
':8080" width="320" height="320">' );
// wait 10 sec for video stream to end
setTimeout(() => {
$('#sec_div').html("");
$('#sec_div').hide();
}, 10000);
}
}
},
error: function( jqXhr, textStatus, errorThrown ){
console.log( errorThrown );
}
});
The IP returned from ajax call is used then as src for an image tag. After 10 sec the image tag is removed. I thought that once there is not socket on the browser side an exception is triggered in the camera stream_video function and stop the while(True) streaming loop.
You’re using ticks_diff wrong it takes two args:
while (time.ticks_diff(time.ticks_ms(), start) < 10000):
clock.tick() # Track elapsed milliseconds between snapshots().
frame = sensor.snapshot()
cframe = frame.compressed(quality=35)
....
client.close()
You should probably also close the client when the loop exists… I tested the example with just these two changes and I can reconnect fine after 10 seconds, I just refresh the browser and it reconnects.
The browser interface looks like
When the button stream_video is pressed then the ajax call is make. After the first press is looking like
then after the second call it freezes
Thank you, I did not see that in in the ticks_diff. I will correct it and try it again. I will try to close the client too.
I tried the two things, it is better now. The second time I get a static image. The problem may have to do with my image tag. After I display the 10 sec stream I remove the image tag from my div. That may not be enough to close the socket on the browser side.
$('#sec_div').html('<img src=' + '"http://' + obj.streamer_ip +
':8080" width="320" height="320">' );
$('#sec_div').html("");
Yes it looks like something with your Ajax code, in the browser I get a static image too but I hit refresh (this probably opens a new socket) and it starts streaming again.
The problem was the image caching. I appended a unique tag at the end of the address based on current time. That seemed to solve the problem. Now everything works as expected.
Thank you very much for your help.
var timestamp = new Date().getTime();
var img = document.createElement("img");
img_address = "http://" + obj.streamer_ip + ":8080" + "?t=" + timestamp;
img.setAttribute('src', img_address);
img.setAttribute('height',320);
img.setAttribute('width',320);
img.setAttribute('id','img_stream');
document.getElementById('sec_div').appendChild(img);
Most browsers I tested support these:
"Cache-Control: no-cache\r\n" \
"Pragma: no-cache\r\n\r\n")
