RTSP in Nicla Vision

sata · January 20, 2023, 8:20am

Will nicla vision be able to run the rtsp example from openmv?

I see that the device is able to connect to my router and I’m able to ping the ip, but cannot get it to start streaming in VLC

I also changed the pixel format to rgb565 but I was expecting to fail at a later stage due to encoding issues? but the rtsp stream function loop seems to be stuck in checking

github.com

openmv/openmv/blob/master/scripts/libraries/rtsp.py#L214


      
                          img_data_len = len(img_data)
                          data_len = self.__send(rtp_header + jpeg_header + img_data)
                          if not data_len: break
                          i += img_data_len
                          l -= img_data_len
                      if l: self.__close_socket()
                  except OSError: self.__close_socket()
          
          
def stream(self, image_callback, quality=90): # public
              while True:
                  if self.__valid_tcp_socket():
                      try:
                          self.__tcp__socket.settimeout(0.1)
                          while True:
                              data = self.__tcp__socket.recv(1400)
                              if data and len(data): self.__parse_rtsp_request(data)
                              if self.__playing: self.__send_rtp(image_callback, quality)
                      except OSError: self.__close_tcp_socket()

The ip that nicla is printing is something like 192.168.xx.xx:554
Trying to stream in vlc using rtsp://192.168.xx.xx:554
But it doesn’t seem to be working, I also tried with different wifi routers and mobile hotspots

Did anyone here try streaming rtsp in nicla vision?

Did anyone manage to successfully stream feed via rtsp?

Any help would be much appreciated. Thanks!

kwagyeman · January 23, 2023, 1:02am

Hi, it won’t work on that camera since it only seems to work with JPEGs from the OV5640 camera module. However, you appear to have a different error. Not sure why.

sata · January 23, 2023, 6:42am

Right? Thank you for the reply, will see if I can provide more info on this.

kwagyeman · January 23, 2023, 4:56pm

A wire shark trace helps of the IP communication.

sata · February 8, 2023, 8:52am

Thank you for the suggestion @kwagyeman , I tried tracing it using wire shark and the request is going from vlc to the nicla vision board.

I edited the rtsp module code and added a few debugging statements.

It is throwing an out of memory error, code 12 on this website:
https://mpython.readthedocs.io/en/master/library/pythonStd/uerrno.html

So I guess, nicla vision just doesn’t have enough space on board to handle it?

kwagyeman · February 8, 2023, 4:53pm

Hi, which line of code is it throwing the error on?

sata · February 9, 2023, 8:29am

Hi, thank you for the reply.

github.com

openmv/openmv/blob/master/scripts/libraries/rtsp.py#L21


      
          
          
def __valid_tcp_socket(self): # private
              if self.__tcp__socket is None:
                  try:
                      s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
                      s.bind(self.__myaddr)
                      s.listen(0)
                      s.settimeout(5)
                      self.__tcp__socket, self.__client_addr = s.accept()
                      s.close()
                  except OSError: self.__tcp__socket = None
              return self.__tcp__socket is not None
          
          
def __close_tcp_socket(self): # private
              self.__tcp__socket.close()
              self.__tcp__socket = None
          
          
def __valid_udp_socket(self): # private
              if self.__udp__socket is None:
                  try:
                      self.__udp__socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)

This is where it fails, I printed the exception here and it fails multiple times before finally showing os error 12

I edit the code a bit to see what exactly vlc sends to the board. These are the commands that I found

DESCRIBE rtsp://192.168.105.221:554 RTSP/1.0 CSeq: 3
User-Agent: LibVLC/3.0.16 (LIVE555 Streaming Media v2016.11.28)
Accept: application/sdp

This works with out any error and the device sent:

3 Content-Type: application/sdp
Content-Length: 98

After this, I got OSError(98,) at the checkpoint, which is Address already in use according to uerrno – System Error Code — mPython board 2.2.2 documentation

This repeated for about 5 times before I either get a OSError(12,) or sometimes I get a connection from VLC again

VLC Sent:

SETUP rtsp://192.168.105.221:554 RTSP/1.0 
CSeq: 0
Transport: RTP/AVP;unicast;client_port=9256-9257

This also worked I guess, not sure about it though

The device continued to raise OSError 12 memory error at this point
I printed the free memory using gc.mem_free() and it shows 159808, which I hope is sufficient?

It continued for some time before the VLC raised the error that it’s failing to connect with the message:
VLC is unable to open the MRL ‘rtsp://192.168.105.221:554’ Check the log for details.

My guess is the socket continues to be in use even after s.close() ?, not sure about it though.

Any help would be much appreciated.

Thank you!