Interupt sensor.snapshot

Arduino Boards
1

I want to send data audio and video streaming via socket. when i call function audio streaming and audio snapshot. Data that i got is bad. like audio data obtained feels laggy (audio data is cut off). when i call it just one function (just sensor.snapshot, or just audio streaming) data that I obtained is good. What sould i do to make data audio and video streaming that i send not being cut off ?

this is my code


# MJPEG Streaming
#
# This example shows off how to do MJPEG streaming to a FIREFOX webrowser
# Chrome, Firefox and MJpegViewer App on Android have been tested.
# Connect to the IP address/port printed out from ifconfig to view the stream.
import sensor, image, time, network, socket, sys, audio
from ulab import numpy as np
from ulab import scipy as sp
import pyb,micropython

#micropython.alloc_emergency_exception_buf(100)
SSID='Data Pusaka 2.4'      # Network SSID
KEY='admin.admin'       # Network key
HOST ='x.x.x.x'     # Use first available interface
PORT = 8080  # Arbitrary non-privileged port
CHANNELS = 1
SIZE = 256//(2*CHANNELS)

raw_buf = None
# Init sensor
sensor.reset()
sensor.set_framesize(sensor.QVGA)
sensor.set_pixformat(sensor.RGB565)
sensor.set_auto_whitebal(False) # Turn off white balance.
audio.init(channels=CHANNELS, frequency=16000, gain_db=24, highpass=0.9883)

# Init wlan module and connect to network
print("Trying to connect... (This may take a while)...")
wlan = network.WLAN(network.STA_IF)
wlan.active(True)
wlan.connect(SSID, KEY)

# We should have a valid IP now via DHCP
print("WiFi Connected ", wlan.ifconfig())

# Create server socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)


# Bind and listen
s.connect([HOST, PORT])

start_time_mic = pyb.millis()
start_time_vid = pyb.millis()

# Define the desired delay in milliseconds
delay_ms = 30
a = 1
data_buffer = bytearray()
start = time.ticks_ms()
current_time = 0
#sensor.set_framebuffers(3) 
#sensor.set_windowing((20, 20, 20, 20))  # Contoh nilai, sesuaikan dengan kebutuhan

# Fungsi callback untuk pengambilan data audio
def audio_callback(buf):
    global start_time_mic, data_buffer,current_time

    current_time = time.ticks_ms()
    elapsed_time_mic = current_time - start_time_mic

    # Menambahkan data ke dalam buffer
    data_buffer.extend(buf)
    global start
    # Start streaming images
    # NOTE: Disable IDE preview to increase streaming FPS.
    while ((current_time - start) > 30) and data_buffer:
#    if elapsed_time_mic >= delay_ms and data_buffer:
        try:
            # Mengirim data dari buffer ke server
            s.sendall(data_buffer)

        except Exception as e:
            print("Error sending data:", e)

#        start_time_mic = current_time
        data_buffer = bytearray()
        start = time.ticks_ms()
        

#    time.sleep_ms(30)



audio.start_streaming(audio_callback)


# Set fungsi callback untuk timer

    

#timer.callback(capture_frames)



while (True):
    try:
#        start_streaming(s)
#        audio.start_streaming(audio_callback)
        latest_frame = sensor.snapshot()
        
#        time.sleep_ms(30)
#        audio.stop_streaming()
        
    except OSError as e:
        print("socket error: ", e)
        audio.stop_streaming()

        #sys.print_exception(e)
2

So, when you comment out latest_frame = sensor.snapshot() it works but when using that it doesn’t?

The audio callback is run via micropython schedule. It’s not an interrupt callback. So, snapshot is blocking resulting in dropped audio.

To fix this you want do:

sensor.set_framebuffers(3)
sensor.snapshot() # Kicks off DMA system on first call if not already on.

After setting up the camera pixformat and resolution. This will then start camera data streaming in the background. After which, you can do:

if sensor.get_frame_available():
    latest_frame = sensor.snapshot()

And snapshot will not block. Then your audio data should be stable.

3

Thanks for your reply. I appreciate that. :slight_smile:

I’ve uploaded this code,

if sensor.get_frame_available():
    latest_frame = sensor.snapshot()

but it cant call function snapshot, because sensor.get_frame_availeable is false when i check output on serial monitor. What did i do wrong ?

4

Can you print out the number of frame buffers in the main loop:

print(sensor.get_framebuffers())

It needs to be 3. If not, please reduce your resolution. Things can also work at 2 but DMA may stop if the system gets busy. At 3 frame buffers the system should always have new images ready.

Note, it’s import to call snapshot first before the loop to kick off the DMA frame capture system.