Streaming audio and recording video at the same time

OpenMV Products
1

Hello,

I have my openMV set up to record video to a MJPEG file. However, I would like to simultaneously receive audio data over SPI from another microcontroller and save to a file on OpenMV’s SD card. How can this be done?

Thanks,
RJ

2

Hi, we don’t support adding audio data to the MJPEG file. However, you can open a second file and store the data there.

As for handling the audio samples from spi… so, the SPI read is hardware accelerated. So, you can definitely pull data really fast though it. The API for that though blocks the CPU while the call is happening.

If that’s acceptable then you can just write a frame and then pull SPI data. Store the fifo buffer on the MCU providing the samples.

If you need a non blocking read then you need to edit our C code and build that yourself.

3

Got it thanks. Would it be possible to set up the openMV as a SPI slave? I know there is the RPC library for receiving a stream of data that is callback based, but how would I use this alongside saving the MJPEG?

4

Well, honestly, I’d use Async serial. You can clock that at 7.5 Mb/s with the H7/Plus. DMA goes directly to a buffer too. So, you can do ping pong buffers.

E.g. When you do a UART readinto call on a buffer it’s all hardware accelerated and writes directly to that buffer no matter the buffer length. So, it’s really what you want.

It will work much better than SPI.

…

Also, the RPC library will let you setup the transaction. Then, once the two devices are in sync you can use the direct read/write calls in the RPC library to move data mad quick. See the frame buffer transfer example I have between OpenMV Cams for inspiration.

5

Mmm, actually, the HAL layer still block you on the operation though. It’s not going to be non-blocking. If you want that then you probably need to look into modifying the C code.

To get the best performance for what you want to do you really kinda have to write a MicroPython driver.

So… this is C code to turn on the SPI bus or UART, then handle interrupts, use the DMA system, and invalidate RAM addresses. Not the easiest.