Nicla Vision audio spectrogram

Hi everyone, I’m new in arduino boards, OpenMV and also in Phyton.
I have an Arduino Nicla Vision and I’m trying to extract audio data from integrated microphone. I’ve started using the tutorial code (audio_fft) but I can’t solve anyway this AttributError: ‘module’ object has not attribute ‘spectrogram’.

I don’t know if the problem is a missing library, but I haven’t found how to install it on OpenMV.
Thank you.
This is the code:

import image, audio, time
from ulab import numpy as np
from ulab import scipy as sp

CHANNELS = 1
SIZE = 256//(2*CHANNELS)

raw_buf = None
fb = image.Image(SIZE+50, SIZE, image.RGB565, copy_to_fb=True)
audio.init(channels=CHANNELS, frequency=16000, gain_db=24, highpass=0.9883)

def audio_callback(buf):
    # NOTE: do Not call any function that allocates memory.
    global raw_buf
    if (raw_buf == None):
        raw_buf = buf

# Start audio streaming
audio.start_streaming(audio_callback)

def draw_fft(img, fft_buf):
    fft_buf = (fft_buf / max(fft_buf)) * SIZE
    fft_buf = np.log10(fft_buf + 1) * 20
    color = (0xFF, 0x0F, 0x00)
    for i in range(0, SIZE):
        img.draw_line(i, SIZE, i, SIZE-int(fft_buf[i]), color, 1)

def draw_audio_bar(img, level, offset):
    blk_size = SIZE//10
    color = (0xFF, 0x00, 0xF0)
    blk_space = (blk_size//4)
    for i in range(0, int(round(level/10))):
        fb.draw_rectangle(SIZE+offset, SIZE - ((i+1)*blk_size) + blk_space, 20, blk_size - blk_space, color, 1, True)

while (True):
    if (raw_buf != None):
        pcm_buf = np.frombuffer(raw_buf, dtype=np.int16)
        raw_buf = None

        if CHANNELS == 1:
            fft_buf = sp.signal.spectrogram(pcm_buf)
            l_lvl = int((np.mean(abs(pcm_buf[1::2])) / 32768)*100)
        else:
            fft_buf = sp.signal.spectrogram(pcm_buf[0::2])
            l_lvl = int((np.mean(abs(pcm_buf[1::2])) / 32768)*100)
            r_lvl = int((np.mean(abs(pcm_buf[0::2])) / 32768)*100)

        fb.clear()
        draw_fft(fb, fft_buf)
        draw_audio_bar(fb, l_lvl, 0)
        if CHANNELS == 2:
            draw_audio_bar(fb, r_lvl, 25)
        fb.flush()

# Stop streaming
audio.stop_streaming()`Preformatted text`

@iabdalkader

This example should work, are you on the latest firmware ?

my filmware is latest and i have same problem

Yes, OpenMV has upload it right before I’ve tried to execute the code

Okay ulab moved that function from sp.signal to utils, you just need to use utils instead, i.e:

from ulab import utils
fft_buf = utils.spectrogram(...)

We will update the examples very soon.

1 Like

is it possible to save the audio record in wav format

Not sure if that’s possible on the Nicla. On our M4/M7/H7 line we do support doing this via the ADC pin.

I’m trying to do that here with this code: Problem-loss of Audio data from Arduino Nicla vision to Computer - Interfacing w/ Software on the Computer - Arduino Forum

I saved a .wav file but I’ve some problems…

Okay, you should be able to get support form Arduino there. You are using their C library.

Is there anybody who can confirm the 2 audio examples (audio_fft.py and micro_speech.py) are working on Nicla Vision. I just bought the board and am trying to get them working. I am able to download the script(s) to the board but no activity is shown in the openMV IDE (firmware 4.4.2).

The FFT example works. Looks like MicroSpeech is broken.

Yes… our focus hasn’t really been on maintaining the audio code there. We’re really just the vision right now. I’ll make a bug tracker for this though.

Hi, this arduno nicla vision camera is not able to capture images properly in night, what is the solution for that? and any other algorithms for doing eye-tracking apart from haar cascade

What do you want to do? And can you give an example of where it fails. The camera cannot see in the dark.

i want to capture images at both day and night, but night i tried with dark in presence of ir light , but the image is not clear, my project is like capturing images at day and night (irled), any possible solutions for it?

You can’t remove the IR cut filter from the lens so it’s going to be unable to see in the dark. The product isn’t really designed for that.

However, if you are doing an application where there’s almost no motion you can increase the image exposure which will allow the camera to see in darker environments.