Frame Buffer & sensor.snapshot()

Hi!

I am trying to get an understanding of how the IDE frame buffer works. It seems that the buffer updates after the first call of sensor.snapshot(). Are images pushed to the framebuffer and pulled from the sensor at the same time? Which comes first?
~ Thanks

The IDE polls for frames using USB packets/requests, reading a frame is split into two main phases: the first phase/request (USBDBG_FRAME_SIZE) either returns the frame size/header if there’s a valid frame (i.e. snapshot() was called at least once after the first frame) or it returns NULL if not, either way when this request is received by the camera, it sets a flag indicating that the IDE has requested a frame:

case USBDBG_FRAME_SIZE:
    if (!fb->ready) {
        // Frame not ready return 0
        ((uint32_t*)buffer)[0] = 0;
    } else {
        // Frame ready return header
        ((uint32_t*)buffer)[0] = fb->w;
        ((uint32_t*)buffer)[1] = fb->h;
        ((uint32_t*)buffer)[2] = fb->bpp;
    }
    fb->request = 1;
    cmd = USBDBG_NONE;
    break;

The next frame capture (next time snapshot is called from a running script probably in a loop), and before overwriting the previous frame, if this flag is set the camera waits for the IDE to read the frame.

while (fb->ready && fb->request) {
    systick_sleep(2);
}
fb->ready = 0;

The IDE then reads the frame using a number of USBDBG_FRAME_DUMP requests.

Here’s the relevant code parts:
https://github.com/openmv/openmv/blob/master/src/omv/sensor.c#L588

https://github.com/openmv/openmv/blob/master/src/omv/usbdbg.c#L94

Note: The first frame after a reset or a script is run, is always ignored by the camera even if the IDE requested a frame, because at this point there’s no frame (in case of a reset) or the resolution/format has changed.

Note 2: Waiting for the IDE to read the frame before it’s overwritten fakes double buffering, if the frame is read at any other time you see flickers.