Frame Buffer & sensor.snapshot()

Discussion related to "under the hood" OpenMV topics.
Posts: 5
Joined: Sat Oct 03, 2015 8:41 pm

Frame Buffer & sensor.snapshot()

Postby stealthmedal » Sun Jul 10, 2016 1:24 am


I am trying to get an understanding of how the IDE frame buffer works. It seems that the buffer updates after the first call of sensor.snapshot(). Are images pushed to the framebuffer and pulled from the sensor at the same time? Which comes first?
~ Thanks
User avatar
Posts: 1214
Joined: Sun May 24, 2015 3:53 pm

Re: Frame Buffer & sensor.snapshot()

Postby iabdalkader » Sun Jul 10, 2016 10:26 am

The IDE polls for frames using USB packets/requests, reading a frame is split into two main phases: the first phase/request (USBDBG_FRAME_SIZE) either returns the frame size/header if there's a valid frame (i.e. snapshot() was called at least once after the first frame) or it returns NULL if not, either way when this request is received by the camera, it sets a flag indicating that the IDE has requested a frame:

Code: Select all

    if (!fb->ready) {
        // Frame not ready return 0
        ((uint32_t*)buffer)[0] = 0;
    } else {
        // Frame ready return header
        ((uint32_t*)buffer)[0] = fb->w;
        ((uint32_t*)buffer)[1] = fb->h;
        ((uint32_t*)buffer)[2] = fb->bpp;
    fb->request = 1;
    cmd = USBDBG_NONE;
The next frame capture (next time snapshot is called from a running script probably in a loop), and before overwriting the previous frame, if this flag is set the camera waits for the IDE to read the frame.

Code: Select all

while (fb->ready && fb->request) {
fb->ready = 0;
The IDE then reads the frame using a number of USBDBG_FRAME_DUMP requests.

Here's the relevant code parts: ... sor.c#L588 ... bdbg.c#L94

Note: The first frame after a reset or a script is run, is always ignored by the camera even if the IDE requested a frame, because at this point there's no frame (in case of a reset) or the resolution/format has changed.

Note 2: Waiting for the IDE to read the frame before it's overwritten fakes double buffering, if the frame is read at any other time you see flickers.

Return to “Technical Discussion”

Who is online

Users browsing this forum: No registered users and 4 guests