Sensor.sub_windowing/Sensor.ioctl Does Not Increase FPS on OpenMV H7 Plus

OpenMV Products
1

Taking a subwindow instead of the full frame should result in higher FPS since the camera isn’t capturing the whole image. However, the OpenMV H7 Plus is not able to achieve higher FPS by setting a smaller region of interest.

import sensor, image, utime, omv

omv.disable_fb(True)

sensor.reset()                     
sensor.set_pixformat(sensor.RGB565) 
sensor.set_framesize(sensor.HD)   
ROI1 = (0,0,200,200)
ROI2 = (0,0,600,600)
ROI3 = (0,0,1200,600)

sensor.set_windowing(ROI1)
#sensor.ioctl(sensor.IOCTL_SET_READOUT_WINDOW, ROI1)

sensor.skip_frames(time = 2000)    

while(True):
    Stime = utime.ticks_ms()/1000
    img = sensor.snapshot()         
    Etime = utime.ticks_ms()/1000
    print("FPS", 1/(Etime-Stime), Etime-Stime)

I’ve tried three different region of interests using two different methods, sensor.set_windowing and sensor.ioctl. The results are summarized below.

___________________________Using Set_windowing___Using Ioctl
ROI1 = (0,0,200,200)_____________28.6 FPS__________37.7 FPS
ROI2 = (0,0,600,600)__28.6 FPS__________37.7 FPS
ROI3 = (0,0,1200,600) 28.6 FPS__________37.7 FPS
Full frame/No ROI________________28.6 FPS__________28.6 FPS

Clearly, reducing the ROI isn’t doing anything. The FPS stays the same regardless of the size of the frame being portioned. Just calling sensor.ioctl is increasing the FPS, but the FPS for all the ROIs are still the same.

2

Which camera is that? The windowing only works for the OV5640.

3

Which camera is that? The windowing only works for the OV5640.

4

You are setting a resolution of HD. Please read the documentation. It says the readout window will not be set to less than the frame size.

5

“readout window will not be set to less than the frame size”

What does this mean? If the frame size is HD (1280x720), then the readout window has to be bigger? Sorry, but that doesn’t make sense to me. I can’t set a ROI in sub_windowing or ioctl that is bigger than 1280x720. So can I only set the ROI to 1280x720? If so then what’s the point of a subwindow?

6

This is the documentation for setwindowing. I’m using the OV5640 sensor.

sensor.set_windowing(roi)
Sets the resolution of the camera to a sub resolution inside of the current resolution. For example, setting the resolution to sensor.VGA and then the windowing to (120, 140, 200, 200) sets sensor.snapshot() to capture the 200x200 center pixels of the VGA resolution outputted by the camera sensor. You can use windowing to get custom resolutions. Also, when using windowing on a larger resolution you effectively are digital zooming.

roi is a rect tuple (x, y, w, h). However, you may just pass (w, h) and the roi will be centered on the frame.

Note that sensor.set_windowing() forces (x, y, w, h) to be even rounded down. This is done to ensure that RGB565/GRAYSCALE/BAYER images all have the same offsets and display correctly given windowing as BAYER images must have an even width/height.

Setting the resolution to VGA and windowing to (120, 140, 200, 200) did not change FPS.

7

It means it will still read VGA and crop to the smaller window, taking the same amount of time. Try QVGA should be faster.

8

Hi,

This should clear things up:

  1. The IOCTL sets the camera frame size that the camera chip itself reads pixels from the sensor array. To prevent things from widly breaking you can only set the ioctl to be greater than the current frame size. E.g. if the frame size is QVGA then you may only request down to QVGA.

  2. Now the camera will read that area from the sensor array. Then it will scale the read pixels to equal the target resolution you set. So, if the ioctl window was VGA sized and you want QVGA then the camera will reduce the output res by 2 in x/y.

  3. Now the STM32 chip will receive the QVGA image. Our set windowing method lets you crop that on the MCU side of things. This allows you to get a resolution that’s not allowed via set_framesize().

So, to be clear, the ioctl and setframesize are modifying camera register settings to get it to generate an image. We’ve written our driver code to be very careful about controlling the camera to make sure things work.