Image degradation with rotation_corr method

OpenMV Products
1

Hi, all. I’m iteratively rotating then testing an image with the Haar Cascade face detection algorithm to account for head and/or camera tilt. For example:

sensor.reset()
sensor.set_contrast(3)
sensor.set_gainceiling(16)
sensor.set_framesize(sensor.HQVGA)
sensor.set_pixformat(sensor.GRAYSCALE)
sensor.skip_frames(time = 2000)

face_cascade = image.HaarCascade(“frontalface”, stages=25)

while (True):
# Capture snapshot
img = sensor.snapshot()

for angle in [-30, 30, 30]:
    img.rotation_corr(x_rotation=0.0, y_rotation=0.0, z_rotation=angle)
    objects = img.find_features(face_cascade, threshold=0.75, scale_factor=1.25)

This works reasonably well with three angles as above, but if I attempt a more granular series of rotations, even with only five angles, performance devolves substantially as does the image itself.

e.g.: for angle in [-30, 15, 15, 15, 15]:

The resultant image after the fifth rotation is very grainy and overall performance suffers.

Any suggestions as to how to optimize this?

Thanks,

Rick

2

Yeah, you should make a copy of the image and rotate the copy. You can repeatedly rotating the same time.

So, alloc and extra fb using the sensor.alloc_extra_fb() method and then copy the camera image into that frame buffer and then rotate that image. Call find_features on that image. And then draw the results on the original image.

3

While that eliminated the image degradation, the performance hit is pretty substantial (FPS dropped almost in half). I’ll keep experimenting. I wonder if the rotation code would benefit from a 2D rotation optimization?

4

I have that in my plans to speedup. Doing lower level stuff first though.

5

I’d be happy to take a look too if that helps. LMK where that module hides in Github.

6

The probably is the bounds check per pixel.

1 Like
7

Generically there can be a bunch of optimizations if any of the parameters to the method are null or zero or default. It’s been a long time since I wrote any C code but I’ll take a look - any docs for the various matrix operation macros you’re using?

8

That’s not the issue. For an RGB565 image we use this loop:

The int → float → int math is not optimal. But, not the performance issue. What’s making it slow is the if() check in the loop as this is done per pixel to avoid going out of bounds.

Then, the loop doesn’t access memory in a cache friendly way… which causes issues too.

…

To make it faster the loop needs to be broken up into pieces that handle the different boundary conditions or none. For example… you could figure out what x/y will produce a sourceX/Y that’s out of bounds by inverting the formula for x/y. After doing that you can easily run most of the loop without boundary checks.