Detecting optical flow

The task is to determine the optical flow.
From the methods presented by you, the “differentiaol_translation” method has the best characteristics of accuracy. But it does not detect a slow movement. How can this be combated?

Hi, the differential translation method will detect slow movement if you lower the sample rate. How slow are you talking?

Rate of displacement <0.5 m/s. What is meant by sample rate? Where to change it?

0.5 Meters/s is quite fast. Did you mean millimeters?

The same rate is controlled by the frame rate the system runs at. By default it just runs as fast as possible. If you’d like to slow down the sample rate just add a pyb.delay(100) to the main loop. Make sure to import the pyb module.

it will give a delay, right? And if you move quickly, you can speed up the data output?

Yes, you can change the delay variably. You’d want the delay to be inversely proportional to the movement speed.

Is it possible to calculate the optical flow only on a part of the frame? If so, сould you show an example of using a function “find_displacement” with all variables.

It was also found that when using the sensor under vibration conditions (for example, on a UAV), there is an offset hunting (to the large side) calculated by the optical flow. What could be causing this?

Hi, the find_displacement() method takes a ROI such that you can target a specific area of the image to run it on. There’s an example showing it being run on multiple locations that comes with OpenMV IDE. See the image-patches-differential-translation.py script under Optical Flow examples.

As for your second question… are you using a power of 2 resolution? Can you give me some more information?

I use method “differential-translation”, then summing the results to obtain the absolute displacement. The framesize is 128x128.

Are there ways to improve accuracy? Maybe filtering?

Hi, can you post your code? The differential method is quite accurate for one call in a power of 2 windows. So, I’m confused by what you are doing…

part of the code:

sum_x = 0.0
sum_y = 0.0

out_x = 0
out_y = 0

while(True):
#clock.tick() # Track elapsed milliseconds between snapshots().

Track elapsed milliseconds between snapshots().

img = sensor.snapshot() # Take a picture and return the image.

displacement = extra_fb.find_displacement(img)
extra_fb.replace(img)

Offset results are noisy without filtering so we drop some accuracy.

sub_pixel_x = int(displacement.x_translation() * 10) / 10.0
sub_pixel_y = int(displacement.y_translation() * 10) / 10.0

#sum
sum_x +=sub_pixel_x
sum_y +=sub_pixel_y

out_x = int(sum_x)
out_y = int(sum_y)

#sum_x += displacement.x_translation();
#sum_y += displacement.y_translation();

#out_x = int(sum_x)
#out_y = int(sum_y)
if(displacement.response() > 0.1): # Below 0.1 or so (YMMV) and the results are just noise.

send_optical_flow_packet(-out_x, out_y, displacement.response())


print("x= {0:+d} y= {1:+d} q= {2} ".format(out_x, out_y, (displacement.response()*100) ))

Ah, okay, I see what you are doing.

So, yes, filtering is necessary to improve the accuracy. Realistically, systems that do very accurate SLAM have numerous filters working with other data inputs like gryo scopes and such to obtain high accuracy. Our method just provides about the same functionality of a computer mouse. It’s good for determining a displacement direction but not for absolute positioning.

That said, performance should improve if it’s being used in a system where the camera is in a rigid body such that the direction of motion has no height variations.

How are you using the method?

This is conditionally “absolute displacement”. Used to hold the UAV in the specified position. Using a simple median filter from find_displacment will improve the results?

Yes, it will. Also, you should definitely lower the res to using B64X32. This is much faster than 128X128. The FFTs used by phase correlation don’t really need much pixels. They just need to see edges to work.

Alternatively, you can use the B64X64 res and then do use the mean_pool(2,2) method on the image to get a 32x32 image. Things will run even faster on that.

You need the speed to make sure that the phase correlation output is accurate. Note there’s also a couple confidence value output by the method. You should use that to determine the weighting of a result and if you should trust it or not.

Thank you! there is a problem with the accumulation of the error. A median filter was applied, but the problem remained. with rectilinear forward-backward motion, the error grows. how to neutralize it?

I honestly don’t know. I just implemented the algorithm and confirmed it indeed works.

Um, question, what’s your FPS? Also, what do you do about the confidence value? Once the confidence falls below a threshold the value is not longer valid to read… in your code I don’t see you filtering by the confidence. A confidence below 0.2 means the reading is likely bad.