Detecting optical flow

OpenMV related project discussion.
user
Posts: 8
Joined: Wed Jun 20, 2018 7:48 am

Detecting optical flow

Postby user » Wed Jun 20, 2018 7:55 am

The task is to determine the optical flow.
From the methods presented by you, the "differentiaol_translation" method has the best characteristics of accuracy. But it does not detect a slow movement. How can this be combated?
User avatar
kwagyeman
Posts: 2042
Joined: Sun May 24, 2015 2:10 pm

Re: Detecting optical flow

Postby kwagyeman » Wed Jun 20, 2018 2:00 pm

Hi, the differential translation method will detect slow movement if you lower the sample rate. How slow are you talking?
Nyamekye,
user
Posts: 8
Joined: Wed Jun 20, 2018 7:48 am

Re: Detecting optical flow

Postby user » Thu Jun 21, 2018 6:50 am

Rate of displacement <0.5 m/s. What is meant by sample rate? Where to change it?
User avatar
kwagyeman
Posts: 2042
Joined: Sun May 24, 2015 2:10 pm

Re: Detecting optical flow

Postby kwagyeman » Thu Jun 21, 2018 12:20 pm

0.5 Meters/s is quite fast. Did you mean millimeters?

The same rate is controlled by the frame rate the system runs at. By default it just runs as fast as possible. If you'd like to slow down the sample rate just add a pyb.delay(100) to the main loop. Make sure to import the pyb module.
Nyamekye,
user
Posts: 8
Joined: Wed Jun 20, 2018 7:48 am

Re: Detecting optical flow

Postby user » Fri Jun 22, 2018 6:22 am

it will give a delay, right? And if you move quickly, you can speed up the data output?
User avatar
kwagyeman
Posts: 2042
Joined: Sun May 24, 2015 2:10 pm

Re: Detecting optical flow

Postby kwagyeman » Fri Jun 22, 2018 5:17 pm

Yes, you can change the delay variably. You'd want the delay to be inversely proportional to the movement speed.
Nyamekye,
user
Posts: 8
Joined: Wed Jun 20, 2018 7:48 am

Re: Detecting optical flow

Postby user » Thu Jun 28, 2018 10:51 am

Is it possible to calculate the optical flow only on a part of the frame? If so, сould you show an example of using a function "find_displacement" with all variables.

It was also found that when using the sensor under vibration conditions (for example, on a UAV), there is an offset hunting (to the large side) calculated by the optical flow. What could be causing this?
User avatar
kwagyeman
Posts: 2042
Joined: Sun May 24, 2015 2:10 pm

Re: Detecting optical flow

Postby kwagyeman » Thu Jun 28, 2018 11:55 am

Hi, the find_displacement() method takes a ROI such that you can target a specific area of the image to run it on. There's an example showing it being run on multiple locations that comes with OpenMV IDE. See the image-patches-differential-translation.py script under Optical Flow examples.

As for your second question... are you using a power of 2 resolution? Can you give me some more information?
Nyamekye,
user
Posts: 8
Joined: Wed Jun 20, 2018 7:48 am

Re: Detecting optical flow

Postby user » Mon Jul 02, 2018 2:31 am

I use method "differential-translation", then summing the results to obtain the absolute displacement. The framesize is 128x128.

Are there ways to improve accuracy? Maybe filtering?
User avatar
kwagyeman
Posts: 2042
Joined: Sun May 24, 2015 2:10 pm

Re: Detecting optical flow

Postby kwagyeman » Mon Jul 02, 2018 3:08 am

Hi, can you post your code? The differential method is quite accurate for one call in a power of 2 windows. So, I'm confused by what you are doing...
Nyamekye,
user
Posts: 8
Joined: Wed Jun 20, 2018 7:48 am

Re: Detecting optical flow

Postby user » Mon Jul 02, 2018 3:18 am

part of the code:

sum_x = 0.0
sum_y = 0.0

out_x = 0
out_y = 0

while(True):
#clock.tick() # Track elapsed milliseconds between snapshots().
# Track elapsed milliseconds between snapshots().
img = sensor.snapshot() # Take a picture and return the image.

displacement = extra_fb.find_displacement(img)
extra_fb.replace(img)

# Offset results are noisy without filtering so we drop some accuracy.
sub_pixel_x = int(displacement.x_translation() * 10) / 10.0
sub_pixel_y = int(displacement.y_translation() * 10) / 10.0

#sum
sum_x +=sub_pixel_x
sum_y +=sub_pixel_y

out_x = int(sum_x)
out_y = int(sum_y)

#sum_x += displacement.x_translation();
#sum_y += displacement.y_translation();

#out_x = int(sum_x)
#out_y = int(sum_y)
if(displacement.response() > 0.1): # Below 0.1 or so (YMMV) and the results are just noise.

send_optical_flow_packet(-out_x, out_y, displacement.response())


print("x= {0:+d} y= {1:+d} q= {2} ".format(out_x, out_y, (displacement.response()*100) ))
User avatar
kwagyeman
Posts: 2042
Joined: Sun May 24, 2015 2:10 pm

Re: Detecting optical flow

Postby kwagyeman » Mon Jul 02, 2018 3:37 am

Ah, okay, I see what you are doing.

So, yes, filtering is necessary to improve the accuracy. Realistically, systems that do very accurate SLAM have numerous filters working with other data inputs like gryo scopes and such to obtain high accuracy. Our method just provides about the same functionality of a computer mouse. It's good for determining a displacement direction but not for absolute positioning.

That said, performance should improve if it's being used in a system where the camera is in a rigid body such that the direction of motion has no height variations.

How are you using the method?
Nyamekye,
user
Posts: 8
Joined: Wed Jun 20, 2018 7:48 am

Re: Detecting optical flow

Postby user » Mon Jul 02, 2018 3:48 am

This is conditionally "absolute displacement". Used to hold the UAV in the specified position. Using a simple median filter from find_displacment will improve the results?
User avatar
kwagyeman
Posts: 2042
Joined: Sun May 24, 2015 2:10 pm

Re: Detecting optical flow

Postby kwagyeman » Mon Jul 02, 2018 2:55 pm

Yes, it will. Also, you should definitely lower the res to using B64X32. This is much faster than 128X128. The FFTs used by phase correlation don't really need much pixels. They just need to see edges to work.

Alternatively, you can use the B64X64 res and then do use the mean_pool(2,2) method on the image to get a 32x32 image. Things will run even faster on that.

You need the speed to make sure that the phase correlation output is accurate. Note there's also a couple confidence value output by the method. You should use that to determine the weighting of a result and if you should trust it or not.
Nyamekye,
user
Posts: 8
Joined: Wed Jun 20, 2018 7:48 am

Re: Detecting optical flow

Postby user » Wed Jul 04, 2018 10:44 am

Thank you! there is a problem with the accumulation of the error. A median filter was applied, but the problem remained. with rectilinear forward-backward motion, the error grows. how to neutralize it?
User avatar
kwagyeman
Posts: 2042
Joined: Sun May 24, 2015 2:10 pm

Re: Detecting optical flow

Postby kwagyeman » Wed Jul 04, 2018 2:23 pm

I honestly don't know. I just implemented the algorithm and confirmed it indeed works.

Um, question, what's your FPS? Also, what do you do about the confidence value? Once the confidence falls below a threshold the value is not longer valid to read... in your code I don't see you filtering by the confidence. A confidence below 0.2 means the reading is likely bad.
Nyamekye,

Return to “Project Discussion”

Who is online

Users browsing this forum: No registered users and 2 guests