keypoint list?

Discussion related to "under the hood" OpenMV topics.
User avatar
jforce
Posts: 3
Joined: Sat Dec 09, 2017 3:14 pm

keypoint list?

Postby jforce » Sun Dec 10, 2017 11:00 am

I would like to try some structure from motion and related epipolar geometry algorithms. Is there a way to get the list of matched keypoints instead of the kptmatch object?
User avatar
kwagyeman
Posts: 1398
Joined: Sun May 24, 2015 2:10 pm

Re: keypoint list?

Postby kwagyeman » Sun Dec 10, 2017 12:31 pm

Right now there isn't, but, we can add it.

What would you like to see implemented?
Nyamekye,
User avatar
jforce
Posts: 3
Joined: Sat Dec 09, 2017 3:14 pm

Re: keypoint list?

Postby jforce » Mon Dec 11, 2017 11:55 am

I would think it would be something analogous to a list of cv::Keypoint.

My end goal is to do camera motion estimation. if I can publish the keypoint information across one of the serial ports, I can also run other post-processing

If the Cam 7 is up to the task, I think a trajectory estimator would be a high value add.

https://docs.opencv.org/3.1.0/d5/dab/tu ... ation.html
User avatar
iabdalkader
Posts: 403
Joined: Sun May 24, 2015 3:53 pm

Re: keypoint list?

Postby iabdalkader » Mon Dec 11, 2017 8:41 pm

Hi,
jforce wrote:
Mon Dec 11, 2017 11:55 am
I would think it would be something analogous to a list of cv::Keypoint.
I'll add this to the next release, however note each keypoint consumes a lot of memory (the descriptor alone is 32 bytes), converting a list of keypoints to Python objects could make the camera run out of memory, so this might not be very usable until our next camera is released.
jforce wrote:
Mon Dec 11, 2017 11:55 am
My end goal is to do camera motion estimation.
You could do this without accessing individual keypoints, using the delta of kptmatch objects centers (cx2-cx1, cy2-cy1).

I also tried this before using optical flow, it worked very well (I draw a map of my movement through the room) I'm attaching a couple of (very rough) scripts I used (one for the camera, the other for host side, it uses pygame to draw the map).
Attachments
optflow.py
(1.35 KiB) Downloaded 10 times
optical_flow_quad.py
(2.32 KiB) Downloaded 9 times
User avatar
jforce
Posts: 3
Joined: Sat Dec 09, 2017 3:14 pm

Re: keypoint list?

Postby jforce » Fri Dec 15, 2017 11:02 pm

These were useful examples, thanks.

In my case I'm travelling down a hallway with a fair number of wall features plus occasional obstacles. I'm not localizing/mapping, so the displacement information isn't as useful as position relative to the side walls or obstacles. I currently have my MV 7 pointing forward, and I'm using find_displacement with two small rectangular areas on the left and right. I subtract their absolute values to find differential forward minus common spin.
diff = math.fabs(delta_x_l) - math.fabs(delta_x_r)

This works fairly well for centering movement down a hallway until there is a junction or obstacle. I suspect I could get more useful information with furthur field segmentation, but with keypoint matches I can go directly to more established sfm algorithms.

There could be some reduction in the data. I don't need the full keypoint info, just the matched point locations in the image. Two lists of point locations where the same indexes represent the match should be good.

On a somewhat related issue, most of what I am doing will be memory intensive so if there is an option on the next design iteration for a larger ram capacity at a higher price, I would be interested in that.
User avatar
kwagyeman
Posts: 1398
Joined: Sun May 24, 2015 2:10 pm

Re: keypoint list?

Postby kwagyeman » Sat Dec 16, 2017 1:28 pm

The H7 model that will come out next year will have 1 MB of RAM. I think about 256KB will be used for the MP heap.
Nyamekye,
User avatar
iabdalkader
Posts: 403
Joined: Sun May 24, 2015 3:53 pm

Re: keypoint list?

Postby iabdalkader » Thu Dec 21, 2017 2:15 pm

Hi, this is now implemented in the development firmware. The keypoints match object has a list of matching keypoints indices (a list of tuples) you can use to subscript keypoints arrays. For example:

Code: Select all

for m in match.match():
    print(kpts1[m[0]], kpts2[m[1]])

The keypoint is a tuple of (x, y, score, octave, angle). Note you'll have to download and flash the firmware (or wait for the release):

https://github.com/openmv/openmv/raw/ma ... rmware.bin

Return to “Technical Discussion”

Who is online

Users browsing this forum: No registered users and 2 guests