keypoint list?

Discussion related to "under the hood" OpenMV topics.
User avatar
jforce
Posts: 3
Joined: Sat Dec 09, 2017 3:14 pm

keypoint list?

Postby jforce » Sun Dec 10, 2017 11:00 am

I would like to try some structure from motion and related epipolar geometry algorithms. Is there a way to get the list of matched keypoints instead of the kptmatch object?
User avatar
kwagyeman
Posts: 2275
Joined: Sun May 24, 2015 2:10 pm

Re: keypoint list?

Postby kwagyeman » Sun Dec 10, 2017 12:31 pm

Right now there isn't, but, we can add it.

What would you like to see implemented?
Nyamekye,
User avatar
jforce
Posts: 3
Joined: Sat Dec 09, 2017 3:14 pm

Re: keypoint list?

Postby jforce » Mon Dec 11, 2017 11:55 am

I would think it would be something analogous to a list of cv::Keypoint.

My end goal is to do camera motion estimation. if I can publish the keypoint information across one of the serial ports, I can also run other post-processing

If the Cam 7 is up to the task, I think a trajectory estimator would be a high value add.

https://docs.opencv.org/3.1.0/d5/dab/tu ... ation.html
User avatar
iabdalkader
Posts: 542
Joined: Sun May 24, 2015 3:53 pm

Re: keypoint list?

Postby iabdalkader » Mon Dec 11, 2017 8:41 pm

Hi,
jforce wrote:
Mon Dec 11, 2017 11:55 am
I would think it would be something analogous to a list of cv::Keypoint.
I'll add this to the next release, however note each keypoint consumes a lot of memory (the descriptor alone is 32 bytes), converting a list of keypoints to Python objects could make the camera run out of memory, so this might not be very usable until our next camera is released.
jforce wrote:
Mon Dec 11, 2017 11:55 am
My end goal is to do camera motion estimation.
You could do this without accessing individual keypoints, using the delta of kptmatch objects centers (cx2-cx1, cy2-cy1).

I also tried this before using optical flow, it worked very well (I draw a map of my movement through the room) I'm attaching a couple of (very rough) scripts I used (one for the camera, the other for host side, it uses pygame to draw the map).
Attachments
optflow.py
(1.35 KiB) Downloaded 42 times
optical_flow_quad.py
(2.32 KiB) Downloaded 48 times
User avatar
jforce
Posts: 3
Joined: Sat Dec 09, 2017 3:14 pm

Re: keypoint list?

Postby jforce » Fri Dec 15, 2017 11:02 pm

These were useful examples, thanks.

In my case I'm travelling down a hallway with a fair number of wall features plus occasional obstacles. I'm not localizing/mapping, so the displacement information isn't as useful as position relative to the side walls or obstacles. I currently have my MV 7 pointing forward, and I'm using find_displacement with two small rectangular areas on the left and right. I subtract their absolute values to find differential forward minus common spin.
diff = math.fabs(delta_x_l) - math.fabs(delta_x_r)

This works fairly well for centering movement down a hallway until there is a junction or obstacle. I suspect I could get more useful information with furthur field segmentation, but with keypoint matches I can go directly to more established sfm algorithms.

There could be some reduction in the data. I don't need the full keypoint info, just the matched point locations in the image. Two lists of point locations where the same indexes represent the match should be good.

On a somewhat related issue, most of what I am doing will be memory intensive so if there is an option on the next design iteration for a larger ram capacity at a higher price, I would be interested in that.
User avatar
kwagyeman
Posts: 2275
Joined: Sun May 24, 2015 2:10 pm

Re: keypoint list?

Postby kwagyeman » Sat Dec 16, 2017 1:28 pm

The H7 model that will come out next year will have 1 MB of RAM. I think about 256KB will be used for the MP heap.
Nyamekye,
User avatar
iabdalkader
Posts: 542
Joined: Sun May 24, 2015 3:53 pm

Re: keypoint list?

Postby iabdalkader » Thu Dec 21, 2017 2:15 pm

Hi, this is now implemented in the development firmware. The keypoints match object has a list of matching keypoints indices (a list of tuples) you can use to subscript keypoints arrays. For example:

Code: Select all

for m in match.match():
    print(kpts1[m[0]], kpts2[m[1]])

The keypoint is a tuple of (x, y, score, octave, angle). Note you'll have to download and flash the firmware (or wait for the release):

https://github.com/openmv/openmv/raw/ma ... rmware.bin
SADEGHRAHMANI1374
Posts: 10
Joined: Sat Feb 24, 2018 7:11 am

Re: keypoint list?

Postby SADEGHRAHMANI1374 » Tue Mar 27, 2018 9:08 am

Hi, I have a problem with matching the key point
I have several items I want to identify with openmv
But when the file size is more than 70 kilobytes
ide returns this error


I think it's because of the low memory performance of the camera
Can we use external memory as executable(ram) memory?
Attachments
New folder.rar
(55.11 KiB) Downloaded 27 times
error.JPG
error.JPG (147.69 KiB) Viewed 1102 times
User avatar
iabdalkader
Posts: 542
Joined: Sun May 24, 2015 3:53 pm

Re: keypoint list?

Postby iabdalkader » Tue Mar 27, 2018 10:06 am

SADEGHRAHMANI1374 wrote: I think it's because of the low memory performance of the camera
Yes the camera has limited amount of memory. Try to lower the number of keypoints (increase the thresholds).
SADEGHRAHMANI1374 wrote: Can we use external memory as executable(ram) memory?
It's possible but the camera doesn't have an external ram on board.

Return to “Technical Discussion”

Who is online

Users browsing this forum: No registered users and 5 guests