Benefits of using floats for reporting blob location

Hi All,

I’ve been playing around with the blob tracking in openMV, and I Have a suggestion for improving the effective tracking resolution.

As background, people might remember the amazing performance of the WiiMote’s IR blob tracking camera. It was one of the first cheap, hackable computer vision systems and lots of projects were made based on it.
What a lot of people don’t know is that the WiiMote’s sensor was only 128x96 pixels native resolution, but by using sub-pixel averaging on the data it reported, an effective resolution of 1024x768 was obtained.

There’s a way to use the same ‘trick’ in OpenMV. In the code here:
https://github.com/openmv/openmv/blob/master/src/omv/img/blob.c
the code tallies up the horizontal and vertical locations of each pixel in the blob, and then divides them by the number of pixels found. In particular these lines:

int mx = blob_cx / blob_pixels; // x centroid
int my = blob_cy / blob_pixels; // y centroid

Because all the data types are integers, rounding error occurs. if blob_cx and blob_pixels could be converted to floats just before the division then much extra accuracy can be gained.

Here’s a simulation of the benefits of sub-pixel averaging. (Python notebook file attached if anyone wants to play with the model).


As you can see, in reasonable cases (circular targets around 36 pixels across), the effective resolution increases around 10-fold.

If you don’t want to change the API to have blob.x and blob.y returned as floats, what about including extra fields such as blob.x_float and blob.y_float?

Cheers,
Gavin
sub-pixel centroid test v01.zip (58.1 KB)

I could do that. Um, right now I’m in the middle of doing the H7 kickstarter campagin. Would you be interested in submitting a PR for what you want? Adding it is quite simple:

Add the new values here:

https://github.com/openmv/openmv/blob/master/src/omv/img/imlib.h#L1052

Set them here (also update them when merging):

https://github.com/openmv/openmv/blob/master/src/omv/img/blob.c#L182

Add all the support for them here:

https://github.com/openmv/openmv/blob/master/src/omv/py/py_image.c#L3632

And finally add a keyword for them here:

https://github.com/openmv/openmv/blob/master/src/omv/py/qstrdefsomv.h#L791

Hi Kwabena,

Thanks for the response. I’m not confident I can do the changes without making a mistake, though. Is someone else able to do it? I don’t mind waiting for the feature.

Good luck with the new kickstarter, I’m looking forward to seeing it!

In case you’re interested, the reason I want sub-pixel blob finding is I’m in the process of making a large interactive table for exploring math functions. The operator will hold a stylus, and the camera will detect where the tip is, and calculate the response of the system to the (complex) number input, which will be drawn onto the table with a laser and galvanometer setup. With the increased resolution of sub-pixel blob finding I can explore much more complex functions (such as fractals), which would less interesting if quantized to 640x480.

Cheers,
Gavin