How to measure volume of objects with irregular shapes

Hi guys,
I wanna use vertical and horizontal cameras to measure the volumes of objects with irregular shapes on a moving conveyor belt.
Now I can get the pixels and area of vertical and horizontal blobs of the same object with openmv’s image.find_blobs().
But it is not accurate to just use the pixels and area of vertical and horizontal blobs of the same object with irregular shape to compute xyz of the blob for the object’s volume.
Could you give me some suggestions?
Looking forward to your answers.
Thank you very much!

Can I get coordinates of every pixels in blob found with find_blobs()?
If getting the coordinates, I can use calculus’s method to compute the volume accurately.

Hmm, I see what you want to do. Um, so, the number of pixels in each blob is exposed. Can you use this value to guess? Otherwise, you need to edit the C firmware to do what you want:

The loop that does the math accumulation is right here: https://github.com/openmv/openmv/blob/master/src/omv/img/blob.c#L250. If you study the find_blobs() code you can add new fields to the blob objects to keep track of the pixels.

Note… I’m going to extend the find_blobs() code this weekend for another customer. I can probably add a new feature for you too if it’s some mathematical value I can just add as a field. If possible, study the code and get back to me ASAP on whatever new calculations you need and I’ll add then.

That said, it has to be something that’s very fast to compute in that inner loop. However, once the blob is finished being mapped there’s a math section at the end: https://github.com/openmv/openmv/blob/master/src/omv/img/blob.c#L333 Where the output values are formed.

Hi,
Thank you so much for your quick help!
Could you add and expose a array pixels[w][h] in the blob class to describe which pixels is inside of the blob and which ones are out of the blob but inside the rectangle bounding the blob:
If pixels_[j]=1, the pixel is inside the blob;
If pixels[j]=0, the pixel is out of the blob but inside the rectangle bounding the blob.
If saving memory, we can also use bits to describe which pixels are in the blob.
Looking forward to your extending the find_blobs() code!
Thank you again!
Augur_

Hi,
Thank you so much for your quick suggestion and help!
Could you add and expose a array pixels[w][h] in the blob class to describe which pixels is inside of the blob and which ones are out of the blob but inside the rectangle bounding the blob:
If pixels [j]=1, the pixel is inside the blob;
If pixels [j]=0, the pixel is out of the blob but inside the rectangle bounding the blob.
If saving memory, we can also use bits to describe which pixels are in the blob.
Looking forward to your extending the find_blobs() code!
Thank you again!
Augur

Hi, I can’t expose that because of how much RAM it would take. I can only expose a mathematical derivative of the pixels inside of the blob. Please try to work out what mathematical operation you need done and then I can implement it in C code.

Please keep in mind that memory on the microcontrollers is not as large as a PC. A blob can be the size of an image and we support VGA image sizes for find_blobs() so the RAM for that would be huge. Also, iterating over that array in python would be terribly slow.

Hi,
Yes, the memory is very limited in openmv.
But I wanna know the number of every column in blob.
Could you have better solutions?
Thank you.

I wanna know the number of pixels in every column of the blob.

Um, can you write the code you need to parase that pixel array you talked about and I’ll see what I can put in the C code.

I understand you need two cameras and have to take a view into account. However, if you do rotation correction on each cameras image then that removes that variable. So, then you just have two flat images from either side of the object to combine.

Anyway, work on the problem. Give me some code for it and I’ll see what’s valuable to add to the camera C code.

Mmm, number of pixels in every column is a reduction.

That’s kinda doable. It’s still a lot of RAM however. Please work on the full algorithm and see if we can get more reductions.

I can do like a flag which would enable population of an array value otherwise which would be null if the flag isn’t passed to the method.

Hi,
Good idea! I rethink my solution again. Thank you!
By the way, Could you add a parameter named pixels_distribution in find_blobs()?
If the function is called with an array parameter myarray[w] like image.find_blobs(,pixels_distribution=myarray,), the function will compute the number of pixels in every column and give myarray.
Otherwise doesn’t compute the number.
Okay?
Thanks.

Hi, your solution is perfect “I can do like a flag which would enable population of an array value otherwise which would be null if the flag isn’t passed to the method.”
Yes, adding a flag parameter in find_blob() and a method named pixels_distribution() which returns one-dimensional array.


Okay. I’ll add the binary image projections:

For both w and h. I can allocate an array that’s static for the width and height of the image and the accumulate into both. Then, after the accumulation I can find the start and stop of non zero bins and then and bin the bins into a histogram where you select how many bins of output you want. Finer detail means more RAM and processing time in python.

I thought about adding the contour but the problem is that I have to store the contour points in the MP heap which isn’t designed to deal with lots of small allocations and will greatly slow the algorithm down along with easily running out of RAM depending on if the image gets too complex.

I could allocate the contour on the frame buffer stack in a temporary heap but then it would still need to be transferred to the MicroPython heap and it would still require a lot of space.

Okay, it’s done.

Just add x_hist_bins_max= and y_hist_bins_max= to find_blobs().

Then you can get the histogram list via the x_hist_bins() and y_hist_bins() properties. X and Y are completely separate. So, enable one, or the other, or both. The returned value above is a python list of ints where the ints are the number of pixels across the object. For X this would be the height per x pixel location, for y this the width per y pixel location.

Later on I will make the returned value a histogram object so you can perform more interesting things on it. Right now it’s just a list. (Probably will work on this feature later when I get to doing histogram upgrades).

Note - this feature takes up a lot of heap RAM. On the M4 is definitely a no-go. On the M7 it’s probably okay. The H7 it’s not possible to break things.

The max bin numbers control how binning is done. If the value there is really large you get per pixel bins. If you make it smaller you get less than per pixel bins but less memory usage. Note that the number of bins returned will never be larger than the object width/height. So, the max is used a max limiter.

Please avoid using callbacks with this on… otherwise I have to keep multiple copies of the pixel lists in RAM. It can get heavy fast.
firmware.zip (916 KB)

Hi kwagyeman,
Wow,quick and good job!
I really appreciate your assistance!
These days I am very busy with exams.
This weekend I will have enough time to develop my project.
I will seek your instruction if I will encounter problems.

Thank you so much. :smiley: