New user here. I’m a hobbyist, just coming up to speed with OpenMV. After searching this forum and the internet, I’d appreciate a little guidance on how to mask the image returned from the sensor. I am using a Nicla Vision and the Nicla Vision Blob Detection example from the Ardiuno site (https://docs.arduino.cc/tutorials/nicla-vision/blob-detection). I have modified it to use a black and white JPG (localmask.jpg) on the local storage. It’s the same size as the sensor image (320x240). The first 120 columns are black and the second 120 columns are white.
I’m able to read the mask in and do a .b_and() with image in the frame buffer but I’m not getting the results I expect. Because half my mask is black, I expect it to zero out half my sensor image and therefore find no blobs in that half of the image. However, it’s behaving as if there’s no mask. Do you see anything obviously wrong with this code?
mask = None
mask=image.Image("/localmask.jpg",copy_to_fb=True)
#mask=mask.to_bitmap(). # don’t think I need this because b_and() will accept a jpg.
while(True):
clock.tick() # Advances the clock
img = sensor.snapshot() # Takes a snapshot and saves it in memory
img = img.b_and(img, mask)
blobs = img.find_blobs([thresholdsApple, thresholdsBanana], area_threshold=2500, merge=True)
Follow up question: If this code was working, should I expect to see half the frame buffer viewer be black?
Hi, so, the masking functions are part of an old API I haven’t yet updated… they should be giving you an error right now on what you are trying to do but since they are older they are not enforcing updates we made on things.
In the mean-time, you need to match the pixformat of the two images:
import sensor, image, time
sensor.reset() # Reset and initialize the sensor.
sensor.set_pixformat(sensor.RGB565) # Set pixel format to RGB565 (or GRAYSCALE)
sensor.set_framesize(sensor.QVGA) # Set frame size to QVGA (320x240)
#sensor.skip_frames(time = 2000) # Wait for settings take effect.
clock = time.clock() # Create a clock object to track the FPS.
mask = None
mask=image.Image("/mask.jpeg",copy_to_fb=True)
mask.to_rgb565()
mask_fb = sensor.alloc_extra_fb(mask.width(), mask.height(), mask.format())
mask_fb.assign(mask)
#while(True):
#mask.assign(mask_fb)
#sensor.flush()
while(True):
clock.tick() # Advances the clock
img = sensor.snapshot() # Takes a snapshot and saves it in memory
img = img.b_and(mask_fb)
#blobs = img.find_blobs([thresholdsApple, thresholdsBanana], area_threshold=2500, merge=True)
Please note that if you save the image as a BMP file you can just do:
import sensor, image, time
sensor.reset() # Reset and initialize the sensor.
sensor.set_pixformat(sensor.RGB565) # Set pixel format to RGB565 (or GRAYSCALE)
sensor.set_framesize(sensor.QVGA) # Set frame size to QVGA (320x240)
#sensor.skip_frames(time = 2000) # Wait for settings take effect.
clock = time.clock() # Create a clock object to track the FPS.
while(True):
clock.tick() # Advances the clock
img = sensor.snapshot() # Takes a snapshot and saves it in memory
img = img.b_and("/mask.bmp")
#blobs = img.find_blobs([thresholdsApple, thresholdsBanana], area_threshold=2500, merge=True)
As this was how the old API was designed to pull the image from disk and apply it as a mask via reading scan lines.
Note that the mask parameter for these functions was added to give control per pixel when applying the AND operation. I designed this API before getting mentorship on how to write fast code and I would recommend not to use the mask parameter as there’s no way to accelerate that operation. When I get to updating the code if you use the mask parameter things will be much slower than normal.
This works in my code. Kwabena, thanks for taking the time to look at this and for the fast response!
Follow-up question: When you recommend to avoid using “the mask parameter as there’s no way to accelerate that operation”, which functions are you talking about? I’m assuming you mean img.b_and() since it’s the only use of the mask inside the while(true) loop. But isn’t that effectively saying avoid using b_and() and avoid masking? What will the recommended approach be for doing an “AND” of two frame buffers? My use case (in OpenCV) has been to mask out areas of the image for both performance reasons and to avoid false positives.
Hi, the b_and() method supports AND’ing with a mask and then it supports a second mask that acts a per pixel on/off.
E.g. b_and(second_image_which_is_typically_a_mask, mask=another_image_to_control_the_process)
So, the first image is typically the mask. However, let’s say you you aren’t using the AND operation to do a mask but are doing something more complex like a logical operation on the image (not sure why you’d want to do anything other than a mask but whatever).
Then the “mask” argument will turn the AND operation on/off per pixel based on the mask image passed.
This argument makes more sense for things like the ADD/OR operations. However, I added it to every method.
Anyway, I say not to use it because when I get to updating these methods the mask argument will only be supported via a slow code path.
