understanding find_rects

I am having a little trouble getting find_rects to find my squares.

Does find)rects find all rects including children rects insode larger rects or once it finds a large rect then it stops looking for children rects inside the larger 1.

Also what does the threshold arg for find_rects do??

Here’s a video showing my problem - YouTube

And this is the code that was running to make the video

import sensor, image, time

sensor.reset()
sensor.set_pixformat(sensor.RGB565) # grayscale is faster (160x120 max on OpenMV-M7)
sensor.set_framesize(sensor.QQVGA)
sensor.skip_frames(time = 2000)
clock = time.clock()

green_threshold = (7, 52, -128, -7, -128, 127)
white_threshold = (57, 83, -30, 85, -27, 48)
yellow_threshold = (46, 82, -30, 31, 28, 67)
blue_threshold = (4, 33, -37, 29, -59, -5)
red_threshold = (7, 49, 12, 56, 2, 57)

while True:
    clock.tick()
    img = sensor.snapshot()
    blue_fb = img.binary([blue_threshold], to_bitmap=True, copy=True)
    blue_fb.erode(1)
    white_fb = img.binary([white_threshold], to_bitmap=True, copy=True)
    white_fb.erode(1)
    green_fb = img.binary([green_threshold], to_bitmap=True, copy=True)
    green_fb.erode(1)
    yellow_fb = img.binary([yellow_threshold], to_bitmap=True, copy=True)
    yellow_fb.erode(1)
    red_fb = img.binary([red_threshold], to_bitmap=True, copy=True)
    red_fb.erode(1)
    yellow_fb.b_or(green_fb).b_or(white_fb).b_or(blue_fb).b_or(red_fb)
    for r in yellow_fb.find_rects(threshold = 1000):
        img.draw_circle(r.x()+r.w()//2, r.y()+r.h()//2, r.w()//2, color = (0, 255, 0))
    print(yellow_fb.compressed_for_ide(), end="")

    print("FPS %f" % clock.fps())

Hi, find rects will not find child rects inside of a larger rect. The method is based on the quad detector from apriltags. We just extracted it and used it by itself for finding rectangle. As for the threshold. After we find a rect (which there could be MANY), we walk the edges of the rect defined by it’s four points and perform the sobel operation to get the edge strength of the rect. Then we discard rects with too low of an edge strength. The threshold part is our own custom logic. So, if you lower the threshold you’ll get basically what apriltags code considers are rects. Since there’s no tag to look at however the method tends to find a lot of candidates.

Have you thought of using find_blobs()??? You can determine if a blob underneath is a rect based on how high the density() of a blob is along with if w()/h() are close to 1.

Have you thought of using find_blobs()??? You can determine if a blob underneath is a rect based on how high the density() of a blob is along with if w()/h() are close to 1.

Some food for though and I will have a play with it. I think running 6 find blobs to detect all the colours then extract from there what I am looking for will greatly lower frame rates to probably lower than usable. The find_blobs or find_rects is what is slow. If I threah out all the colors into binary the join them all with b_or then I only have to run find_rects once compared to find_blobs 6 times.


I think the big problem is that when it finds a large rect then it doesn’t look for child rect inside. I notice that when ever a big rect is detected this is what stops it finding my squares that I am looking for. I will play with higher threaholds to try to stop the find_rects seeing big rects. Is it maybe possible to add more args to the find_rects to give min and max width and hight and also aspect ratio.

Once it finds all the rects including the ones that I am looking for then it is easy for me to reject the ones that I don’t want like ones to big and ones too small and any aren’t in a 3x3 grid of similar size.

Threshold won’t help rejecting big rects. That’s done before thresholding rects by the apriltag code.

Note that find_blobs() also accepts a list of colors to track… It can find 16 colors at the same time.