AprilTag speed up with dynamic ROI

[ecdm2]

I have been using AprilTag tracking with the OpenMV in my work and thought I'd share a simple trick I have been using to increase the frame rate. The idea is that once you have found a tag in one frame, instead of running the algorithm on the whole of the next frame you search only a region of interest defined using the .rect attribute of the found tag:

Code: Select all

# AprilTags with Dynamic ROI Example
# This script will track a single tag by searching a continuosly redefined (dynamic) ROI
# which gives and improved frame rate relative to searching the entirety of each frame.

import sensor, image, time

#constant factors defining size of Dynamic Region of Interest (ROI)
#a is the x (widthways) factor
a = 0.33
#b is the y (lengthways) factor
b = 0.33

#calculate ROI
def roiCalc(a, b, tag, img):

    img.draw_cross(tag.cx(), tag.cy(), color = (0, 255, 0))
    trec = tag.rect()

    #calculate ROI
    ROI[0] = trec[0] - int(a * trec[2])
    if ROI[0] < 0:
        ROI[0] = 0
    ROI[1] = trec[1] - int(b * trec[3])
    if ROI[1] < 0:
        ROI[1] = 0
    ROI[2] = trec[2] + int(2 * a * trec[2])
    if ROI[0] + ROI[2] > img.width():
        ROI[2] = img.width() - ROI[0]
    ROI[3] = trec[3] + int(2 * b * trec[3])
    if ROI[1] + ROI[3] > img.height():
        ROI[3] = img.height() - ROI[1]

    #draw the ROI onto the current frame
    img.draw_rectangle(ROI[0], ROI[1], ROI[2], ROI[3], color= (255, 0, 0))
    return(ROI)

#initialise sensor
sensor.reset()
sensor.set_pixformat(sensor.GRAYSCALE)
sensor.set_framesize(sensor.QQVGA)
sensor.skip_frames(time = 2000)

#initialise ROI, tag found flag and frame counter
ROI = [None, None, None, None]
found = False
i = 0

clock = time.clock()

while(True):

    clock.tick()

    img = sensor.snapshot()

    #if a tag was found in the previous frame (found == True) ...
    #an ROI is defined, so search that ROI for the tag
    if found:

        #reset the found flag
        found = False

        #for an ROI a region of the image is copied in memory so memory errors
        #occur, use a try except block so it doesn't stop running
        try:
            #search for the tag inside the ROI defined in the previous frame
            tags = img.find_apriltags(roi=ROI, families=image.TAG36H11)

        except(MemoryError, OSError):
            print("Memory Error")
            pass

        #if a tag found inside the ROI
        if len(tags) >= 1:
            #tag was found so set flag to true
            found = True
            #parameters used to redefine the ROI based on the current tag position and size
            ROI = roiCalc(a, b, tags[0], img)
        #if no tags found pass
        else:
            pass

    #... otherwise if no tags were found in the previous frame (found == False)
    #search the whole of the current frame
    else:

        #search for the tag in the entire frame (no roi)
        tags = img.find_apriltags(families=image.TAG36H11)
        if len(tags) >= 1:
            found = True
            ROI = roiCalc(a, b, tags[0], img)

    #if a tag was not found then print a statement to say so
    if not found:
        print("Tag NOT Found")

    #print the frame rate once every 10 frames
    if i % 10 == 0:
        print("FRAME RATE = %.2f fps" % clock.fps())

    #increment frame counter
    i += 1
    #reset frame counter
    if i > 99:
        i = 0

I use a flag (found) so that if you lose the tag (i.e. it is not found in the pre defined region of interest) then you search the whole of the next frame. You can also add a few lines so that if the tag isn't found in the ROI you search the whole of the current frame. When you tell the system to search an ROI I think it must create a copy of that ROI in memory which can lead to memory errors on the H7 (you can easily fill the limited memory on this model) so I've placed try and except blocks into the script so that it doesn't stop running when this happens. For this reason, this method also seems to work best on the H7 plus where I found that I could track a tag at 22 fps even at VGA resolution (your frame rate is going to depend on your tag size and magnification.)