I have been using AprilTag tracking with the OpenMV in my work and thought I’d share a simple trick I have been using to increase the frame rate. The idea is that once you have found a tag in one frame, instead of running the algorithm on the whole of the next frame you search only a region of interest defined using the .rect attribute of the found tag:
# AprilTags with Dynamic ROI Example
# This script will track a single tag by searching a continuosly redefined (dynamic) ROI
# which gives and improved frame rate relative to searching the entirety of each frame.
import sensor, image, time
#constant factors defining size of Dynamic Region of Interest (ROI)
#a is the x (widthways) factor
a = 0.33
#b is the y (lengthways) factor
b = 0.33
#calculate ROI
def roiCalc(a, b, tag, img):
img.draw_cross(tag.cx(), tag.cy(), color = (0, 255, 0))
trec = tag.rect()
#calculate ROI
ROI[0] = trec[0] - int(a * trec[2])
if ROI[0] < 0:
ROI[0] = 0
ROI[1] = trec[1] - int(b * trec[3])
if ROI[1] < 0:
ROI[1] = 0
ROI[2] = trec[2] + int(2 * a * trec[2])
if ROI[0] + ROI[2] > img.width():
ROI[2] = img.width() - ROI[0]
ROI[3] = trec[3] + int(2 * b * trec[3])
if ROI[1] + ROI[3] > img.height():
ROI[3] = img.height() - ROI[1]
#draw the ROI onto the current frame
img.draw_rectangle(ROI[0], ROI[1], ROI[2], ROI[3], color= (255, 0, 0))
return(ROI)
#initialise sensor
sensor.reset()
sensor.set_pixformat(sensor.GRAYSCALE)
sensor.set_framesize(sensor.QQVGA)
sensor.skip_frames(time = 2000)
#initialise ROI, tag found flag and frame counter
ROI = [None, None, None, None]
found = False
i = 0
clock = time.clock()
while(True):
clock.tick()
img = sensor.snapshot()
#if a tag was found in the previous frame (found == True) ...
#an ROI is defined, so search that ROI for the tag
if found:
#reset the found flag
found = False
#for an ROI a region of the image is copied in memory so memory errors
#occur, use a try except block so it doesn't stop running
try:
#search for the tag inside the ROI defined in the previous frame
tags = img.find_apriltags(roi=ROI, families=image.TAG36H11)
except(MemoryError, OSError):
print("Memory Error")
pass
#if a tag found inside the ROI
if len(tags) >= 1:
#tag was found so set flag to true
found = True
#parameters used to redefine the ROI based on the current tag position and size
ROI = roiCalc(a, b, tags[0], img)
#if no tags found pass
else:
pass
#... otherwise if no tags were found in the previous frame (found == False)
#search the whole of the current frame
else:
#search for the tag in the entire frame (no roi)
tags = img.find_apriltags(families=image.TAG36H11)
if len(tags) >= 1:
found = True
ROI = roiCalc(a, b, tags[0], img)
#if a tag was not found then print a statement to say so
if not found:
print("Tag NOT Found")
#print the frame rate once every 10 frames
if i % 10 == 0:
print("FRAME RATE = %.2f fps" % clock.fps())
#increment frame counter
i += 1
#reset frame counter
if i > 99:
i = 0
I use a flag (found) so that if you lose the tag (i.e. it is not found in the pre defined region of interest) then you search the whole of the next frame. You can also add a few lines so that if the tag isn’t found in the ROI you search the whole of the current frame. When you tell the system to search an ROI I think it must create a copy of that ROI in memory which can lead to memory errors on the H7 (you can easily fill the limited memory on this model) so I’ve placed try and except blocks into the script so that it doesn’t stop running when this happens. For this reason, this method also seems to work best on the H7 plus where I found that I could track a tag at 22 fps even at VGA resolution (your frame rate is going to depend on your tag size and magnification.)