Openmv AprilTag--Out of fast Frame Buffer Stack Memory!

Hello. I used Openmv M7, When I find AprilTag, I met a Error:" MemoryError: Out of fast Frame Buffer Stack Memory! Please reduce the resolution of the image you are running this algorithm on to bypass this issue!", if the distance is over about 40 centermeter, IDE will jump this error.
I want to increase the Frame rate, so I used ROI to Zoom out the range.
Here is my code:

# AprilTags Example
#
# This example shows the power of the OpenMV Cam to detect April Tags
# on the OpenMV Cam M7. The M4 versions cannot detect April Tags.

import sensor, image, time, math

sensor.reset()
sensor.set_pixformat(sensor.GRAYSCALE)
sensor.set_framesize(sensor.QQVGA) # we run out of memory if the resolution is much bigger...
sensor.skip_frames(time = 2000)
sensor.set_auto_gain(False)  # must turn this off to prevent image washout...
sensor.set_auto_whitebal(False)  # must turn this off to prevent image washout...
clock = time.clock()
roi = (0,0,160,120)
# Note! Unlike find_qrcodes the find_apriltags method does not need lens correction on the image to work.

# The apriltag code supports up to 6 tag families which can be processed at the same time.
# Returned tag objects will have their tag family and id within the tag family.

tag_families = 0
tag_families |= image.TAG16H5 # comment out to disable this family
tag_families |= image.TAG25H7 # comment out to disable this family
tag_families |= image.TAG25H9 # comment out to disable this family
tag_families |= image.TAG36H10 # comment out to disable this family
tag_families |= image.TAG36H11 # comment out to disable this family (default family)
tag_families |= image.ARTOOLKIT # comment out to disable this family

# What's the difference between tag families? Well, for example, the TAG16H5 family is effectively
# a 4x4 square tag. So, this means it can be seen at a longer distance than a TAG36H11 tag which
# is a 6x6 square tag. However, the lower H value (H5 versus H11) means that the false positve
# rate for the 4x4 tag is much, much, much, higher than the 6x6 tag. So, unless you have a
# reason to use the other tags families just use TAG36H11 which is the default family.

def family_name(tag):
    if(tag.family() == image.TAG16H5):
        return "TAG16H5"
    if(tag.family() == image.TAG25H7):
        return "TAG25H7"
    if(tag.family() == image.TAG25H9):
        return "TAG25H9"
    if(tag.family() == image.TAG36H10):
        return "TAG36H10"
    if(tag.family() == image.TAG36H11):
        return "TAG36H11"
    if(tag.family() == image.ARTOOLKIT):
        return "ARTOOLKIT"

while(True):
    clock.tick()
    img = sensor.snapshot()
    apriltags = img.find_apriltags(roi = roi)#, families=tag_families)
    if apriltags:
        for tag in apriltags:
        # defaults to TAG36H11 without "families".
            x = tag.cx()-tag.w()//2-5
            y = tag.cy()-tag.h()//2-5
            w1 = tag.w()+10
            h1 = tag.h()+10
            if x<0:
                x = 0
            if y<0:
                y = 0
            if w1>160:
                w1 = 160
            if h1 >120:
                h1 = 120
            roi = (x,y,w1,h1)
            print("roi:",roi)
            print("apriltag:",tag.rect())
            img.draw_rectangle(tag.rect(), color = (255, 0, 0))
            img.draw_cross(tag.cx(), tag.cy(), color = (0, 255, 0))
            print_args = (family_name(tag), tag.id(), (180 * tag.rotation()) / math.pi)
            print("Tag Family %s, Tag ID %d, rotation %f (degrees)" % print_args)
    else:
        roi = (0,0,160,120)
    print("roi two :",roi)
    print(clock.fps())

Please give me some suggestion! Thanks a lot

Note sure why you are getting the error. Once suggestion. Limit the minimum ROI size to 32x32 pixels. There’s no lower limit on your math right now. This can cause issues.

I limited the minimum ROI size to 32x32 pixels or 40x40 pixels, sometimes, it can work, but sometimes it can’t. :cry: :cry:
Sometimes, As it run, it cause issues. But sometimes, it can run for a while, suddenly, it cause issues.
Here is the latest code:

# AprilTags Example
#
# This example shows the power of the OpenMV Cam to detect April Tags
# on the OpenMV Cam M7. The M4 versions cannot detect April Tags.

import sensor, image, time, math

sensor.reset()
sensor.set_pixformat(sensor.GRAYSCALE)#GRAYSCALE
sensor.set_framesize(sensor.QQVGA) # we run out of memory if the resolution is much bigger...
sensor.skip_frames(time = 2000)
sensor.set_auto_gain(False)  # must turn this off to prevent image washout...
sensor.set_auto_whitebal(False)  # must turn this off to prevent image washout...
clock = time.clock()

roi = (0,0,160,120)
# Note! Unlike find_qrcodes the find_apriltags method does not need lens correction on the image to work.

# The apriltag code supports up to 6 tag families which can be processed at the same time.
# Returned tag objects will have their tag family and id within the tag family.

tag_families = 0
tag_families |= image.TAG16H5 # comment out to disable this family
tag_families |= image.TAG25H7 # comment out to disable this family
tag_families |= image.TAG25H9 # comment out to disable this family
tag_families |= image.TAG36H10 # comment out to disable this family
tag_families |= image.TAG36H11 # comment out to disable this family (default family)
tag_families |= image.ARTOOLKIT # comment out to disable this family

# What's the difference between tag families? Well, for example, the TAG16H5 family is effectively
# a 4x4 square tag. So, this means it can be seen at a longer distance than a TAG36H11 tag which
# is a 6x6 square tag. However, the lower H value (H5 versus H11) means that the false positve
# rate for the 4x4 tag is much, much, much, higher than the 6x6 tag. So, unless you have a
# reason to use the other tags families just use TAG36H11 which is the default family.

def family_name(tag):
    if(tag.family() == image.TAG16H5):
        return "TAG16H5"
    if(tag.family() == image.TAG25H7):
        return "TAG25H7"
    if(tag.family() == image.TAG25H9):
        return "TAG25H9"
    if(tag.family() == image.TAG36H10):
        return "TAG36H10"
    if(tag.family() == image.TAG36H11):
        return "TAG36H11"
    if(tag.family() == image.ARTOOLKIT):
        return "ARTOOLKIT"

while(True):
    clock.tick()
    print(roi)
    img = sensor.snapshot()
    apriltags = img.find_apriltags(roi, families=tag_families)
    if apriltags:
        for tag in apriltags:
        # defaults to TAG36H11 without "families".
            x = tag.cx()-tag.w()//2-5
            y = tag.cy()-tag.h()//2-5
            w1 = tag.w()+10
            h1 = tag.h()+10
            if x<0:
                x = 0
            if y<0:
                y = 0
#            if w1>80:
#                w1 = 80
#            if h1 >60:
#                h1 = 60
            if w1>160:
                w1 = 160
            if w1<40:
                w1 =40
            if h1<40:
                h1= 40
            if h1 >120:
                h1 = 120
            roi = (x,y,w1,h1)
            print("roi:",roi)
            print("apriltag:",tag.rect())
            img.draw_rectangle(tag.rect(), color = (255, 0, 0))
            img.draw_cross(tag.cx(), tag.cy(), color = (0, 255, 0))
            print_args = (family_name(tag), tag.id(), (180 * tag.rotation()) / math.pi)
            print("Tag Family %s, Tag ID %d, rotation %f (degrees)" % print_args)
    else:
        roi = (0,0,160,120)
    print(clock.fps())

Are there any details I ignored ? Please give me some suggestion! Thanks a lot

Mmm, I’m not sure what the error is. I’d just do this for now, wrap the call in a try except block and just ignore the error when it happens. The memory allocation footprint of apriltags is constant… so, the only reason I can think you are getting this error is because of the changing roi settings.

Hi,

I’m facing the same issue with an H7 cam. April tag detection is done continuously and ROI alternates between the 2 halves of the image (160x120). Sometimes the Out of fast Frame Buffer Stack Memory! error occurred and I don’t know why.

Here are my 2 ROIs :

roi_april = [(0, 0, 80, 120),
                 (80, 0, 80, 120)] #(x, y, w, h)

Here is what I’ve done to discard the error :

try:
        tags = img.find_apriltags(roi=roi_april[roi_april_index], families=TAG16H5)
 except MemoryError as e :
        f = open("log.txt", 'a+')
        f.write("Find april tag:"+ str(e) + "\r\n")
        f.close()
        tags = []
        pass

So now it does not crash anymore. However when the MemoryError happens, it seems that the CAM H7 freezes more than 1s in this portion of code. It can not be the write in file, can it ? So when this type of error occurred is it normal that so much time is spent in the function ? Does it have a settable timeout value ?

Waiting for your help.

Best regards,

Hi, the Apriltags algorithm has no hook to block all out of memory errors. The algorithm internally uses a heap which can be exhausted. We have tried to handle many of the out of memory errors… But, for ones we cannot recover from when they happen you get this error.

You should just not print the error to a log and keep going.

There’s not really a fix for this. We don’t have enough RAM onboard to make running out of memory improbable for the algorithm. It will happen depending on the scene.

Mhh too bad to hear that, but ok i can understand.

Concerning the lost time I observed, 1s is pretty big. Do you think it is because of the log ? Or could it come from the find_apriltag function itself ?

Probably the logging. Most likely an erase happened which blocks for a while.

Ok I will remove the logging and see what happens. Thank you.

what is the biggest size of sensor.set_framesize() for apriltag in openmv4 H7 plus?
i find the FHD with ov5640 can’t work for apriltag。
hope apriltag works well with ov5640 in maxsize of framesize

Hi, there’s no frame size limit but a CPU limit and RAM limit.

BTW, even a desktop PC struggles with that much resolution for AprilTags…