if i want to run AprilTag on FHD images with my H7 plus, what i have to do is upgrade my firmware from 4.5.2 to 4.5.6 and ide to lastest. yes or no?
That’s not required. We’ve supported FHD for AprilTags for a long time. Expect a sub-hz frame rate.
# This work is licensed under the MIT license.
# Copyright (c) 2013-2023 OpenMV LLC. All rights reserved.
# https://github.com/openmv/openmv/blob/master/LICENSE
#
# AprilTags Example
#
# This example shows the power of the OpenMV Cam to detect April Tags
# on the OpenMV Cam M7. The M4 versions cannot detect April Tags.
import sensor
import image
import time
import math
sensor.reset()
sensor.set_pixformat(sensor.GRAYSCALE)#《------------------change this
sensor.set_framesize(sensor.FHD)#《------------------change this
sensor.skip_frames(time=2000)
sensor.set_auto_gain(False) # must turn this off to prevent image washout...
sensor.set_auto_whitebal(False) # must turn this off to prevent image washout...
clock = time.clock()
# Note! Unlike find_qrcodes the find_apriltags method does not need lens correction on the image to work.
# The apriltag code supports up to 6 tag families which can be processed at the same time.
# Returned tag objects will have their tag family and id within the tag family.
tag_families = 0
#tag_families |= image.TAG16H5 # comment out to disable this family
#tag_families |= image.TAG25H7 # comment out to disable this family
#tag_families |= image.TAG25H9 # comment out to disable this family
#tag_families |= image.TAG36H10 # comment out to disable this family
tag_families |= image.TAG36H11 # comment out to disable this family (default family)
#tag_families |= image.ARTOOLKIT # comment out to disable this family
# What's the difference between tag families? Well, for example, the TAG16H5 family is effectively
# a 4x4 square tag. So, this means it can be seen at a longer distance than a TAG36H11 tag which
# is a 6x6 square tag. However, the lower H value (H5 versus H11) means that the false positive
# rate for the 4x4 tag is much, much, much, higher than the 6x6 tag. So, unless you have a
# reason to use the other tags families just use TAG36H11 which is the default family.
def family_name(tag):
if tag.family() == image.TAG16H5:
return "TAG16H5"
if tag.family() == image.TAG25H7:
return "TAG25H7"
if tag.family() == image.TAG25H9:
return "TAG25H9"
if tag.family() == image.TAG36H10:
return "TAG36H10"
if tag.family() == image.TAG36H11:
return "TAG36H11"
if tag.family() == image.ARTOOLKIT:
return "ARTOOLKIT"
while True:
clock.tick()
img = sensor.snapshot()
for tag in img.find_apriltags(
families=tag_families
): # defaults to TAG36H11 without "families".
img.draw_rectangle(tag.rect(), color=(255, 0, 0))
img.draw_cross(tag.cx(), tag.cy(), color=(0, 255, 0))
print_args = (family_name(tag), tag.id(), (180 * tag.rotation()) / math.pi)
print("Tag Family %s, Tag ID %d, rotation %f (degrees)" % print_args)
print(clock.fps())
Hi, I apologize for the late response. It appears the system does not work up to FHD (1920x1080). There’s no limiter on our firmware that prevents supporting this other than an issue of there being too many allocations.
The latest firmware can work up to HD (1280x720). You can also make available some more RAM by setting number of frame buffers to 1:
sensor.reset()
sensor.set_pixformat(sensor.GRAYSCALE)
sensor.set_framesize(sensor.HD)
sensor.skip_frames(time=2000)
sensor.set_auto_gain(False) # must turn this off to prevent image washout...
sensor.set_auto_whitebal(False) # must turn this off to prevent image washout...
sensor.set_framebuffers(1)
To get FHD working you’d have to mess with how memory allocations work for AprilTags. The basic problem is that there’s a heap and a hash table used by the algorithm that both need to more or less be as large as possible, but, are in contention for the same RAM. You have the following knobs you can control to unlock this:
- openmv/src/omv/boards/OPENMV4P/omv_boardconfig.h at master · openmv/openmv - Increasing the block size here allows for more points to be allocated on a list of points which AprilTags uses to track tags.
- openmv/src/omv/imlib/apriltag.c at master · openmv/openmv - Modifying how much extra space is reserved for the fb_alloc() stack which is used to store the hash table that will be needed to determine which point rings overlap.
The heap fb_alloc() stack on the H7 Plus can grow to 27 MB currently. So, you don’t need to modify that. Anyway, increasing the numbers in the code I linked to should make it operational. E.g. OMV_UMM_BLOCK_SIZE to 512 or 1024 and if that doesn’t work maybe try increasing the fb_alloc_need multiplier from 6 to 8.
Note that the algorithm will run at FHD after setting the number of frame buffers to 1. But, it will not find anything. This is because the reallocs() on very large list of point clusters are failing. When this happens the algorithm just tosses the point cluster which could have been a tag. You’ll have to play with the above values, as mentioned, to make it work. Increasing the OMV_UMM_BLOCK_SIZE helps because it increases the number of points that fit within a malloc block making realloc() less likely to fail. However, this also reduces the number of UMM heap blocks available as the amount of RAM available to use didn’t increase.
Note that things may not converge and the algorithm may just never work well at FHD given the amount of memory onboard the H7 Plus.
1 Like
thank you. don’t need to apologize for late. 
you said that:
E.g. OMV_UMM_BLOCK_SIZE to 512 or 1024 and if that doesn’t work maybe try increasing the fb_alloc_need multiplier from 6 to 8.
the file of omv_boardconfig.h and apriltag.c is on github, where the same file in my openmvH7plus?
Hi, I linked to the exact line in the files you need to modify above.
The relative path will be the same for a checked out repo.
what is repo？ is it in my openmv?
You have to clone this and build the firmware: openmv/openmv: OpenMV Camera Module
openmv/src at master · openmv/openmv
Note, there’s a minor issue with the H7 Plus right now I have a PR out to fix: ports/stm32: Fix MDMA initialization. by kwagyeman · Pull Request #2563 · openmv/openmv
The tip of master needs that PR to be fixed. So, you might want to work off a stable release git tag.