How use apriltag rotations to correct image rotation

Dear all,

I use 4x 16H5 ApriTtags with size of (90x90) px on a plane surface at the corners of a rectangle area.
Like i can see them with different view angle, i would like to rotate image by using the apriltag informations.

After locating april tags, i use x_rotation, y_rotation, z_rotation in radians of each april tag.
Then, i convert one of them to degrees for using them in rotation correction of image like :

	sensor.snapshot().rotation_corr( x_rotation = math.degrees( apriltag.x_rotation() ), \
                                             	y_rotation = math.degrees( apriltag.y_rotation() ), \
	                                        z_rotation = math.degrees( apriltag.z_rotation() ) )

but the rotation does not work, specially in z rotation.
The values are effectively weird, like close to 180° on x_rotation…
i tried to swap x_rotation and y_rotation but without better result.

Did i badly understand everything ?

Thanks,

This would only work if the tag was directly in the center of the field of view. If you have 4 tags then I’m guessing the tag is not centered.

The rotation of the tag is the rotation in it’s own coordinate space.

Ok thanks.
Is this correct for you :

	....
	sensor.set_windowing( (apriltag.cx(), apriltag.cy(), width, height) ) 
	....
	sensor.snapshot().rotation_corr( x_rotation = 180-math.degrees( apriltag.x_rotation() ), \
                                             	y_rotation = 360-math.degrees( apriltag.y_rotation() ), \
	                                        z_rotation = math.degrees( apriltag.z_rotation() )  )

What is your point ?

No, it’s a 3D math problem. Unless everything but z translation is all 0 you can rotate the image without issue.

Think of the camera in 3D space and the tag in 3D space. Rotations make sense based on the translation of where they are in 3D space.

Ok, it means the use of z rotation is only ok like :

	....
	sensor.set_windowing( (apriltag.cx(), apriltag.cy(), width, height) ) 
	....
	sensor.snapshot().rotation_corr( x_rotation = 0, \
                                             	y_rotation = 0, \
	                                        z_rotation = math.degrees( apriltag.z_rotation() )  )

because the center of image coordinate is only aligned with z axis in this case, but it is not at the same place of apriltag coordinate.

Thanks,

Yes. If you want to move it then you need to write some matrix multiply code to translate and rotate the rotation vector in 3D space.

I would like to use another strategy by using the apriltag rotations to calculate new coordinates of 3 other points known in april tag coordinate.
The idea is to have 4x corners and use rotation_corr function of image.

But i am not sure how to well use the rotations, i mean :

  • in which order i have to use rotations : point in image coordinate = Rz.Ry.Rx . Point in april tag coordonate + TxTy of apriltag
  • or does it exist transformation matrix calculation library ?

Thanks,

Hi, the AprilTag rotation and localization is not well thought out right now. I need to update it to export the homography matrix for use.

Rotation correction needs to be updated to also include 3x3 matrix support.

Anyway, if you pass 4 corners to the rotation correction method it will zoom in and set the frame buffer to those 4 corners streched to fill the frame. The example scripts detail what you have to do.

Yes sure, i use a crop after rotation_corr function to get always the same area (i hope).
This discussion well helped me and i hope it will for other too: https://forums.openmv.io/viewtopic.php?t=538

I could get these problems :

  • sometimes apriltag rotation (x,y,z) are all at -180° and dont change (seems to be a bug)
  • varation on x rotation around 10° without moving april tag
  • goodness is always at 0
  • i dont manage to increase decision_margin more 0.35 by increasing size of apriltag or resolution
  • set_lens_corr is not implemented for ov5640 if i dont say a mistake
  • the result is not very precise due to zoom problem

Thanks,

Hi, I will be massively improving lens_corr() and rotation_corr() soon. set_lens_corr() for the OV5640 is shading correction. It’s not needed.

Regarding the other errors with the AprilTag… sounds like gimble lock when the rotations don’t change.

My goal with the AprilTag code wasn’t necessarily to support 3D localization of them. However, I will improve the library for this.