Okay, so that means the camera may move around with respect to the device. I thought the camera was mounted on a fixed plane looking down at an angle and the device was on a fixed plane too.
I only know how to solve for the rotation given the situation above. If the camera is allowed to move around then that means all the dots won’t be in the same plane anymore.
Mmm, well, you might be able to solve for x and y rotation by looking at what points of off center in the x and y directions and using the same trick I did above.
I.e. compute the centroids of all points. This is the sum of all x positions divided by 4 and the same and all y positions divided by 4. Then, for the y rotation get the max delta x from the centroids an do the opposite for the x rotation. The delta numbers will shrink as the object is rotated more.
Anyway, the above doesn’t give you exact rotation but will let you know if the objects profile appears smaller that it should be.