Project Idea: Automatically point camera at target

I plan to mount the camera on a pan/tilt base and use the pan/tilt shield. I want to find an object and automatically pan and tilt the camera to center the object in the FOV. I was thinking that the object could be an April Tag, but with memory limiting the frame size, I’m thinking that I could use a large red dot instead. Question: Is this easy to do and are there any samples I can study?

Here is how I imagine it working

  1. Find the target (I want to use VGA if I can)
  2. Get the coordinates of the center of the target
  3. Compare those coordinates with the center of the FOV to determine how much to pan and tilt
  4. Move pan and tilt motors (checking constantly to make sure target is now in the middle of FOV)

Also, I was hoping to use the LCD shield to show a live view, but I believe that it conflicts with the pan/tilt shield.


Yeah, this is very easy. You just do a PID loop on each axis. Um, do you need help with this? You just use find blobs and then once you find the target blob you subtract the middle of the X/Y position of the image from the X/Y of the blob and then that’s your error for the X/Y direction. Then you just multiply the error by a constant you have to find by trial and error and then set the servo position to the output of the previous function plus some offset. You do this for both the X and Y direction.

I worked on this yesterday and got it working to detect a certain green square on the wall roughly 6 feet from the camera. I had to fiddle with the thresholds a bit, but the technique of selecting the target color patch with the mouse and using the LAB Color Space histograms makes it easy. I am having some trouble getting the exposure the way I want though. I’ll detail that in another post. So far, I find the patch of green on the wall and output how many pixels it is to the left or right or if it is in the center zone (10 pixels wide). The next step is to put in the code for the pan/tilt servos to center the patch horizontally. I’m not sure yet how to do a PID loop on each axis (sounds like I’m tuning my 3D printer), so I’ll research that.

I attached a picture of what the camera sees.

Here is my code.

# My adaptation of the Single Color RGB565 Blob Tracking Example to detect a single green square
# Scott Murchison

import sensor, image, time, math, utime

threshold_index = 2 # 0 for red, 1 for yellow, 2 for green, 3 for blue

# Color Tracking Thresholds using Lab color model (L Min, L Max, A Min, A Max, B Min, B Max)

thresholds = [(30, 100, 15, 127, 15, 127),    # red_thresholds (30, 100, 15, 127, 15, 127)
              (70, 100, -30, 30, 60, 90),     # yellow_thresholds (70, 90, -30, 30, 60, 90)
              (75, 90, -60, -40, 0, 30),      # green_thresholds (30, 100, -64, -8, -32, 32)
              (20, 50, -20, 20, -45, -10)]    # blue_thresholds (0, 30, 0, 64, -128, 0)

sensor.skip_frames(time = 1000)
sensor.set_auto_gain(False)           # must be turned off for color tracking
sensor.set_auto_whitebal(False)       # must be turned off for color tracking
sensor.set_brightness(0)              # range 0-31
sensor.set_saturation(3)              # range -3 to 3
sensor.set_gainceiling(2)             # 2,4,8,16,32,64,128
sensor.set_contrast(-1)               # range -3 to 3
clock = time.clock()

# Only blobs that with more pixels than "pixel_threshold" and more area than "area_threshold" are
# returned by "find_blobs" below. Change "pixels_threshold" and "area_threshold" if you change the
# camera resolution. "merge=True" merges all overlapping blobs in the image.

    img = sensor.snapshot()
    for blob in img.find_blobs([thresholds[threshold_index]], pixels_threshold=100, area_threshold=100, merge=False):

        # These values are stable all the time.
        # Extract the x,y coordinates of each color
        coordinates = (,
        print("X=%d   Y=%d" % coordinates)

        x_coord =
        y_coord =

        if x_coord < 195:
           h_offset = (200 -
           print ("patch is %d pixels left of center" % h_offset)
        elif x_coord > 205:
           h_offset = ( - 200)
           print ("patch is %d pixels right of center" % h_offset)
           print ("patch is in center region")

        # insert a delay

Find green patch.jpg

Cool glad to see it working. Note I put some camera firmware upgrades in on the tip of the github commits which improve the image quality. I will be restarting firmware pushing now.

I’m not good with Github. How do I get the firmware upgrade to test image quality?

Um, I can just build it for you. Mmm, what camera do you have? An H7 or M7?

I have three H7’s.

Okay, I’ll post a firmware tonight for you.

I updated the development firmware images on github, you can download it from github:

Thank you. First, is there a way to read the version of the firmware currently running on the camera? Also, I figured out that I have to use the Run Bootloader command in the IDE to update the firmware,but which files do I download to my PC first?

Yes, please see the OMV module. You can get the firmware version and etc from it.

Um, the firmware.bin file in the GitHub is the firmware for the camera.

OMV2 is the M4
OMV3 is the M7
OMV4 is the H7

There are 11 files in the OPENMV4 directory. Are you saying that I only need the firmware.bin file?

Also, what do you mean by “see the OMV module”?

I used the Run Bootloader command to install firmware.bin and it worked. I had the camera pointed at a sunny wall and it was overexposed with the helloworld example running. With the new firmware running, the exposure is better, i.e. darker with better contrast. I will run more tests on it.

Why not add something in the IDE to tell us what version of the firmware is running on the connected camera?

Hi, the firmware version is literally displayed in the bottom right corner of the IDE.

The OMV module in python has the version number. See the API documentation.


Now I see it in the bottom right corner. Thanks for pointing this out.