Project Idea: Automatically point camera at target

OpenMV related project discussion.
ScottM
Posts: 85
Joined: Thu May 30, 2019 2:12 pm
Location: Ottawa, Canada

Project Idea: Automatically point camera at target

Postby ScottM » Wed Jul 03, 2019 12:40 pm

I plan to mount the camera on a pan/tilt base and use the pan/tilt shield. I want to find an object and automatically pan and tilt the camera to center the object in the FOV. I was thinking that the object could be an April Tag, but with memory limiting the frame size, I'm thinking that I could use a large red dot instead. Question: Is this easy to do and are there any samples I can study?

Here is how I imagine it working

1. Find the target (I want to use VGA if I can)
2. Get the coordinates of the center of the target
3. Compare those coordinates with the center of the FOV to determine how much to pan and tilt
4. Move pan and tilt motors (checking constantly to make sure target is now in the middle of FOV)

Also, I was hoping to use the LCD shield to show a live view, but I believe that it conflicts with the pan/tilt shield.

Scott
User avatar
kwagyeman
Posts: 3799
Joined: Sun May 24, 2015 2:10 pm

Re: Project Idea: Automatically point camera at target

Postby kwagyeman » Wed Jul 03, 2019 11:31 pm

Yeah, this is very easy. You just do a PID loop on each axis. Um, do you need help with this? You just use find blobs and then once you find the target blob you subtract the middle of the X/Y position of the image from the X/Y of the blob and then that's your error for the X/Y direction. Then you just multiply the error by a constant you have to find by trial and error and then set the servo position to the output of the previous function plus some offset. You do this for both the X and Y direction.
Nyamekye,
ScottM
Posts: 85
Joined: Thu May 30, 2019 2:12 pm
Location: Ottawa, Canada

Re: Project Idea: Automatically point camera at target

Postby ScottM » Thu Jul 04, 2019 10:00 am

I worked on this yesterday and got it working to detect a certain green square on the wall roughly 6 feet from the camera. I had to fiddle with the thresholds a bit, but the technique of selecting the target color patch with the mouse and using the LAB Color Space histograms makes it easy. I am having some trouble getting the exposure the way I want though. I'll detail that in another post. So far, I find the patch of green on the wall and output how many pixels it is to the left or right or if it is in the center zone (10 pixels wide). The next step is to put in the code for the pan/tilt servos to center the patch horizontally. I'm not sure yet how to do a PID loop on each axis (sounds like I'm tuning my 3D printer), so I'll research that.

I attached a picture of what the camera sees.

Here is my code.

Code: Select all

# My adaptation of the Single Color RGB565 Blob Tracking Example to detect a single green square
#
# Scott Murchison

import sensor, image, time, math, utime

threshold_index = 2 # 0 for red, 1 for yellow, 2 for green, 3 for blue

# Color Tracking Thresholds using Lab color model (L Min, L Max, A Min, A Max, B Min, B Max)

thresholds = [(30, 100, 15, 127, 15, 127),    # red_thresholds (30, 100, 15, 127, 15, 127)
              (70, 100, -30, 30, 60, 90),     # yellow_thresholds (70, 90, -30, 30, 60, 90)
              (75, 90, -60, -40, 0, 30),      # green_thresholds (30, 100, -64, -8, -32, 32)
              (20, 50, -20, 20, -45, -10)]    # blue_thresholds (0, 30, 0, 64, -128, 0)


sensor.reset()
sensor.set_pixformat(sensor.RGB565)
sensor.set_framesize(sensor.VGA)
sensor.set_windowing((400,400))
sensor.skip_frames(time = 1000)
sensor.set_auto_gain(False)           # must be turned off for color tracking
sensor.set_auto_whitebal(False)       # must be turned off for color tracking
sensor.set_auto_exposure(False)
sensor.set_brightness(0)              # range 0-31
sensor.set_saturation(3)              # range -3 to 3
sensor.set_gainceiling(2)             # 2,4,8,16,32,64,128
sensor.set_contrast(-1)               # range -3 to 3
clock = time.clock()

# Only blobs that with more pixels than "pixel_threshold" and more area than "area_threshold" are
# returned by "find_blobs" below. Change "pixels_threshold" and "area_threshold" if you change the
# camera resolution. "merge=True" merges all overlapping blobs in the image.

while(True):
    clock.tick()
    img = sensor.snapshot()
    for blob in img.find_blobs([thresholds[threshold_index]], pixels_threshold=100, area_threshold=100, merge=False):

        # These values are stable all the time.
        img.draw_rectangle(blob.rect())
        img.draw_cross(blob.cx(), blob.cy())
        # Extract the x,y coordinates of each color
        coordinates = (blob.cx(), blob.cy())
        print("X=%d   Y=%d" % coordinates)

        x_coord = blob.cx()
        y_coord = blob.cy()

        if x_coord < 195:
           h_offset = (200 - blob.cx())
           print ("patch is %d pixels left of center" % h_offset)
        elif x_coord > 205:
           h_offset = (blob.cx() - 200)
           print ("patch is %d pixels right of center" % h_offset)
        else:
           print ("patch is in center region")


        # insert a delay
        utime.sleep_ms(1000)
Attachments
Find green patch.jpg
Find green patch.jpg (17.58 KiB) Viewed 4280 times
User avatar
kwagyeman
Posts: 3799
Joined: Sun May 24, 2015 2:10 pm

Re: Project Idea: Automatically point camera at target

Postby kwagyeman » Tue Jul 09, 2019 2:01 am

Cool glad to see it working. Note I put some camera firmware upgrades in on the tip of the github commits which improve the image quality. I will be restarting firmware pushing now.
Nyamekye,
ScottM
Posts: 85
Joined: Thu May 30, 2019 2:12 pm
Location: Ottawa, Canada

Re: Project Idea: Automatically point camera at target

Postby ScottM » Tue Jul 16, 2019 4:58 pm

I'm not good with Github. How do I get the firmware upgrade to test image quality?
User avatar
kwagyeman
Posts: 3799
Joined: Sun May 24, 2015 2:10 pm

Re: Project Idea: Automatically point camera at target

Postby kwagyeman » Tue Jul 16, 2019 11:26 pm

Um, I can just build it for you. Mmm, what camera do you have? An H7 or M7?
Nyamekye,
ScottM
Posts: 85
Joined: Thu May 30, 2019 2:12 pm
Location: Ottawa, Canada

Re: Project Idea: Automatically point camera at target

Postby ScottM » Wed Jul 17, 2019 8:51 am

I have three H7's.
User avatar
kwagyeman
Posts: 3799
Joined: Sun May 24, 2015 2:10 pm

Re: Project Idea: Automatically point camera at target

Postby kwagyeman » Wed Jul 17, 2019 1:48 pm

Okay, I'll post a firmware tonight for you.
Nyamekye,
User avatar
iabdalkader
Posts: 1141
Joined: Sun May 24, 2015 3:53 pm

Re: Project Idea: Automatically point camera at target

Postby iabdalkader » Wed Jul 17, 2019 6:11 pm

I updated the development firmware images on github, you can download it from github:

https://github.com/openmv/openmv/tree/m ... re/OPENMV4
ScottM
Posts: 85
Joined: Thu May 30, 2019 2:12 pm
Location: Ottawa, Canada

Re: Project Idea: Automatically point camera at target

Postby ScottM » Thu Jul 18, 2019 1:47 pm

Thank you. First, is there a way to read the version of the firmware currently running on the camera? Also, I figured out that I have to use the Run Bootloader command in the IDE to update the firmware,but which files do I download to my PC first?
User avatar
kwagyeman
Posts: 3799
Joined: Sun May 24, 2015 2:10 pm

Re: Project Idea: Automatically point camera at target

Postby kwagyeman » Thu Jul 18, 2019 2:10 pm

Yes, please see the OMV module. You can get the firmware version and etc from it.
Nyamekye,
User avatar
kwagyeman
Posts: 3799
Joined: Sun May 24, 2015 2:10 pm

Re: Project Idea: Automatically point camera at target

Postby kwagyeman » Thu Jul 18, 2019 2:11 pm

Um, the firmware.bin file in the GitHub is the firmware for the camera.

OMV2 is the M4
OMV3 is the M7
OMV4 is the H7
Nyamekye,
ScottM
Posts: 85
Joined: Thu May 30, 2019 2:12 pm
Location: Ottawa, Canada

Re: Project Idea: Automatically point camera at target

Postby ScottM » Thu Jul 18, 2019 3:21 pm

There are 11 files in the OPENMV4 directory. Are you saying that I only need the firmware.bin file?

Also, what do you mean by "see the OMV module"?
ScottM
Posts: 85
Joined: Thu May 30, 2019 2:12 pm
Location: Ottawa, Canada

Re: Project Idea: Automatically point camera at target

Postby ScottM » Thu Jul 18, 2019 3:43 pm

I used the Run Bootloader command to install firmware.bin and it worked. I had the camera pointed at a sunny wall and it was overexposed with the helloworld example running. With the new firmware running, the exposure is better, i.e. darker with better contrast. I will run more tests on it.

Why not add something in the IDE to tell us what version of the firmware is running on the connected camera?
User avatar
kwagyeman
Posts: 3799
Joined: Sun May 24, 2015 2:10 pm

Re: Project Idea: Automatically point camera at target

Postby kwagyeman » Thu Jul 18, 2019 4:56 pm

Hi, the firmware version is literally displayed in the bottom right corner of the IDE.

The OMV module in python has the version number. See the API documentation.
Nyamekye,
ScottM
Posts: 85
Joined: Thu May 30, 2019 2:12 pm
Location: Ottawa, Canada

Re: Project Idea: Automatically point camera at target

Postby ScottM » Fri Jul 19, 2019 10:24 am

Hi,

Now I see it in the bottom right corner. Thanks for pointing this out.

Scott

Return to “Project Discussion”

Who is online

Users browsing this forum: No registered users and 1 guest