face tracking with servos

Discussion related to "under the hood" OpenMV topics.
sertunc
Posts: 10
Joined: Thu Jun 30, 2016 1:18 am

face tracking with servos

Postby sertunc » Thu Jun 30, 2016 1:50 am

Hi,

I have ladyada's mini pan tilt kit
https://www.adafruit.com/product/1967
Image

I have ide version 1,5
also updated the firmware

I have tested servos with arduino, no problem there, they work.
also tested servos with arduino they work,
finally tested servos with servo_control.py from examples it also works (with some vibration)
but when i load face_tracking.py from examples, it finds my face but servos doesn't move
I feed servos with external 5v, connecting servo data line to P7 and P8, also common ground with external supply and openmv cam.

I noticed face tracking doesnt have import Servo, i have added it, still no juice.

In the youtube videos of Ibrahim Abdelkader have seen he is controling servos with arduino,
is it necessery ? or just openmv connection is enough ?
or may be with arduino are you getting more precise control of servos

Could you please help me to solve how to facetrack

Thanks in advance
User avatar
kwagyeman
Posts: 3832
Joined: Sun May 24, 2015 2:10 pm

Re: face tracking with servos

Postby kwagyeman » Thu Jun 30, 2016 9:43 am

Hi, we didn't write a script to control the servos while tracking a face. You'll have to write the code for that for now.

Please see the Board-Control -> servo_control.py script for how to drive the servos. Then, take that code and bring it into the facetracking script. You'll have to write the proportional control loop yourself.

Do you need help writing the proportional control loop for the servos?
Nyamekye,
sertunc
Posts: 10
Joined: Thu Jun 30, 2016 1:18 am

Re: face tracking with servos

Postby sertunc » Thu Jun 30, 2016 11:28 am

I would really appreciate help

My final project will be something like this :
https://youtu.be/ken1zjEb3OM

After solving pan/tilt I am thinking of forward and backward with wheels, with somehow checking object size (like if it's smaller go forward)

Thanks in advance
User avatar
kwagyeman
Posts: 3832
Joined: Sun May 24, 2015 2:10 pm

Re: face tracking with servos

Postby kwagyeman » Thu Jun 30, 2016 6:01 pm

Alright, the first step is to setup the hardware platform. Please build up everything and connect all the wires and so forth. Once you have everything setup please run the servo example script to have the servos pan around in a circle. Please tweak the servo test script until you're able to get at least 90 degrees of motion from both the pan and tilt axes. Once you know what the ranges are, please let me know what the min and max pulse width for both the pan and tilt servos are and which direction the system moves to for the min and max pulse widths. Once I know this information I can give you the rough starter code for proportional control.

(I'm assuming you understand how servos work and such here. If not please read up about them: http://www.seattlerobotics.org/guide/servos.html).
Nyamekye,
sertunc
Posts: 10
Joined: Thu Jun 30, 2016 1:18 am

Re: face tracking with servos

Postby sertunc » Fri Jul 01, 2016 12:42 am

Hi,

For tilt (horizontal)
Full Range is between 0 and 2400
When you raise (up) the value it moves counter-clockwise
mid point for 1800 (looks direct ahead)
useable range is 1400 (45 degrees to the left) to 2200 (45 degrees to the right)

For pan (vertical)
Full Range is between 800 (looks directly to the ceiling) and 2400 (looks down)
when u raise (up) the value it moves down
mid point for 1500 (looks direct ahead)
usable range is 1100 to 1900 (90-100 difference in degree i think)

Code: Select all

while(True):
    for i in range(800):
        s1.pulse_width(1400 + i)
        time.sleep(5)
    for i in range(800):
        s2.pulse_width(1100 + i)
        time.sleep(5)
    for i in range(800):
        s1.pulse_width(2200 - i)
        time.sleep(5)
    for i in range(800):
        s2.pulse_width(1900 - i)
        time.sleep(5)

also lower time.sleep() works better for my servos
User avatar
kwagyeman
Posts: 3832
Joined: Sun May 24, 2015 2:10 pm

Re: face tracking with servos

Postby kwagyeman » Fri Jul 01, 2016 10:58 am

Okay, open up the face_tracking.py script and replace this part:

Code: Select all

while (True):
    clock.tick()
    img = sensor.snapshot()
    # Extract keypoints using the detect face size as the ROI
    kpts2 = img.find_keypoints(threshold=KEYPOINTS_THRESH, normalized=NORMALIZED)

    if (kpts2):
        # Match the first set of keypoints with the second one
        c=image.match_descriptor(image.FREAK, kpts1, kpts2, threshold=MATCHING_THRESH)
        # If more than 10% of the keypoints match draw the matching set
        if (c[2]>25):
            img.draw_cross(c[0], c[1], size=5)
            img.draw_string(0, 10, "Match %d%%"%(c[2]))

    # Draw FPS
    img.draw_string(0, 0, "FPS:%.2f"%(clock.fps()))
With this:

Code: Select all

from pyb import Servo

x_pos = 1800 # default
y_pos = 1500 # default

x_min = 1400
x_max = 2200
y_max = 1900
y_min = 1100

x_gain = +1.00 # You have to tweak this value to stablize the control loop.
               # You also may need to invert the value if the system goes
               # in the wrong direction.
y_gain = +1.00 # You have to tweak this value to stablize the control loop.
               # You also may need to invert the value if the system goes
               # in the wrong direction.
xServo = Servo(0)
yServo = Servo(1)
xServo.pulse_width(x_pos)
yServo.pulse_width(y_pos)

while (True):
    clock.tick()
    img = sensor.snapshot()
    # Extract keypoints using the detect face size as the ROI
    kpts2 = img.find_keypoints(threshold=KEYPOINTS_THRESH, normalized=NORMALIZED)

    if (kpts2):
        # Match the first set of keypoints with the second one
        c=image.match_descriptor(image.FREAK, kpts1, kpts2, threshold=MATCHING_THRESH)
        # If more than 10% of the keypoints match draw the matching set
        if (c[2]>25):
            img.draw_cross(c[0], c[1], size=5)
            img.draw_string(0, 10, "Match %d%%"%(c[2]))

            x = c[0]
            y = c[1]
            
            x_error = x - (img.width()/2)
            y_error = y - (img.height()/2)
            
            x_pos += x_error * x_gain
            y_pos += y_error * y_gain
            
            # Clamp output between min and max
            if (x_pos > x_max):
                x_pos = x_max
            if (x_pos < x_min):
                x_pos = x_min
            
            # Clamp output between min and max
            if (y_pos > y_max):
                y_pos = y_max
            if (y_pos < y_min):
                y_pos = y_min

            xServo.pulse_width(x_pos)
            yServo.pulse_width(y_pos)
            
    # Draw FPS
    img.draw_string(0, 0, "FPS:%.2f"%(clock.fps()))
I haven't actually checked if the code compiles. However, you should be able to see what I am trying to do. Basically, the control loop will increment/decrement the xPosition/yPosition of each servo. The incrementing/decrementing is clamped between a min and max. The increment/decrement is based on the face position difference from the middle of the image. So, the more the face is away from the middle of the image the stronger the response.

Your job to to make sure that you have the x output assigned to the servo that pans horizontally and the y output connected to the servo than tilts vertically. Then you need to tweak the gain value until you find something that works really well. The gain value can either be negative or positive. If its wrong you'll notice the control loop breaking more or less right away. So, just flip the sign if that happens. If the gain value is too small the system will barely respond. If its too large the system will go crazy.

Anyway, that's about it. Let me know if you have problems. Also, note that I'm only updating the proportional control code when a face is tracked with enough confidence.
Nyamekye,
sertunc
Posts: 10
Joined: Thu Jun 30, 2016 1:18 am

Re: face tracking with servos

Postby sertunc » Fri Jul 01, 2016 1:46 pm

hi again,
after finding my face; it gives error;

Traceback (most recent call last):
File "<stdin>", line 125, in <module>
TypeError: can't convert float to int


the line 125 is:
xServo.pulse_width(x_pos)
yServo.pulse_width(y_pos)
User avatar
kwagyeman
Posts: 3832
Joined: Sun May 24, 2015 2:10 pm

Re: face tracking with servos

Postby kwagyeman » Fri Jul 01, 2016 1:54 pm

Do this:

xServo.pulse_width(int(x_pos))
yServo.pulse_width(int(y_pos))

Python does not convert floats to ints for you.
Nyamekye,
sertunc
Posts: 10
Joined: Thu Jun 30, 2016 1:18 am

Re: face tracking with servos

Postby sertunc » Fri Jul 01, 2016 2:13 pm

after your change, now it gives no error,
but the servos doesn't come to the starting position.

i have tried this code if something were wrong with the servos, but they were working.

Code: Select all

import time
from pyb import Servo

s1 = Servo(1) # P7
s2 = Servo(2) # P8

        s1.pulse_width(1800)
        s2.pulse_width(1500)
i also tried changing main.py with my new facetrack code and powering up with external power for more memory
still no juice
User avatar
kwagyeman
Posts: 3832
Joined: Sun May 24, 2015 2:10 pm

Re: face tracking with servos

Postby kwagyeman » Fri Jul 01, 2016 2:34 pm

Can you try initializing the servos at the start of the script?

Also, are there any errors being printed?
Nyamekye,
sertunc
Posts: 10
Joined: Thu Jun 30, 2016 1:18 am

Re: face tracking with servos

Postby sertunc » Fri Jul 01, 2016 2:44 pm

oh thank you
no errors but
initialising servos first works well
finally i can play with values :))
sertunc
Posts: 10
Joined: Thu Jun 30, 2016 1:18 am

Re: face tracking with servos

Postby sertunc » Sat Jul 02, 2016 7:23 am

hi,

it works good but sometimes it restarts or locks,
no idea why.

I also tried color tracking with servos it is way more smooother

finally,
how can i get , if the object were smaller or bigger a moment ago ?

thanks
User avatar
kwagyeman
Posts: 3832
Joined: Sun May 24, 2015 2:10 pm

Re: face tracking with servos

Postby kwagyeman » Sat Jul 02, 2016 2:33 pm

The face tracking script uses key points which can be unwieldy right now. Use the other face tracking script which just uses the HaarCascade. The face detection script. This one should be more robust.

As for smoother tracking, the resolution determines the refresh rate which determines the the control loop speed, i.e. the smoothness.

As for determining if the object got bigger or smaller. In the face detection script you'll get a bounding box instead of a point. SO, for that you just can look at the box area and if it was larger than previously then the object got closer. If the box area got smaller the object moved away.
Nyamekye,
luik
Posts: 25
Joined: Fri Oct 14, 2016 1:22 pm

Re: face tracking with servos

Postby luik » Fri Oct 14, 2016 1:43 pm

Hi everybody

I received my OpenMV Cam last week and I am really thrilled about it. I have a similar Project like sertunc. I'd like to build a heliostat. (a sun tracker, that moves a mirror in a way that it always reflects the sun to the same spot) Although not entirely new to coding and stuff like that I am having my troubles finding out how I get the ranges of my servos. I ran the servo script from examples, both servos move but in a sometimes very abrupt manner. I let the script run for like 5 minutes, it did not stop, i didn't get an output or something. Also in order to stop the script, I had to disconnect the cam, it just wouldnt stop :D

It would be great if you guys could give me a hint what to do with my servos in order to be (someday) maybe able to use my servo mount for my heliostat project :)
User avatar
kwagyeman
Posts: 3832
Joined: Sun May 24, 2015 2:10 pm

Re: face tracking with servos

Postby kwagyeman » Fri Oct 14, 2016 2:26 pm

Hi, can you post the script you're using? The documentation about servos is here:

http://docs.micropython.org/en/latest/p ... Servo.html

You may wish to set the servo pulse width instead of angle. As for getting the exact value of the min and max pulse width. You have to try mins and maxes out and see what works.
Nyamekye,
luik
Posts: 25
Joined: Fri Oct 14, 2016 1:22 pm

Re: face tracking with servos

Postby luik » Sat Oct 15, 2016 2:10 pm

Hey, thanks for the quick answer!

I've used this:

Code: Select all

# Servo Control Example
#
# This example shows how to use your OpenMV Cam to control servos.

import time
from pyb import Servo

s1 = Servo(1) # P7
s2 = Servo(2) # P8

while(True):
    for i in range(1000):
        s1.pulse_width(1000 + i)
        s2.pulse_width(1999 - i)
        time.sleep(10)
    for i in range(1000):
        s1.pulse_width(1999 - i)
        s2.pulse_width(1000 + i)
        time.sleep(10)
here is a short video that shows my servos in action..
https://www.youtube.com/watch?v=xT6uaKi ... e=youtu.be
User avatar
kwagyeman
Posts: 3832
Joined: Sun May 24, 2015 2:10 pm

Re: face tracking with servos

Postby kwagyeman » Sat Oct 15, 2016 2:17 pm

Okay, so you need to tune the pulse widths in that code. So, remove all the while loop stuff and just set the pulse widths manually.

s1.pulse_width(1000)
s2.pulse_width(1000)

Then check if the servo output is okay and keep changing the pulse widths until you determine the min and max range of the servos.
Nyamekye,
luik
Posts: 25
Joined: Fri Oct 14, 2016 1:22 pm

Re: face tracking with servos

Postby luik » Sat Oct 15, 2016 3:31 pm

super, now I finally got it.

my min for both seems to be at 900 and max is at 2300

I will try out the face tracking code, maybe I can use some of it for my sun tracking heliostat project :)



thank you very much!
dos
Posts: 4
Joined: Wed Jul 03, 2019 6:56 am

Re: face tracking with servos

Postby dos » Thu Jul 04, 2019 5:20 am

Hi everybody!
I want to do project like color tracking with OpenMV and 2 axis servo
Now i cant understand how to conrol by servos
Please, help to me to writing the code
I will be very gratefull for your helpiing:)
dos
Posts: 4
Joined: Wed Jul 03, 2019 6:56 am

Re: face tracking with servos

Postby dos » Thu Jul 04, 2019 5:53 am

For that moment i need help in coding.
I know how to connect the all wires but i can't understand how to add an example code "Color tracking -> single_color_rgb565_blob_tracking" so that the servos work in tandem with the camera OpenMV
Need any help! Thaks)
User avatar
kwagyeman
Posts: 3832
Joined: Sun May 24, 2015 2:10 pm

Re: face tracking with servos

Postby kwagyeman » Fri Jul 05, 2019 11:56 am

Hi, um, this whole forum post is about how to do this. Rather not explain it again. If you have a very specific question on an issue you are having I'm happy to answer.
Nyamekye,
dos
Posts: 4
Joined: Wed Jul 03, 2019 6:56 am

Re: face tracking with servos

Postby dos » Wed Jul 10, 2019 5:11 am

I used this code
Please, can you check this?
Maybe I have any little mistakes at the code
___________________________________________________________________

Code: Select all

# Single Color RGB565 Blob Tracking Example
#
# This example shows off single color RGB565 tracking using the OpenMV Cam.

import sensor, image, time, math
from pyb import Servo

x_pos = 1000 # default
y_pos = 1000 # default

x_min = 600
x_max = 2800
y_max = 1800
y_min = 900

x_gain = +1.00 # You have to tweak this value to stablize the control loop.
               # You also may need to invert the value if the system goes
               # in the wrong direction.
y_gain = +1.00 # You have to tweak this value to stablize the control loop.
               # You also may need to invert the value if the system goes
               # in the wrong direction.
xServo = Servo(1)
yServo = Servo(2)
xServo.pulse_width(x_pos)
yServo.pulse_width(y_pos)


threshold_index = 1 # 0 for red, 1 for green, 2 for blue

# Color Tracking Thresholds (L Min, L Max, A Min, A Max, B Min, B Max)
# The below thresholds track in general red/green/blue things. You may wish to tune them...
thresholds = [(30, 100, 15, 127, 15, 127), # generic_red_thresholds
              (30, 100, -64, -8, -32, 32), # generic_green_thresholds
              (0, 30, 0, 64, -128, 0)] # generic_blue_thresholds

sensor.reset()
sensor.set_pixformat(sensor.RGB565)
sensor.set_framesize(sensor.QVGA)
sensor.skip_frames(time = 2000)
sensor.set_auto_gain(False) # must be turned off for color tracking
sensor.set_auto_whitebal(False) # must be turned off for color tracking
clock = time.clock()

# Only blobs that with more pixels than "pixel_threshold" and more area than "area_threshold" are
# returned by "find_blobs" below. Change "pixels_threshold" and "area_threshold" if you change the
# camera resolution. "merge=True" merges all overlapping blobs in the image.

while(True):
    clock.tick()
    img = sensor.snapshot()
    for blob in img.find_blobs([thresholds[threshold_index]], pixels_threshold=200, area_threshold=200, merge=True):

        # These values are stable all the time.
        img.draw_rectangle(blob.rect())
        img.draw_cross(blob.cx(), blob.cy())

        x = blob.cx()
        y = blob.cy()
        x_error = x - (img.width()/2)
        y_error = y - (img.height()/2)

        x_pos += x_error * x_gain
        y_pos += y_error * y_gain

        # Clamp output between min and max
        if (x_pos > x_max):
            x_pos = x_max
        if (x_pos < x_min):
            x_pos = x_min

        # Clamp output between min and max
        if (y_pos > y_max):
            y_pos = y_max
        if (y_pos < y_min):
            y_pos = y_min

        xServo.pulse_width(int(x_pos))
        yServo.pulse_width(int(y_pos))
        # Note - the blob rotation is unique to 0-180 only.
        img.draw_keypoints([(blob.cx(), blob.cy(), int(math.degrees(blob.rotation())))], size=20)
    print(clock.fps())
User avatar
kwagyeman
Posts: 3832
Joined: Sun May 24, 2015 2:10 pm

Re: face tracking with servos

Postby kwagyeman » Wed Jul 10, 2019 11:25 am

Yeah, that's the basic idea. It looks like you are doing exactly the right thing.
Nyamekye,
dos
Posts: 4
Joined: Wed Jul 03, 2019 6:56 am

Re: face tracking with servos

Postby dos » Thu Jul 11, 2019 6:35 am

Thank you very much! Glad your reply :)
How can improve object tracking?
In my case, the tracking is rough, not smoothed
Can you please tell me what and how to fix it?
Glad to try any ideas!
User avatar
kwagyeman
Posts: 3832
Joined: Sun May 24, 2015 2:10 pm

Re: face tracking with servos

Postby kwagyeman » Thu Jul 11, 2019 1:56 pm

Hi dos, I don't know how to do everything.

If you want it to be smoother you should filter the input blob location through an average filter and then reduce the x_gain and y_gain so the system tracks more slowly.
Nyamekye,

Return to “Technical Discussion”

Who is online

Users browsing this forum: MSN [Bot] and 6 guests