openmv to arduino with template matching examples

General discussion about topics related to OpenMV.
arella
Posts: 8
Joined: Wed Jun 14, 2017 12:33 am

openmv to arduino with template matching examples

Postby arella » Wed Jun 14, 2017 12:54 am

Hi, I want to use camera to recognize the direction of the arrow( only tow options of left or right ) and send the result to the arduino UNO. I use the template-matching example in software, but come across some problems. First, When I run the example, it will show "fb alloc collision". I change search.ex to search.ds, and it is Ok. Why this will happen? Second, I want to figure out how to recognize the arrow, since I try this example and it seems both left and right arrows are recognized to be right. I mean, I store a left arrow as a template, and the cam seems to recognize both right and left arrow as similar to the template. Is my understanding of this example wrong? Could you help me to figure out the mistakes?
THANKS !!
User avatar
kwagyeman
Posts: 3056
Joined: Sun May 24, 2015 2:10 pm

Re: openmv to arduino with template matching examples

Postby kwagyeman » Wed Jun 14, 2017 1:39 am

Hmmm, that seems broken. Sorry about this. Template matching has not received any attention in a while I've been robustifying the API. Since template matching is not rotation invariant left and right arrows should be different.

Ibrahim, do you have any ideas on how to fix this?

...

The best thing I can think to do would be to shrink all the images down so you can just use template matching in normal mode. That is, being able to load a template image completely into RAM. To do this your template image needs to be about 80x60 pixels. Once you have the template image in RAM you can then search for it in your main image. However, the template has to be the same size as the picture in the main image. To fix any scaling issues you can pool the main image by doing img.mean_pool() (http://docs.openmv.io/library/omv.image ... ean_pooled) which will allows you to shrink the captured image. Once you've matched the main image and the template image you should be good.
Nyamekye,
arella
Posts: 8
Joined: Wed Jun 14, 2017 12:33 am

Re: openmv to arduino with template matching examples

Postby arella » Wed Jun 14, 2017 2:20 am

Oh, it works! Thank you! By the way, could you help me on the transformation problems from cam to arduino? I change the template-matching example to this
add the following code:

from pyb import UART
uart = UART(3, 9600)

and change the ending part to this: ( the part after this sentence r=img.find_template(template,0.70,.......) )
if r:
uart.write("r")
elif l:
uart.write("l")
else:
uart.write("n")

However, my arduino can only recieve value of -1 . Why did this happen? Is my usage of uart.write wrong?[
Thanks!
User avatar
kwagyeman
Posts: 3056
Joined: Sun May 24, 2015 2:10 pm

Re: openmv to arduino with template matching examples

Postby kwagyeman » Wed Jun 14, 2017 2:23 am

Please put your code in the code tag. I can't read it right now.
Nyamekye,
arella
Posts: 8
Joined: Wed Jun 14, 2017 12:33 am

Re: openmv to arduino with template matching examples

Postby arella » Wed Jun 14, 2017 2:47 am

Code: Select all

import time, sensor, image
from image import SEARCH_EX, SEARCH_DS
from pyb import UART
sensor.reset()
sensor.set_contrast(1)
sensor.set_gainceiling(16)
sensor.set_framesize(sensor.QQVGA)
sensor.set_pixformat(sensor.GRAYSCALE)
template = image.Image("/template.pgm")
template1=image.Image("/lefttemplate.pgm")
clock = time.clock()
uart = UART(3, 9600)
while (True):
    clock.tick()
    img = sensor.snapshot()
    r = img.find_template(template, 0.87, step=4, search=SEARCH_DS)
    l = img.find_template(template1, 0.87, step=4, search=SEARCH_DS)
    if r:
        uart.write("r")
    elif l:
        uart.write("l")
    else:
        uart.write("n")

    print(clock.fps()) 
THANK YOU!
User avatar
kwagyeman
Posts: 3056
Joined: Sun May 24, 2015 2:10 pm

Re: openmv to arduino with template matching examples

Postby kwagyeman » Wed Jun 14, 2017 2:57 am

A received value of -1 on your Arduino means it received nothing. Please verify which statement executes by putting a print after each one. Also, be aware of the character timeout on the UART. http://docs.openmv.io/library/pyb.UART.html#methods

Make sure to pass a large value for timeout and timeout char.

Anyway, it looks like you have the UART setup right. Characters should come out. Please verify that what if statement executes and then make sure your serial code is hooked up right.
Nyamekye,
arella
Posts: 8
Joined: Wed Jun 14, 2017 12:33 am

Re: openmv to arduino with template matching examples

Postby arella » Wed Jun 14, 2017 4:40 am

Thank you for the answer, but it still doesn't work. I add the below code:

Code: Select all

uart.init(9600, bits=8, parity=None, stop=1, *, timeout=10000, flow=0, timeout_char=10000, read_buf_len=64)
And it shows" syntax invalid."
Where is the mistake?
Thanks a lot!!
User avatar
deshipu
Posts: 74
Joined: Sat Jan 09, 2016 5:36 am
Location: Zürich
Contact:

Re: openmv to arduino with template matching examples

Postby deshipu » Wed Jun 14, 2017 1:20 pm

That lone "*" is a syntax error. Remove it.
User avatar
iabdalkader
Posts: 845
Joined: Sun May 24, 2015 3:53 pm

Re: openmv to arduino with template matching examples

Postby iabdalkader » Wed Jun 14, 2017 2:47 pm

I see you got that working, going to answer your questions anyway:
* FB alloc collision means the cam ran out of memory (probably using a big template)
* EX search uses twice as much memory as DS, that's why DS works and EX doesn't.
* If left and right match, check the threshold, with a very low threshold anything will match the template.
mjs513
Posts: 74
Joined: Sun Apr 30, 2017 12:52 pm

Re: openmv to arduino with template matching examples

Postby mjs513 » Wed Jun 14, 2017 4:38 pm

Not sure if you got this working, but recommend two things. Check that the baud rate on the Arduino matches the baudrate in your openmv sketch. I also added a slight delay after I send the characters to the Arduino.
arella
Posts: 8
Joined: Wed Jun 14, 2017 12:33 am

Re: openmv to arduino with template matching examples

Postby arella » Thu Jun 15, 2017 4:17 am

deshipu wrote:
Wed Jun 14, 2017 1:20 pm
That lone "*" is a syntax error. Remove it.
I remove it, but it still shows syntax invalid.......
arella
Posts: 8
Joined: Wed Jun 14, 2017 12:33 am

Re: openmv to arduino with template matching examples

Postby arella » Thu Jun 15, 2017 4:21 am

iabdalkader wrote:
Wed Jun 14, 2017 2:47 pm
I see you got that working, going to answer your questions anyway:
* FB alloc collision means the cam ran out of memory (probably using a big template)
* EX search uses twice as much memory as DS, that's why DS works and EX doesn't.
* If left and right match, check the threshold, with a very low threshold anything will match the template.
Oh, I see. It now can recognize the arrow, but my arduino can't receive it.........
arella
Posts: 8
Joined: Wed Jun 14, 2017 12:33 am

Re: openmv to arduino with template matching examples

Postby arella » Thu Jun 15, 2017 5:09 am

mjs513 wrote:
Wed Jun 14, 2017 4:38 pm
Not sure if you got this working, but recommend two things. Check that the baud rate on the Arduino matches the baudrate in your openmv sketch. I also added a slight delay after I send the characters to the Arduino.
Add a delay to the openmv IDE? I add a delay to arduino, and it still doesn't work........
User avatar
kwagyeman
Posts: 3056
Joined: Sun May 24, 2015 2:10 pm

Re: openmv to arduino with template matching examples

Postby kwagyeman » Thu Jun 15, 2017 10:48 am

Post your code.
Nyamekye,
arella
Posts: 8
Joined: Wed Jun 14, 2017 12:33 am

Re: openmv to arduino with template matching examples

Postby arella » Fri Jun 16, 2017 2:35 am

This is our code on openmv.

Code: Select all

import time, sensor, image
from image import SEARCH_EX, SEARCH_DS
from pyb import UART
sensor.reset()
sensor.set_contrast(1)
sensor.set_gainceiling(16)
sensor.set_framesize(sensor.QQVGA)
sensor.set_pixformat(sensor.GRAYSCALE)
template = image.Image("/template.pgm")
template1=image.Image("/lefttemplate.pgm")
clock = time.clock()
uart = UART(3, 9600)
while (True):
    clock.tick()
    img = sensor.snapshot()
    r = img.find_template(template, 0.87, step=4, search=SEARCH_DS)
    l = img.find_template(template1, 0.87, step=4, search=SEARCH_DS)
    if r:
        uart.write("r")
    elif l:
        uart.write("l")
    else:
        uart.write("n")

    print(clock.fps())
User avatar
kwagyeman
Posts: 3056
Joined: Sun May 24, 2015 2:10 pm

Re: openmv to arduino with template matching examples

Postby kwagyeman » Fri Jun 16, 2017 3:07 am

Code: Select all

import time, sensor, image
from image import SEARCH_EX, SEARCH_DS
from pyb import UART
sensor.reset()
sensor.set_contrast(1)
sensor.set_gainceiling(16)
sensor.set_framesize(sensor.QQVGA)
sensor.set_pixformat(sensor.GRAYSCALE)
template = image.Image("/template.pgm")
template1=image.Image("/lefttemplate.pgm")
clock = time.clock()
uart = UART(3, 9600, timeout_char = 1000)
while (True):
    clock.tick()
    img = sensor.snapshot()
    r = img.find_template(template, 0.87, step=4, search=SEARCH_DS)
    l = img.find_template(template1, 0.87, step=4, search=SEARCH_DS)
    if r:
        uart.write("r")
        print("r")
    elif l:
        uart.write("l")
        print("l")
    else:
        uart.write("n")
        print("n")

    print(clock.fps())
Please let me know what the above prints.
Nyamekye,
User avatar
kwagyeman
Posts: 3056
Joined: Sun May 24, 2015 2:10 pm

Re: openmv to arduino with template matching examples

Postby kwagyeman » Wed Jun 21, 2017 11:16 am

Can you let me know ow what it prints...
Nyamekye,
prasap1
Posts: 16
Joined: Wed Dec 12, 2018 4:37 am
Location: India

Re: openmv to arduino with template matching examples

Postby prasap1 » Thu Dec 13, 2018 2:39 am

Dear Ms OpenMV,

I am new to openmv. I am planning to work on image processing. I am stuck in identifying image when image processing is going through camera. May i know how to solve this problem by the by i have stored matching image in the sd card.

Below is the code i am using.. can you debug the code?

import time, sensor, image
from image import SEARCH_EX, SEARCH_DS

# Reset sensor
sensor.reset()

# Set sensor settings
sensor.set_contrast(1)
sensor.set_gainceiling(16)
# Max resolution for template matching with SEARCH_EX is QQVGA
sensor.set_framesize(sensor.QQVGA)
# You can set windowing to reduce the search image.
#sensor.set_windowing(((640-80)//2, (480-60)//2, 80, 60))
sensor.set_pixformat(sensor.GRAYSCALE)

# Load template.
# Template should be a small (eg. 32x32 pixels) grayscale image.
template = image.Image("/Remote1.PGM")

clock = time.clock()

# Run template matching
while (True):
clock.tick()
img = sensor.snapshot()

# find_template(template, threshold, [roi, step, search])
# ROI: The region of interest tuple (x, y, w, h).
# Step: The loop step used (y+=step, x+=step) use a bigger step to make it faster.
# Search is either image.SEARCH_EX for exhaustive search or image.SEARCH_DS for diamond search
#
# Note1: ROI has to be smaller than the image and bigger than the template.
# Note2: In diamond search, step and ROI are both ignored.
r = img.find_template(template, 0.70, step=4, search=SEARCH_EX) #, roi=(10, 0, 60, 60))
if r:
img.draw_rectangle(r)

print(clock.fps())
User avatar
kwagyeman
Posts: 3056
Joined: Sun May 24, 2015 2:10 pm

Re: openmv to arduino with template matching examples

Postby kwagyeman » Thu Dec 13, 2018 12:30 pm

Please start a new thread and repost using the

Code: Select all

 tags.
Nyamekye,
prasap1
Posts: 16
Joined: Wed Dec 12, 2018 4:37 am
Location: India

Re: openmv to arduino with template matching examples

Postby prasap1 » Fri Dec 21, 2018 11:19 am

hai sir i need your help for doing template matching and when the template is matched pin1 want to on, by using that i will control my process.
can u give the code for the same.

Thanks in advance
Regards
Prasath
User avatar
kwagyeman
Posts: 3056
Joined: Sun May 24, 2015 2:10 pm

Re: openmv to arduino with template matching examples

Postby kwagyeman » Sat Dec 22, 2018 7:02 pm

Ibrahim, can you help on this?
Nyamekye,
User avatar
iabdalkader
Posts: 845
Joined: Sun May 24, 2015 3:53 pm

Re: openmv to arduino with template matching examples

Postby iabdalkader » Wed Dec 26, 2018 11:27 am

prasap1 wrote:
Fri Dec 21, 2018 11:19 am
hai sir i need your help for doing template matching and when the template is matched pin1 want to on, by using that i will control my process.
can u give the code for the same.

Thanks in advance
Regards
Prasath
Hi, here's an example for pin control:

Code: Select all

import sensor, image
from image import SEARCH_EX, SEARCH_DS
from pyb import Pin

sensor.reset()
sensor.set_contrast(1)
sensor.set_gainceiling(16)
sensor.set_framesize(sensor.QQVGA)
sensor.set_pixformat(sensor.GRAYSCALE)

pin1 = Pin('P1', Pin.OUT_PP, Pin.PULL_DOWN)
pin1.value(0)
template = image.Image("/template.pgm")

while (True):
    img = sensor.snapshot()
    r = img.find_template(template, 0.70, step=4, search=SEARCH_EX)
    if r:
        pin1.value(1)
prasap1
Posts: 16
Joined: Wed Dec 12, 2018 4:37 am
Location: India

Re: openmv to arduino with template matching examples

Postby prasap1 » Fri Dec 28, 2018 11:14 am

iabdalkader wrote:
Wed Dec 26, 2018 11:27 am
prasap1 wrote:
Fri Dec 21, 2018 11:19 am
hai sir i need your help for doing template matching and when the template is matched pin1 want to on, by using that i will control my process.
can u give the code for the same.

Thanks in advance
Regards
Prasath
Hi, here's an example for pin control:

Code: Select all

import sensor, image
from image import SEARCH_EX, SEARCH_DS
from pyb import Pin

sensor.reset()
sensor.set_contrast(1)
sensor.set_gainceiling(16)
sensor.set_framesize(sensor.QQVGA)
sensor.set_pixformat(sensor.GRAYSCALE)

pin1 = Pin('P1', Pin.OUT_PP, Pin.PULL_DOWN)
pin1.value(0)
template = image.Image("/template.pgm")

while (True):
    img = sensor.snapshot()
    r = img.find_template(template, 0.70, step=4, search=SEARCH_EX)
    if r:
        pin1.value(1)

Thank you sir

Its working :)

But without computer how can i use this.
I can upload program into OpenMV?
User avatar
kwagyeman
Posts: 3056
Joined: Sun May 24, 2015 2:10 pm

Re: openmv to arduino with template matching examples

Postby kwagyeman » Fri Dec 28, 2018 12:49 pm

Yes, go to Tools -> Save Script to OpenMV Cam.

Then after the script has been saved click the: Tools -> Reset OpenMV Cam button.
Nyamekye,
prasap1
Posts: 16
Joined: Wed Dec 12, 2018 4:37 am
Location: India

Re: openmv to arduino with template matching examples

Postby prasap1 » Sat Dec 29, 2018 10:13 am

kwagyeman wrote:
Fri Dec 28, 2018 12:49 pm
Yes, go to Tools -> Save Script to OpenMV Cam.

Then after the script has been saved click the: Tools -> Reset OpenMV Cam button.
Thanks for quick support. :D

Return to “General Topics”

Who is online

Users browsing this forum: No registered users and 205 guests