openmv to arduino with template matching examples

Hi, I want to use camera to recognize the direction of the arrow( only tow options of left or right ) and send the result to the arduino UNO. I use the template-matching example in software, but come across some problems. First, When I run the example, it will show “fb alloc collision”. I change search.ex to search.ds, and it is Ok. Why this will happen? Second, I want to figure out how to recognize the arrow, since I try this example and it seems both left and right arrows are recognized to be right. I mean, I store a left arrow as a template, and the cam seems to recognize both right and left arrow as similar to the template. Is my understanding of this example wrong? Could you help me to figure out the mistakes?
THANKS !!

Hmmm, that seems broken. Sorry about this. Template matching has not received any attention in a while I’ve been robustifying the API. Since template matching is not rotation invariant left and right arrows should be different.

Ibrahim, do you have any ideas on how to fix this?

The best thing I can think to do would be to shrink all the images down so you can just use template matching in normal mode. That is, being able to load a template image completely into RAM. To do this your template image needs to be about 80x60 pixels. Once you have the template image in RAM you can then search for it in your main image. However, the template has to be the same size as the picture in the main image. To fix any scaling issues you can pool the main image by doing img.mean_pool() (image — machine vision — MicroPython 1.15 documentation) which will allows you to shrink the captured image. Once you’ve matched the main image and the template image you should be good.

Oh, it works! Thank you! By the way, could you help me on the transformation problems from cam to arduino? I change the template-matching example to this
add the following code:

from pyb import UART
uart = UART(3, 9600)

and change the ending part to this: ( the part after this sentence r=img.find_template(template,0.70,…) )
if r:
uart.write(“r”)
elif l:
uart.write(“l”)
else:
uart.write(“n”)

However, my arduino can only recieve value of -1 . Why did this happen? Is my usage of uart.write wrong?[
Thanks!

Please put your code in the code tag. I can’t read it right now.

import time, sensor, image
from image import SEARCH_EX, SEARCH_DS
from pyb import UART
sensor.reset()
sensor.set_contrast(1)
sensor.set_gainceiling(16)
sensor.set_framesize(sensor.QQVGA)
sensor.set_pixformat(sensor.GRAYSCALE)
template = image.Image("/template.pgm")
template1=image.Image("/lefttemplate.pgm")
clock = time.clock()
uart = UART(3, 9600)
while (True):
    clock.tick()
    img = sensor.snapshot()
    r = img.find_template(template, 0.87, step=4, search=SEARCH_DS)
    l = img.find_template(template1, 0.87, step=4, search=SEARCH_DS)
    if r:
        uart.write("r")
    elif l:
        uart.write("l")
    else:
        uart.write("n")

    print(clock.fps())

THANK YOU!

A received value of -1 on your Arduino means it received nothing. Please verify which statement executes by putting a print after each one. Also, be aware of the character timeout on the UART. class UART – duplex serial communication bus — MicroPython 1.15 documentation

Make sure to pass a large value for timeout and timeout char.

Anyway, it looks like you have the UART setup right. Characters should come out. Please verify that what if statement executes and then make sure your serial code is hooked up right.

Thank you for the answer, but it still doesn’t work. I add the below code:

uart.init(9600, bits=8, parity=None, stop=1, *, timeout=10000, flow=0, timeout_char=10000, read_buf_len=64)

And it shows" syntax invalid."
Where is the mistake?
Thanks a lot!!

That lone “*” is a syntax error. Remove it.

I see you got that working, going to answer your questions anyway:

  • FB alloc collision means the cam ran out of memory (probably using a big template)
  • EX search uses twice as much memory as DS, that’s why DS works and EX doesn’t.
  • If left and right match, check the threshold, with a very low threshold anything will match the template.

Not sure if you got this working, but recommend two things. Check that the baud rate on the Arduino matches the baudrate in your openmv sketch. I also added a slight delay after I send the characters to the Arduino.

I remove it, but it still shows syntax invalid…

Oh, I see. It now can recognize the arrow, but my arduino can’t receive it…

Add a delay to the openmv IDE? I add a delay to arduino, and it still doesn’t work…

Post your code.

This is our code on openmv.

import time, sensor, image
from image import SEARCH_EX, SEARCH_DS
from pyb import UART
sensor.reset()
sensor.set_contrast(1)
sensor.set_gainceiling(16)
sensor.set_framesize(sensor.QQVGA)
sensor.set_pixformat(sensor.GRAYSCALE)
template = image.Image("/template.pgm")
template1=image.Image("/lefttemplate.pgm")
clock = time.clock()
uart = UART(3, 9600)
while (True):
    clock.tick()
    img = sensor.snapshot()
    r = img.find_template(template, 0.87, step=4, search=SEARCH_DS)
    l = img.find_template(template1, 0.87, step=4, search=SEARCH_DS)
    if r:
        uart.write("r")
    elif l:
        uart.write("l")
    else:
        uart.write("n")

    print(clock.fps())
import time, sensor, image
from image import SEARCH_EX, SEARCH_DS
from pyb import UART
sensor.reset()
sensor.set_contrast(1)
sensor.set_gainceiling(16)
sensor.set_framesize(sensor.QQVGA)
sensor.set_pixformat(sensor.GRAYSCALE)
template = image.Image("/template.pgm")
template1=image.Image("/lefttemplate.pgm")
clock = time.clock()
uart = UART(3, 9600, timeout_char = 1000)
while (True):
    clock.tick()
    img = sensor.snapshot()
    r = img.find_template(template, 0.87, step=4, search=SEARCH_DS)
    l = img.find_template(template1, 0.87, step=4, search=SEARCH_DS)
    if r:
        uart.write("r")
        print("r")
    elif l:
        uart.write("l")
        print("l")
    else:
        uart.write("n")
        print("n")

    print(clock.fps())

Please let me know what the above prints.

Can you let me know ow what it prints…

Dear Ms OpenMV,

I am new to openmv. I am planning to work on image processing. I am stuck in identifying image when image processing is going through camera. May i know how to solve this problem by the by i have stored matching image in the sd card.

Below is the code i am using… can you debug the code?

import time, sensor, image
from image import SEARCH_EX, SEARCH_DS

Reset sensor

sensor.reset()

Set sensor settings

sensor.set_contrast(1)
sensor.set_gainceiling(16)

Max resolution for template matching with SEARCH_EX is QQVGA

sensor.set_framesize(sensor.QQVGA)

You can set windowing to reduce the search image.

#sensor.set_windowing(((640-80)//2, (480-60)//2, 80, 60))
sensor.set_pixformat(sensor.GRAYSCALE)

Load template.

Template should be a small (eg. 32x32 pixels) grayscale image.

template = image.Image(“/Remote1.PGM”)

clock = time.clock()

Run template matching

while (True):
clock.tick()
img = sensor.snapshot()

find_template(template, threshold, [roi, step, search])

ROI: The region of interest tuple (x, y, w, h).

Step: The loop step used (y+=step, x+=step) use a bigger step to make it faster.

Search is either image.SEARCH_EX for exhaustive search or image.SEARCH_DS for diamond search

Note1: ROI has to be smaller than the image and bigger than the template.

Note2: In diamond search, step and ROI are both ignored.

r = img.find_template(template, 0.70, step=4, search=SEARCH_EX) #, roi=(10, 0, 60, 60))
if r:
img.draw_rectangle(r)

print(clock.fps())

Please start a new thread and repost using the

 tags.

hai sir i need your help for doing template matching and when the template is matched pin1 want to on, by using that i will control my process.
can u give the code for the same.

Thanks in advance
Regards
Prasath