Detecting Letters

Hello! I am getting my OpenMV H7 on Monday, and I am trying to learn about the camera already, so I can prepare some code and try some things with the camera this week.

One thing I did not find is if it is possible, to detect Letters with the OpenMV H7. If yes, how would that work?


Thank you,
Finn :slight_smile:

Yep, I’ll have the IDE release out that enables this over the weekend. Just see the chars74K example. Under the CMSIS-NN stuff.

Note, we are working on porting Tensor Flow mobile to the OpenMV Cam which will be replacing our current frame work.

Hey! Thank you for the link :slight_smile:

If I just want certain characters to be seen, and not every single character, how would the code need to be modified? I can’t test it right now, that is why I am asking. Also, the camera will probably not arrive before Thursday…so if you could provide me with some coding help for the camera, I would be really happy :slight_smile:

And if I have the camera scanning these certain letters, how can I tell my Arduino Mega that there is a letter and it now has to perform an action, relating to this certain letter?

Thank you, Finn :slight_smile:

Hi, it just classifies characters. It’s a network. So, it assumes there’s a character in front of it. There’s no modifying the code. you have to re-train the net to do something else. A trival character check is to look for straight lines and check how many you see.

Ok. So how would it work, if I want the camera to look for letters, and if there is no letter, it just does nothing. But if there is a letter (one of the letters I want to see), it sees which one and sends some kind of notification to the Arduino Mega so that the Arduino executes a certain function, according to the letter seen?

Thank you,
Finn

I’d just use the line segment detector to find a cluster of lines and then use the network on the location where the lines are to classify the letter. This would be a very small change to the base code provided.

We will also be offering TensorFlow soon where you can train a network yourself using more modern tools than Caffe.

Ok, let’s say, I wanted to modify this code, so it compares the lines to the three letters. Let’s say, we want to detect A, as an example.

# Find Line Segments Example
#
# This example shows off how to find line segments in the image. For each line object
# found in the image a line object is returned which includes the line's rotation.

# find_line_segments() finds finite length lines (but is slow).
# Use find_line_segments() to find non-infinite lines (and is fast).

enable_lens_corr = False # turn on for straighter lines...

import sensor, image, time

sensor.reset()
sensor.set_pixformat(sensor.RGB565) # grayscale is faster
sensor.set_framesize(sensor.QQVGA)
sensor.skip_frames(time = 2000)
clock = time.clock()

# All lines also have `x1()`, `y1()`, `x2()`, and `y2()` methods to get their end-points
# and a `line()` method to get all the above as one 4 value tuple for `draw_line()`.

while(True):
    clock.tick()
    img = sensor.snapshot()
    if enable_lens_corr: img.lens_corr(1.8) # for 2.8mm lens...

    # `merge_distance` controls the merging of nearby lines. At 0 (the default), no
    # merging is done. At 1, any line 1 pixel away from another is merged... and so
    # on as you increase this value. You may wish to merge lines as line segment
    # detection produces a lot of line segment results.

    # `max_theta_diff` controls the maximum amount of rotation difference between
    # any two lines about to be merged. The default setting allows for 15 degrees.

    for l in img.find_line_segments(merge_distance = 0, max_theta_diff = 5):
        img.draw_line(l.line(), color = (255, 0, 0))
        print(l)

    print("FPS %f" % clock.fps())

How would I find out, that the lines detected are equal to the lines of the letter A? Is there some kind of library, that I could use?

Thank you, Finn

Hi, the CNN running on the H7 does that. All the lines do will tell you where the character is.

Please wait till the next OpenMV IDE release. We will have the net for you then. I’ve finished the windows builds. Doing the Linux ones now.

When will that new IDE be released? And what is that CNN you are talking about? Sorry, I am new to OpenMV :unamused:

Hi, the video with me detecting characters - chars74k. I’m working on the IDE release. OpenMV isn’t like a big company. Asking me questions literally delays the IDE release :wink: - because I’m the one doing it.

Um, if you are using windows I can give you the windows IDE link. I’ve already built it and published it to our staging server.

I’ve finished building windows/mac/linux32/linux64. I just need to fix an issue with raspberryPi which I will do tomorrow.

If you could send me the link, that would be awesome :slight_smile:

Oh, and I have another short question: Is there a proper and detailed documentation on how to connect the camera to the arduino board, and how to write the arduino code according to what the camera sees?

Thank you for all your help,
Finn

Hi, the point of the OpenMV Cam is to not use an Arduino. It can do quite a bit of board I/O control itself. Anyway, in the examples folder there are examples showing how to connect an Arduino.

Which IDE version do you want? Windows, Mac, Linux?

Windows, please. Thank you!

http://upload.openmv.io/openmv-ide-windows-2.2.0/openmv-ide-windows-2.2.0.exe

See the ML examples, chars74K, then use the Tools->Machine Vision->Model Explorer to copy the model over.

Thank you for the link, but I can’t install the new version. There is an error, that says: “Windows cannot access the specified device, path or file. You may not have the appropriate permission to access the item.”

Is it something I do wrong, or an error with that file?

Came here to look for an update regarding the raspi. Found this. Thanks!!

Installs fine on 3 PCs I tested… where does the error happen?

Also, please always post with like context on what an error is. I get enough help support emails now where might start ignoring folks.

Sorry, my bad. My anti virus application was it, that caused the problem x)