Linear regression

OpenMV Boards
1

Hi,
I’m struggling and could use some help. I’m sure its probably simple…

Objective: detect Yellow lines.
Code: Use LAB thresholds to detect yellow and transform image into binary. This works rather well (outside light influence outcome). Pic 1.
Then I wan to use linear regression to find the line that matches best. This part goes wrong.
Even when I REVERT the image, results stay the same…
Code never finds a LINE. N/A

I prefer not to use gray scale as the color contrast is not helping to detect my yellow lines.

What is wrong in my code setup. Is it because I do not use area and pixel thresholds.

I also included pictures of what happens.
Pic 1: result without linear regression. Only binary threshold. => Clear lines
Pic 2: with regression. (used (1,1) as its binary image, so only use the “black” pixels. Or does this not work?)
Seems bottom of the frame is all messed up.


My code:

sensor.reset()
sensor.set_pixformat(sensor.RGB565)
sensor.set_framesize(sensor.QQVGA)
sensor.skip_frames(time = 2000)
sensor.set_auto_gain(False) # must be turned off for color tracking
sensor.set_auto_whitebal(False) # must be turned off for color tracking
clock = time.clock()

while(True):
    clock.tick()
    img = sensor.snapshot()
    img.binary([(0, 100, -43, 16, -36, 11)], zero = False, to_bitmap=True) #Yellow and Green

    #img=img.invert()

    line = img.get_regression([(1,1)], robust = True)

    if (line): img.draw_line(line.line(), color = 127)
    print("FPS %f, mag = %s" % (clock.fps(), str(line.magnitude()) if (line) else "N/A"))

Picture2.jpg
Picture2.jpg160×120 6.24 KB

Picture1.jpg
Picture1.jpg160×120 3.6 KB

2

Hi, binary pixels are just 0 and 255. So, set your (1,1) threshold to (200, 255).

If you execute the .to_bitmap() method on an image this will make the image truely binary with values of 0/1. However, always assume things are 8bit.

3

I will try, but I believe I also tried that.
But still no line was found.
Will give an update tomorrow

4

Hi, can you run the hello world script (witht he right color/res combo you want), then right click o the frame buffer, save the image as a BMP file, and then post that here? You might have to zip the file up to post.

I’ll just give you the code for this.

Thanks,

5

Hi,

Sure I will do this. Will post

Just as info, I tried the settings you suggested on the linear regression function (250,255).
Both on a normal Binary image but also on a inverted Binary image. As I was starting to doubt if 250-255 looks for white or black pixels.
But both give me the same result. NO LINE can be found. MAG number => N/A

One thing that looks weird, the regression function seems to add some weird line on the left upper side. I added 2 pictures to show (regression on normal and regression on inverted)

Pic Binary (using the thresholds)
Pic Regress (after doing regress function => See weird line in red oval) Cant Find a line
Pic Regress Inv (also here a weird line) => Cant find a line
Picture Regress.JPG
Picture Binary.jpg
Picture Regress inverted.jpg

6

Hello,

second update: to be sure I also tried (200,255) threshold on the linear regression, but no changes as expected (image should only contain 0 or 255).

Related to your other question, I have added a zip file containing 2 bmp files of the original screen frames and 2 bmp files when I applied the binary conversion filter. Using Hello World example.
Main purpose is to detect the 2 lines separate, but if 1 would work, its already a first step.
Then I could use framing of the image to scan for more.

Regards,
Yorick
Lane detect.zip (81.9 KB)

7

Hi, here were the issues… first, the to_bitmap argument for binary seems to be bugged. I’ll need to look into that argument. I don’t think it updates the frame buffer correctly. Second, you were getting tripped up on what colors are in the image. binary() doesn’t change the color space of an image. It just makes all the pixels black or white. They are still RGB565 pixels. So, the color bounds you pass to get_regression have to be for RGB565. I would suggest using 6 color space tuples in all your code to avoid confusion. If you look at my code below, I converted the image to grayscale to get around that issue. Grayscale images only need two color tuples.

import sensor, image, time

sensor.reset()
sensor.set_pixformat(sensor.RGB565)
sensor.set_framesize(sensor.QQVGA)
sensor.skip_frames(time = 2000)
sensor.set_auto_gain(False) # must be turned off for color tracking
sensor.set_auto_whitebal(False) # must be turned off for color tracking
clock = time.clock()

while(True):
    clock.tick()
    img = image.Image("OriginalPic1.bmp", copy_to_fb=True)
    img.binary([(0, 100, -43, 16, -36, 11)], zero = False) #Yellow and Green

    img=img.invert().to_grayscale()

    line1 = img.get_regression([(200,255)], roi=(0, 0, img.width()//2, img.height()), robust = True)
    line2 = img.get_regression([(200,255)], roi=(img.width()//2, 0, img.width()//2, img.height()), robust = True)

    if (line1): img.draw_line(line1.line(), color = 127)
    if (line2): img.draw_line(line2.line(), color = 127)
    print("FPS %f, mag = %s" % (clock.fps(), str(line1.magnitude()) if (line1) else "N/A"))

Anyway, you need to correct the perspective to make this easier. Do something like:

import sensor, image, time

sensor.reset()
sensor.set_pixformat(sensor.RGB565)
sensor.set_framesize(sensor.QQVGA)
sensor.skip_frames(time = 2000)
sensor.set_auto_gain(False) # must be turned off for color tracking
sensor.set_auto_whitebal(False) # must be turned off for color tracking
clock = time.clock()

while(True):
    clock.tick()
    img = image.Image("OriginalPic1.bmp", copy_to_fb=True).rotation_corr(y_rotation=85, zoom=3)
    img.binary([(0, 100, -43, 16, -36, 11)], zero = False) #Yellow and Green

    img=img.invert().to_grayscale()

    line1 = img.get_regression([(200,255)], roi=(0, 0, img.width()//2, img.height()), robust = True)
    line2 = img.get_regression([(200,255)], roi=(img.width()//2, 0, img.width()//2, img.height()), robust = True)

    if (line1): img.draw_line(line1.line(), color = 127)
    if (line2): img.draw_line(line2.line(), color = 127)
    print("FPS %f, mag = %s" % (clock.fps(), str(line1.magnitude()) if (line1) else "N/A"))

I know that we need a general image transform method to map any 4 points in one view to another. I will add that soon, however, there are other firmware tasks in the mean-time.

1 Like
8

Thank you for the help.

OK, I got it. First a binary transform and then a gray scale step in between to really transform the pixels from RGB.
I will give it a go.

One question: where do you save/store the .bmp file so that the IDE can find it? Very helpful to do simulation work.

PS: I’m really looking forward on using the tool with CNN. Dont get me wrong, filtering/processing images can do a lot, but a simple neural network to detect lines for example should be so much more robust.

Thank you.

9

You save the bmp file on an SD card attached to the OpenMV Cam. Please remember the IDE is doing no work really.

Regarding pixel segmentation using a CNN. Technically, you can already do this… You just need to collect a dataset of the lines, then label the pixels which are lines, and train a CNN. We support image segmentation output from TensorFlow right now.

The only issue is that there’s no tooling with OpenMV IDE to make this easy so it’s on you to be a TensorFlow master right now.