find_lines vs cv2.HoughLines ... which is better?

Discussion related to "under the hood" OpenMV topics.
slow_one
Posts: 66
Joined: Fri Jun 02, 2017 11:25 am

find_lines vs cv2.HoughLines ... which is better?

Postby slow_one » Tue Jun 06, 2017 4:18 pm

I've been playing around with some of the examples on /usr/

I've done HoughLine detection using cv2.HoughLines()... is find_lines() "better" (I'm looking at the find_lines.py example)?
I'm just trying to figure out what all libraries/ modules have been included by default or are used for a specific reason I don't know about.

Thank you!
User avatar
iabdalkader
Posts: 344
Joined: Sun May 24, 2015 3:53 pm

Re: find_lines vs cv2.HoughLines ... which is better?

Postby iabdalkader » Tue Jun 06, 2017 5:54 pm

It's a different implementation, optimized to run on OpenMV. Note Open-CV libraries are Not available on OpenMV
User avatar
kwagyeman
Posts: 1078
Joined: Sun May 24, 2015 2:10 pm

Re: find_lines vs cv2.HoughLines ... which is better?

Postby kwagyeman » Tue Jun 06, 2017 9:24 pm

If you were to use our find lines function on a pi it would be significantly faster than the CV find lines function. We use the output from a sobel filter to feed the Hough transform. OpenCV does this in many, many more steps.

That said, our method is somewhat naturally noiser, but not explicitly worse.

OpenCV takes a more decoupled approach to machine vision code requiring you to build up an algorithm in steps. Our goal with our find functions is to more or less do all possible work in one method call and give you the results you likely want.
Nyamekye,
slow_one
Posts: 66
Joined: Fri Jun 02, 2017 11:25 am

Re: find_lines vs cv2.HoughLines ... which is better?

Postby slow_one » Wed Jun 07, 2017 10:58 am

thanks for all of the answers!
is there a list (I'm sure I'm missing its location) of the OpenMV functions/ modules that have been implemented?
User avatar
iabdalkader
Posts: 344
Joined: Sun May 24, 2015 3:53 pm

Re: find_lines vs cv2.HoughLines ... which is better?

Postby iabdalkader » Wed Jun 07, 2017 11:05 am

slow_one wrote:
Wed Jun 07, 2017 10:58 am
thanks for all of the answers!
is there a list (I'm sure I'm missing its location) of the OpenMV functions/ modules that have been implemented?
See the docs
slow_one
Posts: 66
Joined: Fri Jun 02, 2017 11:25 am

Re: find_lines vs cv2.HoughLines ... which is better?

Postby slow_one » Wed Jun 07, 2017 11:15 am

iabdalkader wrote:
Wed Jun 07, 2017 11:05 am
See the docs
Thanks! I'd found that page but hadn't scrolled down far enough!
slow_one
Posts: 66
Joined: Fri Jun 02, 2017 11:25 am

Re: find_lines vs cv2.HoughLines ... which is better?

Postby slow_one » Thu Jun 15, 2017 4:03 pm

Is there a numpy like module or function?
I haven't found anything that jumped out at me ...

In python I'd use it for functions like numpy.array or numpy.max ...

thank you!
User avatar
kwagyeman
Posts: 1078
Joined: Sun May 24, 2015 2:10 pm

Re: find_lines vs cv2.HoughLines ... which is better?

Postby kwagyeman » Thu Jun 15, 2017 9:00 pm

No, there's no numpy support. What do you need that for?
Nyamekye,
slow_one
Posts: 66
Joined: Fri Jun 02, 2017 11:25 am

Re: find_lines vs cv2.HoughLines ... which is better?

Postby slow_one » Fri Jun 16, 2017 2:08 pm

kwagyeman wrote: No, there's no numpy support. What do you need that for?
I use in during a object recognition task.
I look at flow out of a nozzle ... detect the edges of the flow and "draw lines" showing the edges of the flow to see if the nozzle needs cleaning.
(high level view of the steps: image thresholding, Canny Edge Detection, Canny Contours, Hough LinesP, find "highest left, lowest left" and "highest right, lowest right" points on the outer edges of the flow, draw two lines one top left to bottom left and one top right to bottom right ... and then see if the flow out of the nozzle is "straight down" or "cone shaped").

The step that finds the top and bottom points on each edge requires an array.
User avatar
kwagyeman
Posts: 1078
Joined: Sun May 24, 2015 2:10 pm

Re: find_lines vs cv2.HoughLines ... which is better?

Postby kwagyeman » Fri Jun 16, 2017 11:33 pm

Okay, I see.

So, fine_line_segments does just what you need:

http://docs.openmv.io/library/omv.image ... e_segments

And it returns a list of these things:

http://docs.openmv.io/library/omv.image ... ine-object

...

You just need to setup your thresholds and merge settings and you'll basically get two lines on either side of the nozel that start and end on either side of the stream until they hit the nozel or bottom of the image.

Anyway, there's a find_line_segments example script for this... start with it.

...

Also, since you know what openCV does to find lines, here's what we do: https://github.com/openmv/openmv/blob/m ... mg/hough.c

Basically, we sobel filter the image and use the gradient and mag output from each pixel to feed a hough accumulator array. By doing this we skip the canny process and doing the 0-180 degree hough line iteration on each pixel. This method is a little more noisy but an order of magnitude faster. After which, we look for hough peaks above your given threshold and we also make sure to only return true peaks (i.e. peaks with no value near them that are greater than them). Next we merge all hough lines which have similar rho and pheta values. We do this because the raw output of the hough transform produces a ton of lines. By merging lines you get cleaner output. Finally, we compute the start and end points of each infinite line.

Find line segments uses the above code and then walks each infinite line while running the sobel algorithm on the pixels underneath to determine the start and end points of line segments on that infinite line. A better method would have been to keep track of what points contributed to the infinite line when we ran the above algorithm but we don't have memory for that. Anyway, to make walking the infinite line work better I actually 3 three lines in parallel to each infinite line to deal with slight curves. Then finally, the real magic happens during merging. So, walking lines with the above approach produces a lot of small unconnected lines right next to each other. Our merge algorithm then joins all these small line segments into one line segment that's returned to you.

Anyway, the point of both these methods is to get you the output you want without having to deal with all the details between.
Nyamekye,
slow_one
Posts: 66
Joined: Fri Jun 02, 2017 11:25 am

Re: find_lines vs cv2.HoughLines ... which is better?

Postby slow_one » Mon Jun 19, 2017 10:24 am

kwagyeman wrote:
Fri Jun 16, 2017 11:33 pm
... stuff ...
Dude.

Let me go through and read that stuff ... but wow.
That sounds pretty awesome.

And that much faster? Wow. I wonder why OpenCV doesn't do it that way ...
slow_one
Posts: 66
Joined: Fri Jun 02, 2017 11:25 am

Re: find_lines vs cv2.HoughLines ... which is better?

Postby slow_one » Mon Jun 19, 2017 10:47 am

kwagyeman wrote:
Fri Jun 16, 2017 11:33 pm

find_line_segments:
It looks like this operates on a region of interest ... cool idea.
Is there an "easy" way to have this look at two regions of interest within a single image without having to call the algorithm twice?
(ie, the left and right side of the spray... instead of looking at the entire image ...? Breaking the image down in to two smaller sections may be faster than finding lines/ edges over a larger viewing area)
User avatar
kwagyeman
Posts: 1078
Joined: Sun May 24, 2015 2:10 pm

Re: find_lines vs cv2.HoughLines ... which is better?

Postby kwagyeman » Mon Jun 19, 2017 11:17 am

By definition when you use a region of interest the algorithm only does work on that region. So, you need to call the algorithm for each region but it does less work since each region is smaller than the whole image.

In the latest version of OpenMV IDE you can select an area in the frame buffer to get the region of interest coordinates.

...

OpenCV does this differently because there steps are decoupled. The sobel trick to feed the Hough accumulator only works if you tie the two algorithms together. OpenCV takes the more accademic view on things and does the general operation but tires to be less specific for the application.

For another example, look how to find color blobs in OpenCV and then compare that to our find_blobs method.
Nyamekye,
slow_one
Posts: 66
Joined: Fri Jun 02, 2017 11:25 am

Re: find_lines vs cv2.HoughLines ... which is better?

Postby slow_one » Mon Jun 19, 2017 11:28 am

kwagyeman wrote:
Mon Jun 19, 2017 11:17 am
By definition when you use a region of interest the algorithm only does work on that region. So, you need to call the algorithm for each region but it does less work since each region is smaller than the whole image.

In the latest version of OpenMV IDE you can select an area in the frame buffer to get the region of interest coordinates.

...

OpenCV does this differently because there steps are decoupled. The sobel trick to feed the Hough accumulator only works if you tie the two algorithms together. OpenCV takes the more accademic view on things and does the general operation but tires to be less specific for the application.

For another example, look how to find color blobs in OpenCV and then compare that to our find_blobs method.
That makes a lot of sense.
General vs Specialized Application.

in the image.find_line_segments method ... would syntax be something like :

for l in img.find_line_segments(roi = (1,1,200,200), threshold = 1000, theta_margin = 15, rho_margin = 15, segment_threshold = 100):
img.draw_line(l.line(), color = (255, 0, 0))

for the roi defined by the tuple: (1,1,200,200) or is it treated like an array [x,y,w,h] ?
(sorry, this isn't in the example)
User avatar
kwagyeman
Posts: 1078
Joined: Sun May 24, 2015 2:10 pm

Re: find_lines vs cv2.HoughLines ... which is better?

Postby kwagyeman » Mon Jun 19, 2017 11:37 am

Yes, all Rois are x,y,w,h. Select an area in the frame buffer of the IDE and it should tell you the ROI for that area.
Nyamekye,
slow_one
Posts: 66
Joined: Fri Jun 02, 2017 11:25 am

Re: find_lines vs cv2.HoughLines ... which is better?

Postby slow_one » Fri Jun 23, 2017 3:05 pm

I've been playing around with the find_line_segments() method.

Using this example loop I can return the line object:

Code: Select all

   for l in img.find_line_segments(threshold = 1000, theta_margin = 15, rho_margin = 15, segment_threshold = 100):
        img.draw_line(l.line(), color = (255, 0, 0))
       print(l)
However, i'd like to return the "longest line" found by img.find_line_segments.
I know that if do

Code: Select all

print(l[4])
I can get the line length for the lines that are being returned ... but, I can't do

Code: Select all

print(max(l[4]))
because I get an error ... ie, 'l' is not being returned like an array (I'm used to Matlab and being able to iterate over an array...).
How should I approach something like this?
User avatar
kwagyeman
Posts: 1078
Joined: Sun May 24, 2015 2:10 pm

Re: find_lines vs cv2.HoughLines ... which is better?

Postby kwagyeman » Fri Jun 23, 2017 4:18 pm

Um, see the line following code under the color tracking examples. There's some code there that does exactly what you want. Basically, you need to use a lambda lethod. Google python get max of object in array.

I think what you want looks like this:

Max(img.find, key=lambda x: x.length())

Basically, the key argument of the max function tells max how to find the max of each item. It then returns the item in the list with the max key. Note that you don't need the for loop anymore.
Nyamekye,
slow_one
Posts: 66
Joined: Fri Jun 02, 2017 11:25 am

Re: find_lines vs cv2.HoughLines ... which is better?

Postby slow_one » Tue Jul 18, 2017 3:46 pm

Max(img.find, key=lambda x: x.length())


Is this the length of x1 - x2 ?
or is this the euclidian distance of (x1, y1) to (x2, y2)?

I ask because for the lines that have been found as the "max" ... the theta is always equal to zero ... that seems... hinky.
User avatar
kwagyeman
Posts: 1078
Joined: Sun May 24, 2015 2:10 pm

Re: find_lines vs cv2.HoughLines ... which is better?

Postby kwagyeman » Tue Jul 18, 2017 8:11 pm

Length is the euclidean distance. If you are getting a theta of zero that would be a horizontal line in the image somewhere.
Nyamekye,
slow_one
Posts: 66
Joined: Fri Jun 02, 2017 11:25 am

Re: find_lines vs cv2.HoughLines ... which is better?

Postby slow_one » Wed Jul 19, 2017 10:10 am

I am getting a theta of zero for several values ... but that seems to happen when there aren't any lines in the ROI to be detected.
Is there a way to filter those out ?

I'm thinking a series of "if's" ... if theta!=0 then max(img. ... ) ... although I'm not quite sure how to pull out the theta term for the comparison.

EDIT:
This *doesn't* work:
rl = max(img.find_line_segments(roi = r_r, threshold = 1000, theta_margin = 15, rho_margin = 15, segment_threshold = 100), key = lambda x: x.length() if key = lambda x:x.theta()!=0)

I didn't expect it to ... but hey :)
User avatar
kwagyeman
Posts: 1078
Joined: Sun May 24, 2015 2:10 pm

Re: find_lines vs cv2.HoughLines ... which is better?

Postby kwagyeman » Wed Jul 19, 2017 11:03 am

The key statement for the max method is too complex. You'll want to use the filter statement in python. See the pixy emulation scripts for examples on how to filter objects using the filter command. It's quite easy.

In general, use magnitude to threshold lines and pick out garbage ones.
Nyamekye,
slow_one
Posts: 66
Joined: Fri Jun 02, 2017 11:25 am

Re: find_lines vs cv2.HoughLines ... which is better?

Postby slow_one » Wed Jul 19, 2017 11:11 am

I'll take a look at the filter command ... but why isn't this working?

Code: Select all

for rl in img.find_line_segments(roi = r_r, threshold = 1000, theta_margin = 15, rho_margin = 15, segment_threshold = 100):
    for rl[6] != 0 : #rl[6] should be the theta value
        print(rl)
I'm getting a syntax error on the

Code: Select all

for rl[6] != 0:
line ... ?
slow_one
Posts: 66
Joined: Fri Jun 02, 2017 11:25 am

Re: find_lines vs cv2.HoughLines ... which is better?

Postby slow_one » Wed Jul 19, 2017 11:41 am

Ok. That sort of solves an issue ... but that returns a list ... and I am trying to draw the lines returned as if returned from find_line_segments() .
here's what I'm doing

Code: Select all

.
.
.
def line_filter(line):
    if(line[6] == 0):
        return False
    elif(line[6] > 0):
        return ( max(img.find_line_segments(roi = r_r, threshold = 1000, theta_margin = 15, rho_margin = 15, segment_threshold = 100), key = lambda x: x.length()) )   

while(True):
    clock.tick()
    img = sensor.snapshot() if snapshot_source else img_reader.next_frame(copy_to_fb=True, loop=True)
    img.draw_rectangle(l_r, color = 155 )
    img.binary(binary_thresholds, invert = False)
    rl = list(filter(line_filter, img.find_line_segments(roi = r_r, threshold = 1000, theta_margin = 15, rho_margin = 15, segment_threshold = 100))) 
    print(rl)
    #img.draw_line(rl.line(), color = 155)
    print(clock.fps())
because it's returning a 'list', img.draw_line() doesn't know what to do ... is there a good way to back out of that? Not have it return a list?
If I don't stack it all in a list, the filter() object complains the 'filter' object has no attribute 'line'
User avatar
kwagyeman
Posts: 1078
Joined: Sun May 24, 2015 2:10 pm

Re: find_lines vs cv2.HoughLines ... which is better?

Postby kwagyeman » Wed Jul 19, 2017 11:45 am

Filter just removed elements from a list by testing if your filter function returns false on the element. It still returns a list.

As for your loop. Please review python syntax. In general, in a for loop you have to iterate over a list and then you can put an if statement in the loop to check the element value.
Nyamekye,
slow_one
Posts: 66
Joined: Fri Jun 02, 2017 11:25 am

Re: find_lines vs cv2.HoughLines ... which is better?

Postby slow_one » Wed Jul 19, 2017 1:38 pm

I think I get what's going on ...
at some point, there's no line segments found ... so there's nothing to filter ... so rl doesn't exist.
slow_one
Posts: 66
Joined: Fri Jun 02, 2017 11:25 am

Re: find_lines vs cv2.HoughLines ... which is better?

Postby slow_one » Wed Jul 19, 2017 4:45 pm

Ok. Now I'm thoroughly confused ... this isn't making sense to me.
Why does this portion of the script fail if "arg is an empty sequence"?

Code: Select all

rl = max(img.find_line_segments(roi = r_r, threshold = 1000, theta_margin = 15, rho_margin = 15, segment_threshold = 100), key = lambda x: x.length())
    #print(rl)
    if rl is not None:
        if rl[6] > 0 :
            img.draw_line(rl.line(), color = 155)
            print("RL")
            print(rl)


The "if rl is not None:" should prohibit it from ever failing if max() doesn't return a value ...

EDIT:
I'm starting to think that there's an error with the way that the max() function returns values. If I remove the if statement completely and have no other processing afterwards, I still get the error.
User avatar
kwagyeman
Posts: 1078
Joined: Sun May 24, 2015 2:10 pm

Re: find_lines vs cv2.HoughLines ... which is better?

Postby kwagyeman » Wed Jul 19, 2017 5:58 pm

Do not indent unless you need to please. There's an unnecessary indent.

Anyway, what's the exact error you are getting and on which line? Post all the code?
Nyamekye,
slow_one
Posts: 66
Joined: Fri Jun 02, 2017 11:25 am

Re: find_lines vs cv2.HoughLines ... which is better?

Postby slow_one » Thu Jul 20, 2017 9:50 am

This is how it was fixed:

Code: Select all

segments_l = img.find_line_segments(roi = l_r, threshold = 1000, theta_margin = 15, rho_margin = 15, segment_threshold = 100)
        if segments_l :
            max_segment_l = max(segments_l, key = lambda x: x.length())
I broke the max() statement out in to this to get around the cases where max() doesn't return a value ... fine_line_segments *seems* to always return a line... but it seems like max() doesn't always return a maximum line.
I can't seem to get any more insight than that however.

Here is the error I was seeing :

Code: Select all

  
  Traceback (most recent call last):
File "<stdin>", line 77, in <module>
ValueError: arg is an empty sequence
MicroPython d23b594 on 2017-07-05; OPENMV3 with STM32F765
Type "help()" for more information.
User avatar
kwagyeman
Posts: 1078
Joined: Sun May 24, 2015 2:10 pm

Re: find_lines vs cv2.HoughLines ... which is better?

Postby kwagyeman » Thu Jul 20, 2017 11:30 am

Ah, okay, I see. Max might not like the fact that find lines may not return a line sometimes. Then max complains.
Nyamekye,
slow_one
Posts: 66
Joined: Fri Jun 02, 2017 11:25 am

Re: find_lines vs cv2.HoughLines ... which is better?

Postby slow_one » Thu Jul 20, 2017 1:05 pm

that makes sense (although it's weird because I don't see any "skipping" in finding a line... although I don't have insight at that level, really)


and now to figure out why it sometimes takes several resets for the IDE to load/run the program.

Return to “Technical Discussion”

Who is online

Users browsing this forum: No registered users and 2 guests