Lines Detected Change on successive images

Discussion related to "under the hood" OpenMV topics.
mjs513
Posts: 71
Joined: Sun Apr 30, 2017 12:52 pm

Lines Detected Change on successive images

Postby mjs513 » Sat Aug 05, 2017 2:15 pm

I am trying to understand the line-segment example that was added in the last release. It works rather well except for one thing. As I let it run the lines detected change in successive frames. They go back and forth between the lines. I made a little video of what I mean at this link, https://drive.google.com/open?id=0BwzZj ... GlhNTVUMkk. Was wondering if there was any way to get to get consistency between frames? Have this same problem with other edge detection methods as well.

Thanks
Mike

PS> When is the M8 Coming out :)
User avatar
kwagyeman
Posts: 1175
Joined: Sun May 24, 2015 2:10 pm

Re: Lines Detected Change on successive images

Postby kwagyeman » Sat Aug 05, 2017 2:52 pm

Yeah, that's due to how the fast method we're using to detect lines works along with the averaging of lines... if you turn down all the averaging options and whatnot you'll be able to see the real output of the algorithm... and how there are a lot of detections to deal with.

For example, lower the threshold and lower the merging towards zero, you'll start to get a lot more lines.

I think part of the issue is how I do merging, when I first wrote that code for blob detection I thought it was fine and used it in a bunch of other places but I'm starting to think now that the merging operation is not that stable. The merging operation right now kinda acts like an FIR filter versus a weighted average. E.g. the order of merges matters for the resulting output line and the order of merged lines can easily change.

If you have an idea for how to improve this I'm all ears, but, I don't have time to make it better right now:

https://github.com/openmv/openmv/blob/m ... ugh.c#L250

https://github.com/openmv/openmv/blob/m ... ugh.c#L635

...

ST hasn't released the STM32H7 yet... so, I can't say. For now, we're just building an audience for it with more M7 boards and trying to get the price on it down.
Nyamekye,
mjs513
Posts: 71
Joined: Sun Apr 30, 2017 12:52 pm

Re: Lines Detected Change on successive images

Postby mjs513 » Sat Aug 05, 2017 4:27 pm

I am going to take a look at it. But for now is there a way to do it just on a single image?

Also have you seen this, looks interesting: http://www.ipol.im/pub/art/2012/gjmr-lsd/, https://github.com/primetang/LSD-OpenC ... ster/pylsd. Unfortunately looks like it wants numpy :(.
User avatar
kwagyeman
Posts: 1175
Joined: Sun May 24, 2015 2:10 pm

Re: Lines Detected Change on successive images

Postby kwagyeman » Sat Aug 05, 2017 4:56 pm

Heh, heh, so, those are all stills... so, you don't know if their code is any better. That said, it probably is.

Anyway, I think the main source of error is the fact that the camera output is rather jumpy. If you look at just what the camera sees you'll notice that the color and gain level are rapidly moving around a lot. The algorithm I wrote isn't that different that what was presented in the first paper you linked to.

As for making the camera more stable... OmniVision will not provide us with the "Golden Register Settings" for the OV7725 sensor. If you'd like to contribute the thing to do would be to look into modifying some of the camera register settings to make it more stable.

https://cdn.shopify.com/s/files/1/0803/ ... 6050138641

... I also have a private datasheet from my CMUcam4 days about how to program register settings that may be helpful but I can't share it online. Let me know if you actually would like to look into making the register settings better.
Nyamekye,
User avatar
kwagyeman
Posts: 1175
Joined: Sun May 24, 2015 2:10 pm

Re: Lines Detected Change on successive images

Postby kwagyeman » Sat Aug 05, 2017 5:02 pm

Mmm, it looks like they've got a C code library for it. I could port it to the M7 pretty easily.

Yeah, the code is quite simple. Should be easy to port...

...

Okay, I'll schedule to do this.
Nyamekye,
mjs513
Posts: 71
Joined: Sun Apr 30, 2017 12:52 pm

Re: Lines Detected Change on successive images

Postby mjs513 » Sat Aug 05, 2017 8:05 pm

Hi kwagyeman,

Hmm. Thought there was a video embedded in the fist link that I sent you. I did notice that the histogram was jumping quite a bit which I thought might be a contributor but I didn't know how to stop that. I have tried changing the gain and brightness in the past but.....

Now, looking at register settings and modifying them is probably something I can understand. I wouldn't mind giving it a shot. Can't make any promises :) but I can try. You want me to send you my email or post it here.

Mike
User avatar
kwagyeman
Posts: 1175
Joined: Sun May 24, 2015 2:10 pm

Re: Lines Detected Change on successive images

Postby kwagyeman » Sat Aug 05, 2017 8:24 pm

Just email openmv for the docs.

Okay, so, it's possible to port that c code. I took a look at it and it's doable... but, somewhat a lot of work. Like... a lot of the OpenCV stuff there is no effort not to waste memory on pointless amounts of resolution. So, I'll have to modify a lot of things in there code to get it operational. Given this, I'm going to look into just making my own code less jumpy, it shouldn't really be so jumpy in the first place anyway...

That said, I wish I'd known about the LSD code first before I wrote my own line segment finder. Would have been less work.
Nyamekye,
User avatar
kwagyeman
Posts: 1175
Joined: Sun May 24, 2015 2:10 pm

Re: Lines Detected Change on successive images

Postby kwagyeman » Sat Aug 05, 2017 8:42 pm

Mmm, so, I just tried my method out again and the results aren't really that bad... Um, it's not going to work really well pointing it at random stuff. That was not my goal, have you tried just having like one thing in the image? The camera is still a sensor and has to be used in a focused way.

I do agree though that results are quite jumpy. There may be some things I can do to make that better. If you use the regular find_lines method for infinite lines and lower the merging margins you'll be able to see why the algorithm is doing certain things.
Nyamekye,
mjs513
Posts: 71
Joined: Sun Apr 30, 2017 12:52 pm

Re: Lines Detected Change on successive images

Postby mjs513 » Sat Aug 05, 2017 8:45 pm

Only got it just now because I got my rove semi working with the M7 and was looking for something more stable so I finally tried the line segments example and decided to post. Sorry about that. Anyway in looking for more info on 7725 registers I cam across the Arducam which is interesting since it can be attached to the Arduino and that I understand better than Python. Took a look at it and it had a register map with the settings and the changes they made. Haven't gone much further yet. Here is the link if you are interested, https://github.com/ArduCAM/Arduino/blob ... 725_regs.h
User avatar
iabdalkader
Posts: 363
Joined: Sun May 24, 2015 3:53 pm

Re: Lines Detected Change on successive images

Postby iabdalkader » Sat Aug 05, 2017 9:44 pm

Could you try disabling fast AGC/AEC (see COM8) ?
mjs513
Posts: 71
Joined: Sun Apr 30, 2017 12:52 pm

Re: Lines Detected Change on successive images

Postby mjs513 » Sun Aug 06, 2017 6:47 am

Hi iabdalkader,

EDIT: Ok turned them both off and screen went black. Seems to be driven by auto exposure. I did experiment a little bit and just used:
sensor.set_auto_gain(False,value=0)
sensor.set_gainceiling(2)

It seemed to get better if the edges were very distinct. If there were any shadows it would be jumpy in that area.

If I use the edges example in that same are I can see the outlines of objects very clearly.


thanks
mike
User avatar
iabdalkader
Posts: 363
Joined: Sun May 24, 2015 3:53 pm

Re: Lines Detected Change on successive images

Postby iabdalkader » Sun Aug 06, 2017 1:29 pm

I tried it and it doesn't seem to make any difference, but it shouldn't blank the output so I think you disabled something else. For the future here's how to disable a bit in a reg:

Code: Select all

reg = sensor.__read_reg(addr)
sensor.__write_reg(addr, reg&mask)
mjs513
Posts: 71
Joined: Sun Apr 30, 2017 12:52 pm

Re: Lines Detected Change on successive images

Postby mjs513 » Sun Aug 06, 2017 2:08 pm

I used sensor.set_auto_exposure(False, value=0) to turn the exposure off, unless its suppose to +1? Didn't notice that you were able to set the reg using a sensor command. Just looked it up now that you mentioned it.
User avatar
kwagyeman
Posts: 1175
Joined: Sun May 24, 2015 2:10 pm

Re: Lines Detected Change on successive images

Postby kwagyeman » Tue Aug 08, 2017 1:13 am

Hey, I got a lot of progression on porting the lsd algorithm to the cam. It will work on it... however, I had to modify a lot of the code. Since we need to do a release soon this feature will not be in the next release. I'll cut you a firmware with the fixes though afterwards.
Nyamekye,
mjs513
Posts: 71
Joined: Sun Apr 30, 2017 12:52 pm

Re: Lines Detected Change on successive images

Postby mjs513 » Tue Aug 08, 2017 10:10 pm

Thanks kwagyeman. I am surprised that you decided to work on it. I have started to look at the register mapping and the settings. Quite a bit in there.
mjs513
Posts: 71
Joined: Sun Apr 30, 2017 12:52 pm

Re: Lines Detected Change on successive images

Postby mjs513 » Wed Aug 09, 2017 5:04 pm

kwagyeman. You might find this document of use in your efforts. Found it doing a web search. Has some stuff I was looking into: http://www.electrodragon.com/w/images/4 ... n_note.pdf
mjs513
Posts: 71
Joined: Sun Apr 30, 2017 12:52 pm

Re: Lines Detected Change on successive images

Postby mjs513 » Wed Aug 09, 2017 8:14 pm

Playing with lighting options:

Code: Select all

# Hello World Example
#
# Welcome to the OpenMV IDE! Click on the green run arrow button below to run the script!

import sensor, image, time

sensor.reset()                      # Reset and initialize the sensor.
sensor.set_pixformat(sensor.RGB565) # Set pixel format to RGB565 (or GRAYSCALE)
sensor.set_framesize(sensor.QVGA)   # Set frame size to QVGA (320x240)
sensor.skip_frames(time = 2000)     # Wait for settings take effect.


clock = time.clock()                # Create a clock object to track the FPS.

def light_mode_auto():
    #sensor.__write_reg(addr, reg&mask)
    sensor.__write_reg(0x13, 0xff)  #AWB off
    sensor.__write_reg(0x0e, 0x65)
    sensor.__write_reg(0x2d, 0x00)
    sensor.__write_reg(0x2e, 0x00)

def light_mode_home():
    sensor.__write_reg(0x13, 0xfd)  #AWB off
    sensor.__write_reg(0x01, 0x96)
    sensor.__write_reg(0x02, 0x40)
    sensor.__write_reg(0x0e, 0x65)
    sensor.__write_reg(0x2d, 0x00)
    sensor.__write_reg(0x2e, 0x00)
    
def light_mode_night():
    #sensor.__write_reg(addr, reg&mask)
    sensor.__write_reg(0x13, 0xff)  #AWB on
    sensor.__write_reg(0x0e, 0xe5)
    
def light_mode_cloudy():
    sensor.__write_reg(0x13, 0xfd)  #AWB off
    sensor.__write_reg(0x01, 0x58)
    sensor.__write_reg(0x02, 0x60)
    sensor.__write_reg(0x0e, 0x65)
    sensor.__write_reg(0x2d, 0x00)
    sensor.__write_reg(0x2e, 0x00)

light_mode_auto()

while(True):
    clock.tick()                    # Update the FPS clock.
    img.lens_corr(2.0)
    img = sensor.snapshot()         # Take a picture and return the image.

    print(clock.fps())              # Note: OpenMV Cam runs about half as fast when connected
                                    # to the IDE. The FPS should increase once disconnected.
User avatar
kwagyeman
Posts: 1175
Joined: Sun May 24, 2015 2:10 pm

Re: Lines Detected Change on successive images

Postby kwagyeman » Wed Aug 09, 2017 9:53 pm

OMG. That's solid gold there. Yeah, we can do a lot with that.
Nyamekye,
mjs513
Posts: 71
Joined: Sun Apr 30, 2017 12:52 pm

Re: Lines Detected Change on successive images

Postby mjs513 » Thu Aug 10, 2017 6:33 pm

I started playing around with the modes some more along with adjusting for banding. Here is a test sketch. May help.

Code: Select all

# Find Line Segments Example
#
# This example shows off how to find line segments in the image. For each line object
# found in the image a line object is returned which includes the line's rotation.

# Note: Line detection is done by using the Hough Transform:
# http://en.wikipedia.org/wiki/Hough_transform
# Please read about it above for more information on what `theta` and `rho` are.

enable_lens_corr = True # turn on for straighter lines...

import sensor, image, time

sensor.reset()
sensor.set_pixformat(sensor.RGB565) # grayscale is faster
sensor.set_framesize(sensor.QQVGA)
sensor.skip_frames(time = 2000)
clock = time.clock()

def light_mode_auto():
    #sensor.__write_reg(addr, reg&mask)
    sensor.__write_reg(0x13, 0xff)  #AWB off
    sensor.__write_reg(0x0e, 0x65)
    sensor.__write_reg(0x2d, 0x00)
    sensor.__write_reg(0x2e, 0x00)

    sensor.__write_reg(0x22, 0x89); #60Hz banding filter
    sensor.__write_reg(0x23, 0x03); #4 step for 60hz

def light_mode_home():
    sensor.__write_reg(0x13, 0xfd)  #AWB off
    sensor.__write_reg(0x01, 0x96)
    sensor.__write_reg(0x02, 0x40)
    sensor.__write_reg(0x0e, 0x65)
    sensor.__write_reg(0x2d, 0x00)
    sensor.__write_reg(0x2e, 0x00)
    
    sensor.__write_reg(0x22, 0x7f); #60Hz banding filter
    sensor.__write_reg(0x23, 0x03); #4 step for 60hz
    
def light_mode_night():
    #sensor.__write_reg(addr, reg&mask)
    sensor.__write_reg(0x13, 0xff)  #AWB on
    sensor.__write_reg(0x0e, 0xe5)
    sensor.__write_reg(0x11, 0x03);
    sensor.__write_reg(0x22, 0x7f); #60Hz banding filter
    sensor.__write_reg(0x23, 0x03); #4 step for 60hz
    
def light_mode_cloudy():
    sensor.__write_reg(0x13, 0xfd)  #AWB off
    sensor.__write_reg(0x01, 0x58)
    sensor.__write_reg(0x02, 0x60)
    sensor.__write_reg(0x0e, 0x65)
    sensor.__write_reg(0x2d, 0x00)
    sensor.__write_reg(0x2e, 0x00)
    
def light_mode_office():
    sensor.__write_reg(0x13, 0xfd);
    sensor.__write_reg(0x01, 0x84);
    sensor.__write_reg(0x02, 0x4c);
    sensor.__write_reg(0x0e, 0x65);
    sensor.__write_reg(0x2d, 0x00);
    sensor.__write_reg(0x22, 0x7f); #60Hz banding filter
    sensor.__write_reg(0x23, 0x03); #4 step for 60hz

#sensor.set_auto_gain(False)
#sensor.set_auto_exposure(False)


light_mode_auto()

# All lines also have `x1()`, `y1()`, `x2()`, and `y2()` methods to get their end-points
# and a `line()` method to get all the above as one 4 value tuple for `draw_line()`.

while(True):
    clock.tick()
    img = sensor.snapshot()
    if enable_lens_corr: img.lens_corr(1.8, 1.0) # for 2.8mm lens...

    # `threshold` controls how many lines in the image are found. Only lines with
    # edge difference magnitude sums greater than `threshold` are detected...

    # More about `threshold` - each pixel in the image contributes a magnitude value
    # to a line. The sum of all contributions is the magintude for that line. Then
    # when lines are merged their magnitudes are added togheter. Note that `threshold`
    # filters out lines with low magnitudes before merging. To see the magnitude of
    # un-merged lines set `theta_margin` and `rho_margin` to 0...

    # `theta_margin` and `rho_margin` control merging similar lines. If two lines
    # theta and rho value differences are less than the margins then they are merged.

    # Setting both the above to zero will greatly increase segment detection at the
    # cost of a lot of FPS. This is because when less lines are merged more pixels
    # are tested... which takes longer but covers more possibilities...

    # `segment_threshold` controls line segment extraction. It's a threshold on the
    # magnitude response per pixel under an infinite line. Pixels with a magnitude
    # above threshold are added to the line segment.

    # `find_line_segments` merges detected lines that are no more than 5 pixels apart
    # and no more than 15 degrees different to create nice continous line segments.

    for l in img.find_line_segments(threshold = 800, theta_margin = 15, rho_margin = 15, segment_threshold = 100):
        img.draw_line(l.line(), color = (255, 0, 0))
        # print(l)

    print("FPS %f" % clock.fps())
Cheers
Mike
mjs513
Posts: 71
Joined: Sun Apr 30, 2017 12:52 pm

Re: Lines Detected Change on successive images

Postby mjs513 » Fri Aug 11, 2017 10:28 am

Nyamekye, I posted a couple of documents that you might find interesting on the omnivision cams in case you change the sensor: https://drive.google.com/open?id=0BwzZj ... GlhNTVUMkk
mjs513
Posts: 71
Joined: Sun Apr 30, 2017 12:52 pm

Re: Lines Detected Change on successive images

Postby mjs513 » Sun Aug 20, 2017 9:00 am

kwagyeman - I started taking a closer look at the settings they are using for the different light modes. For other than direct sun light and night modes 3 of the register settings (0x65, 0x2e and 0x2d) are the same. That leaves only the Red and Blue gains being adjusted depending on lighting. If you do a plot of lux or color temperature you get almost a linear relationship between the values used for those settings. I am attaching a couple of plots - yes I converted the hex values to decimal.

LUX
Lux-Blue.PNG
Lux-Blue.PNG (6.68 KiB) Viewed 1122 times
Lux-Red.PNG
Lux-Red.PNG (6.36 KiB) Viewed 1122 times
COLOR TEMPERATURE
ColorTemp.PNG
ColorTemp.PNG (8.66 KiB) Viewed 1122 times
mjs513
Posts: 71
Joined: Sun Apr 30, 2017 12:52 pm

Re: Lines Detected Change on successive images

Postby mjs513 » Wed Aug 30, 2017 6:19 pm

Hey, I got a lot of progression on porting the lsd algorithm to the cam.
Was wondering if you ever had a chance to finish the porting of the algorithm. Know you are all busy, but was just wondering. Still tweaking and testing the rover with the camera. Had a couple of good runs but not consistent.
User avatar
kwagyeman
Posts: 1175
Joined: Sun May 24, 2015 2:10 pm

Re: Lines Detected Change on successive images

Postby kwagyeman » Wed Aug 30, 2017 6:24 pm

No, I haven't been working on it? Are you building a line follower? There's some good code I've been working on for that under OpenMV/open projects. I've been uploading my DYI robocar race code.

I can work on this again very soon actually.
Nyamekye,
mjs513
Posts: 71
Joined: Sun Apr 30, 2017 12:52 pm

Re: Lines Detected Change on successive images

Postby mjs513 » Thu Aug 31, 2017 8:26 am

No. Not a line follower. I doing obstacle detection and avoidance, see viewtopic.php?f=5&t=276 using a openmv camera and a single vl53 distance sensor. Depending on the lighting and contrast edges may or may not be detected. Got around the issue of changing detection values by analyzing at least 5 frames on a teensy 3.5 before making decision on which way to turn. Looking to improve detection.

Thanks
Mike
User avatar
kwagyeman
Posts: 1175
Joined: Sun May 24, 2015 2:10 pm

Re: Lines Detected Change on successive images

Postby kwagyeman » Thu Aug 31, 2017 12:47 pm

Ok, okay. Yes, I'll try to finish porting that in about 2 weeks then. I was almost done before.
Nyamekye,
mjs513
Posts: 71
Joined: Sun Apr 30, 2017 12:52 pm

Re: Lines Detected Change on successive images

Postby mjs513 » Thu Aug 31, 2017 5:03 pm

Thanks man. No rush so please take you time.

Mike
User avatar
kwagyeman
Posts: 1175
Joined: Sun May 24, 2015 2:10 pm

Re: Lines Detected Change on successive images

Postby kwagyeman » Sun Sep 03, 2017 3:56 pm

Okay, I got it kinda working. It generates a very stable output but runs at 2 FPS @ 160x120. 8 FPS at 80x60. Code is crashing right now though if the input shifts around too much. Have to trace where there's a memory violation bug (this is always the case with porting desktop libs which use the stack like its free). While tracking the code I'll look to see where all the speed is lost too. The low FPS doesn't make that much sense given how the algorithm works...
Nyamekye,
mjs513
Posts: 71
Joined: Sun Apr 30, 2017 12:52 pm

Re: Lines Detected Change on successive images

Postby mjs513 » Sun Sep 03, 2017 7:50 pm

Thanks for keeping me posted. Everything turns out to be more work than planned :). I created the pull request for the Autonomous Rover. Got a little carried away with the write up. Feel free to edit to your hearts content :)
User avatar
kwagyeman
Posts: 1175
Joined: Sun May 24, 2015 2:10 pm

Re: Lines Detected Change on successive images

Postby kwagyeman » Sun Sep 03, 2017 8:40 pm

Here's a demo with the better line segment detector. The same line segment detector script released with OpenMV IDE will work to run the code. It has much better segment detection performance... but, no parameters. So, all the stuff in the find_line_segments call isn't needed.

I'll see what I can do on making it faster now. It doesn't crash anymore however. There was an infinite loop in the code due to floating point precision reduction.
Attachments
firmware.zip
(1.99 MiB) Downloaded 9 times
Nyamekye,
User avatar
kwagyeman
Posts: 1175
Joined: Sun May 24, 2015 2:10 pm

Re: Lines Detected Change on successive images

Postby kwagyeman » Sun Sep 03, 2017 9:10 pm

Okay, the code is so slow because it uses sin/cos/atan2 in a loop that's called a lot. So, I'll just use lookup tables for these values and it should be really fast...
Nyamekye,
User avatar
kwagyeman
Posts: 1175
Joined: Sun May 24, 2015 2:10 pm

Re: Lines Detected Change on successive images

Postby kwagyeman » Sun Sep 03, 2017 9:39 pm

Mmm, changing that only improved FPS by 1. I see that this isn't the problem... but, how it sets up it's data structures (lots of poorly structured inner loops). I won't be able to get it to go faster without redoing a lot at the algorithmic level.

Question, for find line segments is precision more valuable than speed? I'm thinking I'll just change the implementation to this and get rid of the old one. Also, what's your opinion on merging lines? This algorithm doesn't do that so you're going to get a lot of tiny lines. I need to put a front end line merger on it.

Try it out and let me know if you think it outputs too many tiny lines.
Nyamekye,
mjs513
Posts: 71
Joined: Sun Apr 30, 2017 12:52 pm

Re: Lines Detected Change on successive images

Postby mjs513 » Sun Sep 03, 2017 10:31 pm

Hi Nyamekye, Can't tell you how much I appreciate the time you are spending on this. As to your question on speed vs accuracy. Isn't that always the issue.

I know this may sound like a cope out but I think there is probably a break even point between the two, you want to kind of balance the two. In other words you want to get it as accurate as possible while keeping the speed at least reasonable. For instance, when using my current algorithm I'll get double digit FPS but the minute I try to send a image stream over wifi it drops to about 2FPS, which is unacceptable while with out it is acceptable.

What I think may be more important is consistency from frame to frame analysis. Right now the data changes so quickly from frame to frame its hard to do any sort of further analysis of the image. In this case yes I would sacrifice some speed for accuracy or consistency from frame to frame.

As for the finding a bunch of tiny lines vs merging lines? I have to give a try to see how bad it really is, there must be some threshold values that can be set. I know from playing around with the lighting modes you do have some control - I am running night mode more often than not which is rather surprising. I give it a try as soon as I can over the next couple of days - found doing things late at night I make more mistakes :)

Thanks
Mike

PS. I saw the merge :)
User avatar
kwagyeman
Posts: 1175
Joined: Sun May 24, 2015 2:10 pm

Re: Lines Detected Change on successive images

Postby kwagyeman » Sun Sep 03, 2017 11:02 pm

Okay, here's my final result along with a test script. It runs at 3 FPS at 160x120. 8 FPS at 80x60. It's still usable at 80x60. Let me know if this is useful to you. Some other folks wanted this algorithm to be more stable... so, I'm thinking of just changing to this code. I have normal find lines for fast line tracking.
Attachments
openmv.zip
(1.99 MiB) Downloaded 12 times
Nyamekye,
mjs513
Posts: 71
Joined: Sun Apr 30, 2017 12:52 pm

Re: Lines Detected Change on successive images

Postby mjs513 » Mon Sep 04, 2017 11:17 am

Hi K. Started playing with it and yes it can be useful. Finding it interesting that it works a bit better in lower brightness (i used auto setting from my previous posts). The hard lines are much more consistent between frames. Going to play more tonight and tomorrow - getting ready for the party.

Mike
mjs513
Posts: 71
Joined: Sun Apr 30, 2017 12:52 pm

Re: Lines Detected Change on successive images

Postby mjs513 » Mon Sep 04, 2017 7:13 pm

This is getting more interesting. I decided to print the line info and looked like there were more lines on the screen than there was line info, or it could just be timing for the print. Anyway, wanted to know if there was a way to access the individual line info that is printed:

Code: Select all

{x1:41, y1:67, x2:64, y2:66, length:23, magnitude:19, theta:88, rho:69}
In other words something like this:
line().coord
line().length
line().magnitude
......

Thinking that you may be able to filter for all magnitude lines > certain value?

Thanks
Mike
User avatar
kwagyeman
Posts: 1175
Joined: Sun May 24, 2015 2:10 pm

Re: Lines Detected Change on successive images

Postby kwagyeman » Mon Sep 04, 2017 10:37 pm

Yeah, it works exactly like that:

http://docs.openmv.io/library/omv.image ... ine-object

And yes, there are more lines than you think now because it does a bunch of small segments when not joining anything.

...

Note that magnitude is the strength of the line quality. The lower the more likely that line is noise.
Nyamekye,
mjs513
Posts: 71
Joined: Sun Apr 30, 2017 12:52 pm

Re: Lines Detected Change on successive images

Postby mjs513 » Fri Sep 08, 2017 9:03 pm

Hi K. Sorry for not posting sooner. I have been busy with a couple other projects. Anyway, I've been playing around with the function for a few days on and off now including using magnitude as a filter as well as looking lines only in the lower have of the screen. It does work nicely if there is sufficient contrast in the image as well as lighting to see distinct edges. In my case I am looking at scenes where the gradient in colors or grayscale is not always distinct. I got it functioning better by using the auto lighting mode from the app note as well as boosting the contrast to 2. One of the things I noticed is that in some cases I am still getting variation in lines from frame to frame (like its cycling). Was wondering if there is a way to take say x number of frames, read the lines and then create a composite frame of the three and maybe the merge the lines?
User avatar
kwagyeman
Posts: 1175
Joined: Sun May 24, 2015 2:10 pm

Re: Lines Detected Change on successive images

Postby kwagyeman » Fri Sep 08, 2017 11:15 pm

Yeah, just write the frames to the SD card, then read them back and use the blend method to blend them together. It would be optimal if snapshot could blend a frame while capturing it... this is possible but not implemented currently. You can then call find_line_segments on the blended data.

Please let me know if I should replace the previous algorithm...
Nyamekye,
mjs513
Posts: 71
Joined: Sun Apr 30, 2017 12:52 pm

Re: Lines Detected Change on successive images

Postby mjs513 » Sat Sep 09, 2017 9:03 am

Yeah. That's what I thought would have to be done but just was curious if there was another way. As for replacing the old method with the new method. Not 100% sure. On one hand the new method is probably better at finding line segments as well as for me less variables to adjust but if there are a lot of lines found there is a big performance hit versus the old method. The old method is quicker at finding lines and probably with some more tweaking of the variables it would probably find the lines of interest. I did make a video comparing the two methods, https://drive.google.com/open?id=0BwzZj ... WFSQjh6VE0 so you can see for yourself. Please note that the test was in a very dark room with night mode set for both tests.

For me the frame rate hit is not that big of a deal and I could adjust for it but for most users and how they are probably going to use it the old method would be better. Like I said there are pros and cons to both. One is a fast method and the second is a slow method. Not sure which would be better. If the frame rate on the new method was on par I would definitely say the new method. Any way to have both?

I will leave it up to you, as you know the user environment better.

Mike
mjs513
Posts: 71
Joined: Sun Apr 30, 2017 12:52 pm

Re: Lines Detected Change on successive images

Postby mjs513 » Mon Sep 11, 2017 8:23 am

I guess given a choice I would prefer the new method.

Return to “Technical Discussion”

Who is online

Users browsing this forum: No registered users and 1 guest