Global Shutter

General discussion about topics related to OpenMV.
rajeshktym
Posts: 23
Joined: Wed Mar 07, 2018 7:04 pm
Location: KOCHI
Contact:

Global Shutter

Postby rajeshktym » Wed Mar 07, 2018 7:09 pm

IS openMV module available with any Global Shutter Sensor? any body tried to modify this hardware for capture moving object with help of hardware trigger?
User avatar
iabdalkader
Posts: 912
Joined: Sun May 24, 2015 3:53 pm

Re: Global Shutter

Postby iabdalkader » Wed Mar 07, 2018 7:44 pm

Hi, our new camera will have an optional global shutter sensor module, in addition to the new H7 (400MHz) MCU upgrade. The new camera will be released this year (hopefully).
rajeshktym
Posts: 23
Joined: Wed Mar 07, 2018 7:04 pm
Location: KOCHI
Contact:

Re: Global Shutter

Postby rajeshktym » Wed Mar 07, 2018 8:38 pm

Thanks , I am working with image process development in different platforms, but new in openmv. for starting, can u suggest your latest tutorial links? my project is develop colour mark detection from moving object and its x,y coordinates send through wireless communication.
User avatar
iabdalkader
Posts: 912
Joined: Sun May 24, 2015 3:53 pm

Re: Global Shutter

Postby iabdalkader » Wed Mar 07, 2018 8:45 pm

We don't really have official tutorials, but if you search the forums you'll find many posts about color tracking, wifi etc.. and the IDE comes with examples too that you should use as a starting point.
User avatar
kwagyeman
Posts: 3188
Joined: Sun May 24, 2015 2:10 pm

Re: Global Shutter

Postby kwagyeman » Thu Mar 08, 2018 2:12 am

Hi, the IDE has a lot of examples and it's really a learn by doing thing right now. Please see the File->Examples folder in the IDE.
Nyamekye,
michaelp
Posts: 4
Joined: Fri Mar 30, 2018 11:22 am

Re: Global Shutter

Postby michaelp » Fri Mar 30, 2018 11:29 am

Can you share which global shutter camera sensor in particular you are going to use?
I'm asking because I'm working on a project where OpenMV would be the perfect platform for, but it's quite sensitive to the particular camera characteristics, so knowing the exact sensor would help a lot in deciding whether I should base the project on OpenMV or not.
User avatar
kwagyeman
Posts: 3188
Joined: Sun May 24, 2015 2:10 pm

Re: Global Shutter

Postby kwagyeman » Fri Mar 30, 2018 11:50 am

Sure, it's the MT9V034. It's the cheapest global shutter sensor of good quality that can do grayscale and color. We'll be buying the Bayer pattern version of the sensor. Unfortunately, it doesn't have much of an ISP onboard so we will be doing our best to process the image and do color correction, sharpening, etc.
Nyamekye,
michaelp
Posts: 4
Joined: Fri Mar 30, 2018 11:22 am

Re: Global Shutter

Postby michaelp » Sat Mar 31, 2018 2:17 am

Thanks, that's good news! I was mostly concerned about light sensitivity but that shouldn't be a problem with the MT9V034!
rajeshktym
Posts: 23
Joined: Wed Mar 07, 2018 7:04 pm
Location: KOCHI
Contact:

Re: Global Shutter

Postby rajeshktym » Fri Sep 28, 2018 10:05 pm

and finally I show ur announcement about global shutter launch, I also order one through kickstart and waiting, kickstart says march 2019, but I saw some where ur comments same in September 2018, can you please tell me when may be it can expect .
User avatar
kwagyeman
Posts: 3188
Joined: Sun May 24, 2015 2:10 pm

Re: Global Shutter

Postby kwagyeman » Fri Sep 28, 2018 11:09 pm

Um, September is when we launch the Kickstarter. Next year is when we deliver. The schedule is determined by the lead-time from ST to get the main MCU chip (because we will deliver all goods at the same time). I put the PO in for the 10k ST chips in yesterday so that's on it's way now. We were able to do do that with funds saved up from selling the OpenMV Cam M7. The Kickstarter funds will then be put towards buying everything but the main MCU.
Nyamekye,
rajeshktym
Posts: 23
Joined: Wed Mar 07, 2018 7:04 pm
Location: KOCHI
Contact:

Re: Global Shutter

Postby rajeshktym » Sat Apr 27, 2019 8:24 am

Hi,
I have received a tracking No ( 'RF730155359SG') for my global shutter OPENMVand waiting to deliver with in two days, I haven't find any documents that which pin should configure for external trigger. can you share any tutorial about it ? Thanks
User avatar
kwagyeman
Posts: 3188
Joined: Sun May 24, 2015 2:10 pm

Re: Global Shutter

Postby kwagyeman » Sat Apr 27, 2019 11:16 am

Hi, there's an example that comes with the IDE to put the camera into trigger mode. One that done snapshot triggers the camera.

As for syncing the trigger to another source you need to just use regular I/O pins and wait for a low to high transition and then call snapshot.
Nyamekye,
rajeshktym
Posts: 23
Joined: Wed Mar 07, 2018 7:04 pm
Location: KOCHI
Contact:

Re: Global Shutter

Postby rajeshktym » Sat Apr 27, 2019 8:22 pm

"As for syncing the trigger to another source you need to just use regular I/O pins "

Which I/O Pin I should use for External Trigger?
User avatar
kwagyeman
Posts: 3188
Joined: Sun May 24, 2015 2:10 pm

Re: Global Shutter

Postby kwagyeman » Sat Apr 27, 2019 10:49 pm

Any, you have to write the external trigger in software. I.e create a loop waiting for the pin to go low and then high. Don't use interrupts. You cannot call snapshot in an interrupt.
Nyamekye,
rajeshktym
Posts: 23
Joined: Wed Mar 07, 2018 7:04 pm
Location: KOCHI
Contact:

Re: Global Shutter

Postby rajeshktym » Thu May 02, 2019 7:05 am

Ok Thanks I have received my OpenMV board with additional MT9V034 Module, it working with examples and now I just change MT9V034 Module, grayscale format it is working, but other formats are not, Is it won't support colour?
User avatar
kwagyeman
Posts: 3188
Joined: Sun May 24, 2015 2:10 pm

Re: Global Shutter

Postby kwagyeman » Thu May 02, 2019 10:57 am

Yeah, it doesn't support color. We can't do color with that sensor because it has no onboard ISP that will do debayering for you. So, 100% of the processor time would be spent debayering the image instead of processing it.

Anyway, given that. For pretty much every application we heard from customers who wanted that sensor color is not needed.
Nyamekye,
rajeshktym
Posts: 23
Joined: Wed Mar 07, 2018 7:04 pm
Location: KOCHI
Contact:

Re: Global Shutter

Postby rajeshktym » Fri May 03, 2019 12:53 am

Hi Sir,

Any plan for this sensor with color detection and other features?


Rajesh Kumar
User avatar
kwagyeman
Posts: 3188
Joined: Sun May 24, 2015 2:10 pm

Re: Global Shutter

Postby kwagyeman » Fri May 03, 2019 1:23 am

No, given the previous point. You can buy a color version of the sensor however and build a board with it. The parts can all be bought online if you want to build it. The C code to do debayering is even already there too.
Nyamekye,
bogflap
Posts: 44
Joined: Mon Apr 16, 2018 10:03 am

Re: Global Shutter

Postby bogflap » Fri May 03, 2019 4:15 am

Shame here is no colour. Reduces the practical uses of the module for me dramatically to the point of being useless
rajeshktym
Posts: 23
Joined: Wed Mar 07, 2018 7:04 pm
Location: KOCHI
Contact:

Re: Global Shutter

Postby rajeshktym » Fri May 03, 2019 8:03 am

really I was waiting this release for my colour object tracking project, which I mentioned beginning. previous post was mentioned planning to use bayer sensor, any way if it won't support colour,I will get one colour sensor. In my place MT9V032 colour sensor available, I will start experiment with it.
User avatar
kwagyeman
Posts: 3188
Joined: Sun May 24, 2015 2:10 pm

Re: Global Shutter

Postby kwagyeman » Fri May 03, 2019 5:00 pm

@blogflap - Okay. We built the sensor for folks using the system in production environments. The use cases are for capturing precision images of objects moving on factory production lines. Every example use case did not require color.

@rajeshktyn - Just buy the MT9V034C sensor that supports color. That said, you're on your own for image processing. The image has to be in VGA or higher res. Then, you have to de-bayer and down-sample. This is extremely CPU intensive.
Nyamekye,
User avatar
kwiley
Posts: 107
Joined: Fri Dec 21, 2018 5:22 pm

Re: Global Shutter

Postby kwiley » Sun May 05, 2019 12:11 am

I'm looking forward to the imminent arrival of my H7 and global shutter. Color is not at all important to me (well, that's overstating it -- given unlimited resources, I would happily *accept* color, haha). My question is about the theoretical 200fps framerate. Do the higher framerates require downsampling? Is 200fps only at, oh say, 40x30 pixels or something of that sort? Also, for all practical purposes, what sort of processing to you realistically expect the H7 to be able to perform at high framerates even if the shutter itself can produce such rates? I mean, a 200fps sensor attached to a processor that can only perform any useful algorithm a all (even just iterate through the pixels for heaven's sake) at a much lower is confusing to me. Or, alternatively, is the 200fps only achievable in blind recording, just dumping sensor data to an SD card as fast as possible without imposing any computational overhead in the processor?

Thanks.
User avatar
kwagyeman
Posts: 3188
Joined: Sun May 24, 2015 2:10 pm

Re: Global Shutter

Postby kwagyeman » Sun May 05, 2019 12:27 am

200 FPS is at the 80x60. 400 fps is 40x30. As for you actually being able to do something at that frame rate, that's not too important. What's important is that the latency is extremely low.

I.e. if you are building a line following robot, doing quad copter stablization, etc. the time between the current image and the next one is <10ms. For example with the OV7725 camera you can't get much beyond 80 FPS. Generally, you can only achieve half of what the raw rate is so your algorithm will only hit 40 fps (25ms) with the OV7725. With a 200 FPS frame rate and dropping every other frame you're now in the 10 ms area. At 400 fps you're in the 5ms area.

...

In regards to the frame rates coming into the system, the 400 fps is the rate at which is present in RAM after the DCMI hardware sucks the image in. Assuming you write some tight C code - yes, you can do something with the image. You'll have to plan what you want to do carefully however.

...

For our Russian line follower friends, they will be able to use get_regession() (not-robust) to track a line under the robot at easily 100-200 FPS as the non-robust linear regression is a very fast single pass algorithm.

...

One note however - to get the high frame rate you have to lower the exposure. So, the image becomes dark. You can use gamma correction to make the image brighter however.
Nyamekye,
User avatar
kwiley
Posts: 107
Joined: Fri Dec 21, 2018 5:22 pm

Re: Global Shutter

Postby kwiley » Sun May 05, 2019 2:01 am

I'm familiar with the exposure trade off and gamma corrective measures. I was experimenting with a 3500 fps camera recently. Thanks.
rajeshktym
Posts: 23
Joined: Wed Mar 07, 2018 7:04 pm
Location: KOCHI
Contact:

Re: Global Shutter

Postby rajeshktym » Wed May 08, 2019 4:11 am

Hi
Anybody tried global shutter frame capturing based on Hardware exposure pin ( PB4 - SPI3_MISO) ?
User avatar
kwagyeman
Posts: 3188
Joined: Sun May 24, 2015 2:10 pm

Re: Global Shutter

Postby kwagyeman » Wed May 08, 2019 3:51 pm

There's an example showing how to turn on hardware triggering. It's built-into the firmware.

I know already of the customer who used it to capture images of moving objects.
Nyamekye,
rajeshktym
Posts: 23
Joined: Wed Mar 07, 2018 7:04 pm
Location: KOCHI
Contact:

Re: Global Shutter

Postby rajeshktym » Wed May 08, 2019 8:04 pm

Hi, Can you please share that Hardware Trigger tutorial / sample
User avatar
kwagyeman
Posts: 3188
Joined: Sun May 24, 2015 2:10 pm

Re: Global Shutter

Postby kwagyeman » Thu May 09, 2019 1:57 pm

Examples -> Global Shutter -> Triggered Mode.

When you call snapshot it takes a picture. If you want to sync to an external source just use pyb to wait for an I/O pin to go low then high. E.g:

Code: Select all

while pyb.PIN(0).value(): # wait for pin to be low
    pass
while not pyb.PIN(1).value(): # wait for pin to be high
    pass
img = sensor.snapshot()
Nyamekye,
rajeshktym
Posts: 23
Joined: Wed Mar 07, 2018 7:04 pm
Location: KOCHI
Contact:

Re: Global Shutter

Postby rajeshktym » Fri May 10, 2019 12:26 am

Hi,

Thanks for your help, I have tried this but didn't give expected result, my object is moving fast (15 FPS) and capture only a small potion of the moving object means my trigger will work every 66.6 milliseconds but the position accuracy need to maintain 60 micro Second. so I think I need to use hardware trigger directly exposure pin of MT9V034 which connected to PB4 of STM32. so I have map STM32 PB4 pin to openMV output pin and use directly from trigger encoder. any body have any other suggestion to achieve this with openMV board.
User avatar
kwiley
Posts: 107
Joined: Fri Dec 21, 2018 5:22 pm

Re: Global Shutter

Postby kwiley » Fri May 17, 2019 9:45 am

On the whole color debate above, one very application-specific approach would be to record Bayer filter data (i.e., raw sensor data) straight onto the SD card, such that it could be deBayered and processed at a later time. In such a way, one could use a global shutter to record high framerate color data for later offline processing. This would be a totally unrelated application from any intent at online color-image processing, regardless of framerate, because as you said, deBayering on the H7 would ruin your performance.

I also recognize that the proposal of recording Bayer data for later offline use, without necessarily using the sensor data online at all, is too specific for you to have designed the H7 or the global shutter module toward, so no worries.

...although now that I think about it, I can imagine a use case in which you record Bayer data for offline use, but directly process the Bayer matrix in grayscale form online. A simple 2X decimation would give you a decent grayscale luminance image for, perhaps, object tracking -- of an object that you might then later process in color offline.

...but, I can't recall now whether it is possible to simultaneously record to SD card and process images "in-app", or whether you have to choose between shuttling the frame buffer directly to the card or otherwise processing it on the fly.
User avatar
kwagyeman
Posts: 3188
Joined: Sun May 24, 2015 2:10 pm

Re: Global Shutter

Postby kwagyeman » Fri May 17, 2019 11:37 am

Well, you can build a unit with a color sensor. You just have to take our files and switch out the MT9V034 part to the color color one. Our code then won't line the difference. As long as you keep the res above vga you have a bayer image.
Nyamekye,
jl1982fm
Posts: 11
Joined: Thu Jul 26, 2018 4:09 am

Re: Global Shutter

Postby jl1982fm » Wed Jun 19, 2019 3:55 am

kwagyeman wrote:
Fri May 17, 2019 11:37 am
Well, you can build a unit with a color sensor. You just have to take our files and switch out the MTV part to.thr color one. Our code then won't line the difference. As long as you keep the res above vga you have a bayer image.
Would be great if you can explain this a bit further, what it the MTV and where those files are located?

Thanks a lot
User avatar
kwagyeman
Posts: 3188
Joined: Sun May 24, 2015 2:10 pm

Re: Global Shutter

Postby kwagyeman » Wed Jun 19, 2019 10:49 am

Hi, please see the Openmv boards repo and see the OMV4 board and the sensor modules of it. The design files for the global shutter camera are there.
Nyamekye,

Return to “General Topics”

Who is online

Users browsing this forum: No registered users and 23 guests