Binocular camera based on Portenta H7 and OpenMV

OpenMV related project discussion.
Sergey Burenkov
Posts: 4
Joined: Tue Feb 09, 2021 8:30 am

Binocular camera based on Portenta H7 and OpenMV

Postby Sergey Burenkov » Tue Feb 09, 2021 11:30 am

Hi,
I started my hobbyist project, which is goal to get a binocular camera and investigate algorithms for processing stereo images on a micropython. My camera will based on pair of MT9V034 monochrome sensors and MAX10 Intel FPGA. The host board is Portenta H7, as I already have it. My first prototype is consist of Portenta itself, carrier board, Arrow Deca MAX10 evaluation kit and some amount of wires.
20210209_172721.jpg
At the first step I want to get internally generated image from FPGA, I generated simple monochrome bars with 320x240 frame size and output to DCMI. I don't want to implement my own driver at this moment, so I try to emulate HM01B0 image sensor which is already supported by OpenMV for Portenta H7 and Vision shield.
I connected DCMI interface(DATA, VS, HS) and I2C to FPGA. And I faced with first problem - In some reasons I can't see any I2C transaction during IDE connection. And of course IDE didn't recognize image sensor. So could you please give me start point where in the sources I can see connection procedure? All I can found is sensor_init(), and if I understand correctly, I2C scanning sould performed here in any case, so I should see it inside fpga, but I cannot see it... I thing I missed something.
I'm sure about I2C connection, it uses same pins as Vision shield. Aslo if I push Tools->Reset OpenMV Cam, I see something on this pins, so physical connection alive. After connection I received following in terminal :

Code: Select all

Traceback (most recent call last):
  File "/main.py", line 11, in <module>
Exception: IDE interrupt
MicroPython: v1.13-r63 OpenMV: v3.9.2 HAL: v1.9.0 BOARD: PORTENTA-STM32H747
Type "help()" for more information.
>>> 
Don't sure it is important, I try to run micropython after that(with commented sensor-related lines) and it runs successfully...
Any advice might be helpful
Thanks!
User avatar
kwagyeman
Posts: 5001
Joined: Sun May 24, 2015 2:10 pm

Re: Binocular camera based on Portenta H7 and OpenMV

Postby kwagyeman » Tue Feb 09, 2021 12:54 pm

Hi, do you have a protocol analyzer? You should defiantly see a bunch of I2C toggles on startup by the OpenMV Cam. Note that the camera scans the I2C bus on power-on. Not on when you script is run.

Also, make sure the I2C bus has pull-ups.
Nyamekye,
Sergey Burenkov
Posts: 4
Joined: Tue Feb 09, 2021 8:30 am

Re: Binocular camera based on Portenta H7 and OpenMV

Postby Sergey Burenkov » Wed Feb 10, 2021 2:14 am

kwagyeman wrote:
Tue Feb 09, 2021 12:54 pm
Hi, do you have a protocol analyzer? You should defiantly see a bunch of I2C toggles on startup by the OpenMV Cam. Note that the camera scans the I2C bus on power-on. Not on when you script is run.

Also, make sure the I2C bus has pull-ups.
Yes, I use something like I2C protocol analyzer inside fpga. I found I2C toggles on startup! It was my fault, I searched it on IDE connection before. Now sensor detected!
1.png
1.png (4.25 KiB) Viewed 184 times
Thanks, now I will continue to work on sensor protocol supporting.
User avatar
kwagyeman
Posts: 5001
Joined: Sun May 24, 2015 2:10 pm

Re: Binocular camera based on Portenta H7 and OpenMV

Postby kwagyeman » Wed Feb 10, 2021 11:34 am

Regarding stereo image processing. This is interesting to me. If you make a dev board with the FPGA that generates a 2x width image from each camera combined I’ll write SIMD accelerated block matching code for you.

So, focus on making the hardware that can generate the image for the Arduino Portenta and I’ll jump in with the stereo algorithm.
Nyamekye,
Sergey Burenkov
Posts: 4
Joined: Tue Feb 09, 2021 8:30 am

Re: Binocular camera based on Portenta H7 and OpenMV

Postby Sergey Burenkov » Wed Feb 10, 2021 2:52 pm

Thank you, it is great! Let's discuss hardware some later, as soon as I get image on my current development boards. I need to estimate FPGA resources required and choose chip capacity before starting PCB development.
It is background project, so progress will not fast :)
Sergey Burenkov
Posts: 4
Joined: Tue Feb 09, 2021 8:30 am

Re: Binocular camera based on Portenta H7 and OpenMV

Postby Sergey Burenkov » Wed Feb 17, 2021 8:38 am

Got video finally!
2.png
Next step is assemble PCB with pair of MT9V034 and and get video from at least one sensor.
Components is ready for now
20210217_163533.jpg
User avatar
kwagyeman
Posts: 5001
Joined: Sun May 24, 2015 2:10 pm

Re: Binocular camera based on Portenta H7 and OpenMV

Postby kwagyeman » Wed Feb 17, 2021 12:35 pm

Nice!
Nyamekye,

Return to “Project Discussion”

Who is online

Users browsing this forum: No registered users and 1 guest