Use the PortentaVisionShield using the Arduino IDE

I realize this topic is a bit contentious, because Arduino is going to say that it is OpenMV’s responsibility and OpenMV is going to say that it is Arduino’s responsibility, but at some point a customer is going to purchase a PortentaVisionShield not read all the information about it and be really angry that it doesn’t work using any of the 3 Arduino IDE’s.

I know or have forgotten about 40 computer programming languages, so I don’t really care, but it does take years to become expert at any one language. It would be good for everyone if this shield worked on both the Arduino IDE and OpenMV IDE. Both IDE’s have amazing positives that neither one is going to overshadow the other.

Just saying.

What will probably happen is some person is probably going to hack the shield to work on the Arduino. (Enough engineers and makers are on the Arduino forum and sparkfun has a ton of information about the HM01B0 camera). The problem with a hacked library is that it might be really buggy and then that looks bad for both OpenMV and Arduino.

Um, the point of OpenMV IDE is that getting vision working with the Arduino IDE is pointless. It doesn’t have enough tooling. You can take our camera driver and have it working with the Arduino IDE. However, the serial terminal they have is not going to cut it.

Arduino’s partnership with OpenMV is based on the understanding that for computer vision staying in the Arduino C++ world is a little bit limited.

All our C code is online. It’s not hard for anyone to make a driver. However, it’s not like that’s all you have to do. There’s other things you have to add like memory allocation support, SDRAM control, DMA support, etc. You basically end up making OpenMV again.

Good points, Kwagyeman, but all the more reason for Arduino and openMV to make basic working software for the PortentaVisionShield, in C++ for the Arduino IDE. I agree with you that Machine Learning will be much easier using OpenMV , and the Arduino serial connection will be a huge hurdle, but to suggest C, C++ can not handle Machine Learning code as well as Python is just not accurate.

Also your shield has Ethernet, JTAG, an SD card and microphones which many Arduino users are going to want to access in C or C++ even if they don’t use the Camera. I know several potential purchasers who just want the JTAG connectors.

Neither company wants angry customers going all grumpy on social media about issues with the new Portenta or Vision Shield, so best to be proactive and just encourage Arduino to get basic working software ready for your shield. It will not take users very long to realize how much easier your software is to use, and so many trained Machine Learning experts are most comfortable in Python than probably any other language.

Um, it’s not really our shield. You should email Arduino about this. We’ve partnered to support all functionality of it. My goal is to get quality drivers working in our firmware and support for features folks want and need. If you are looking for Arduino to offer more drivers in C for everything then that would be a question directly for them. However, I think the point of our partnership is for them to leverage us for getting everything operational.

Also… it’s an STM32. Like, you can just program it with real professional level tools. STM32 devices come with a HAL and it’s quite easy to use that HAL too bootup hardware and get things going.

And yes, you can write machine learning code in C, this is True :wink:, just more work.

Just thought I would run it past you first to make sure there wasn’t any agreement that only Python would be used on this board. I agree the C, C++ support is something Arduino needs to do. I thought perhaps Arduino made the board, as it looks so similar to the Portenta. Both of them are really nice looking bits of hardware.


Any chance we could work on simplifying the Vision shields direct connections to the J1 and J2 High Density connectors. Everything I have seen has been fairly confusing. It would be nice to see some fixed up diagram like the attached. I just need the information but could generate the diagram. The info must be around somewhere. Users would really need it organized (color) by: JTAG, mic1, mic2, Camera, SD Card, Ethernet.

Note: Attachment has none of those specific connections yet. After thought. I just started understanding this Pin diagram at https://content.arduino.cc/assets/Pinout_VisionShield_latest.pdf Not sure why some connections are labelled with the J1, J2 connectors but other connections aren’t.

I’ll let me know about this.

Hi,
Arduino supports vision shield on IDE. It just takes time to provide full support on various platforms and we felt that the most interesting use of this board would be with openmv that has lots of libraries that can help you quickly do more than just capturing a picture.
We already have a primitive image capture library and an audio example that will be published soon. It would be nice to understand which would be your use cases within arduino language so that we can better tailor support for it.

Thanks,

Dario Pennisi

Hi Dario:

I teach Robotics at a Canadian High School, and an after school kids Machine Learning course with https://www.gearbots.org/stem-academy/ . I will soon choose what hardware to purchase for my Feb 2021 Robotics course and will again next year. I am very interested in the Portenta, the Vision Shield and all other shields for the Portenta including any MKR shields, but like all new products there are a few bugs, (looks like I broke my Vision Shield). Most of the Portenta ones are getting worked out. My Github about the Portenta is at https://github.com/hpssjellis/my-examples-for-the-arduino-portentaH7 and my youtube video playlist about the Portenta is at https://www.youtube.com/playlist?list=PL57Dnr1H_egtm0pi-okmG0iE_X5dROaLw. I work really fast and make lots of mistakes.

I learn everything I can about a product so that my students can make whatever project they want to. I would be most interested in basic abilities with all the visions shields hardware.

  1. How to get Ethernet working with wifi.h,

  2. How to store an audio file on the sd card and on a connected computer

  3. How to store an image file on the sd card and on a connected computer

  4. Examples of how to use JTAG with various debuggers. (I have an ST-link-v2 debugger, Segger J-link EDU mini debugger, but I really like the drag and drop file abilities of the Particle debugger)

All of the above would allow my students to use the Vision Shield in whatever way was helpful for the projects they want to work on.


The OpenMV micropython IDE is very impressive and the Machine Learning capabilities are amazing, especially when connected with ML training using https://www.edgeimpulse.com which if I could get it working appears to have lots of potential, but presently my courses are in C, C++ and Javascript.


I am fairly active on the facebook Portenta group which a few Arduino employees occasionally post. Portenta Facebook Group


Thank you for your interest in my views.

Jeremy

Hello Jeremy and Darrio,

What is the latest on this thread?

I also have the following to ‘play’ with:

https://www.tindie.com/products/tinyvision_ai/upduino-v30-low-cost-lattice-ice40-fpga-board/ - and the Himax adapter (made by tinyvision) that uses the same camera as the Vision Shield.

Regards,
John W.