Hello, how to connect a display to nicla vision. I have a project running in OpenMV IDE, basically it’s a machine learning model for classifying tomatoes. My project is running without problems, but I want to connect a display to see the same thing that the openmv ide camera shows me.
I was watching this tutorial (https://youtu.be/21gAfCXcHSk) and they are using an LCD Display but I’m not sure if it is compatible with arduino nicla vision.
Yes, you can use a display with the system.
You need to hook the SPI pins to these: openmv/src/omv/boards/ARDUINO_NICLA_VISION/omv_boardconfig.h at master · openmv/openmv (github.com)
You can get the Nicla Pinout here: ABX00051-full-pinout.pdf (arduino.cc)
And the LCD shield pinout here: https://openmv.io/cdn/shop/files/lcd-pinout_2048x.png?v=1613516978
There’s 6 data pins and power/ground you have to hookup. Then you can use the SPI Disiplay module to send images to the screen.
See the LCD Examples. Here’s the class here: class SPIDisplay – SPI Display Driver — MicroPython 1.23 documentation (openmv.io)
Note, the pins mentioned in the class aren’t the same the Nicla uses. Please follow my links above for that.
Hi, I don’t know how the LCD shield you show above works. Please provide the datasheet for it.
Hi, so, that display looks like the same display we use for the OpenMV Cam normally. However, there are strap pins on these displays which change the read-out order and etc. Since you are not using the hardware we sell I can only provide limited help. If you need to modify the driver you’ll need to do it yourself in Python. That said, other customers have done this and it’s not impossible.
Anyway, as described in this post: DIsplay to Nicla Vision - #2 by kwagyeman… you have the wiring correct as far as I can tell. It’s not clear that the A0 pin is equal to the RS pin and that SDA is equal to MOSI. Assuming this is right then it should work.
LED being set to 3.3V should light the display.
Anyway, run the LCD shield example on the Nicla and you should get some sort of image. The bytes may be reversed, if so, then you can turn on a flag to fix that. class SPIDisplay – SPI Display Driver — MicroPython 1.23 documentation
firmware.zip (1.0 MB)
Sorry, it’s not enabled by default. Here’s a firmware snapshot with it enabled.
import sensor, time, ml, uos, gc, display
sensor.reset() # Reset and initialize the sensor.
sensor.set_pixformat(sensor.RGB565) # Set pixel format to RGB565 (or GRAYSCALE)
sensor.set_framesize(sensor.QVGA) # Set frame size to QVGA (320x240)
sensor.set_windowing((160, 160)) # Set 160x160 window.
sensor.skip_frames(time=2000) # Let the camera adjust.
net = None
labels = None
lcd = display.SPIDisplay()
try:
# load the model, alloc the model file on the heap if we have at least 64K free after loading
net = ml.Model("trained.tflite", load_to_fb=uos.stat('trained.tflite')[6] > (gc.mem_free() - (64*1024)))
except Exception as e:
print(e)
raise Exception('Failed to load "trained.tflite", did you copy the .tflite and labels.txt file onto the mass-storage device? (' + str(e) + ')')
try:
labels = [line.rstrip('\n') for line in open("labels.txt")]
except Exception as e:
raise Exception('Failed to load "labels.txt", did you copy the .tflite and labels.txt file onto the mass-storage device? (' + str(e) + ')')
clock = time.clock()
while(True):
clock.tick()
img = sensor.snapshot()
predictions = net.predict([img])[0].flatten().tolist()
predictions_list = list(zip(labels, predictions))
# Encontrar la clase con el porcentaje más alto
top_class = max(predictions_list, key=lambda x: x[1])
top_label, top_score = top_class
# Mostrar la clase clasificada en el buffer de OpenMV
#img.draw_string(0, 0, f"{top_label}: {top_score:.2f}", color=(255, 0, 0), scale=1)
# Mostrar la clase clasificada en el buffer de OpenMV
img.draw_string(0, 10, f"{top_label}", color=(255, 0, 0), scale=0.9) # Nombre de la clase
img.draw_string(0, 20, f"{top_score:.2f}", color=(255, 0, 0), scale=1) # Porcentaje en la línea de abajo
# Imprimir resultados en la consola
for label, score in predictions_list:
print(f"{label} = {score:.2f}")
print(f"Clasificación: {top_label} = {top_score:.2f}")
print(clock.fps(), "fps")
lcd.write(img)
Hi, I’d recommend adding a physical switch to just remove battery or USB-C power. The RT1062 has an on off button. The Nicla does not.
Hi @kwagyeman , I am trying to use Nicla Vision and OpenMV LCD Shield for display.
I have used the LCD shield pinout and the Nicla pinout you mentioned and below are the combinations of pins
OpenMV LCD Shield >> Nicla Vision
SPI Data In - to LCD >> COPI
SPI Clock >> SCLK
SPI Chip Select >> CS
Backlight Control + 3.3V Rail >> (OUT) VDDIO_EXT
SPI Command >> D0/LPIO0
LCD Reset >> D3/LPIO3
GND Rail >> GND
Following the code i have used in openmv is
import sensor, display
# Setup camera.
sensor.reset()
sensor.set_pixformat(sensor.RGB565)
sensor.set_framesize(sensor.LCD)
sensor.skip_frames()
lcd = display.SPIDisplay()
# Show image.
while(True):
lcd.write(sensor.snapshot())
I have also flashed the firmware you attached before. The code runs without any errors, but the display does not show the images (display just glows white). Could you provide any insights on what might be wrong?
Hi, it’s these pins:
I just hooked it up and it totally works. However, if your wires are bad this will cause it not to work.
Note that you have to enable the display module on the nicla:
Thanks @kwagyeman for your reply.
Currently i have downloaded the firmware you provided in previous post (shown in screenshot)
and flashed it on nicla vision using OpenMV IDE, but i am not sure after updating this MICRO_PY_DISPLAY , how i can flash this. Can you please provide any source or provide guidance for doing it.
firmware.zip (1007.3 KB)
Here’s the firmware attached.
Hi @kwagyeman , I have tried using this firmware the code runs but it still shows just white screen.
Can you please verify my connections with yours. I am providing a diagram below.
I also checked for voltage output of each pin from nicla vision to LCD Shield
COPI to SDI is 1.7V
SCLK to SPI CLK is 1.5 to 1.6V
CS to SPI CHP Select 0V
PG12 to RS(SPI command) 3.3V
PG1 to RST 3.3V
Hi, that pinout is correct. When I hooked it up and ran the code with that firmware, it just worked.
Note that you don’t need to connect LED to VCC as it’s already that way internally on the LCD screen. However, this wouldn’t cause it not to work.
If you have a multimeter or a scope, you may wish to check if there’s a signal toggling. The 1.7V and 1.5V to 1.6V imply there’s a data waveform on the COPI and SPI pins.
