Not able to deploy a pre-trained model on Nicla Vision using OpenMV IDE

Arduino Boards
, ,
1

I have a small code that identifies the color of the object as red, green, or blue and then prints the same color accordingly, with the LED turning to the same color as well. This is a simple real-time color tracking problem, but I’m having trouble deploying a pre-trained model. I’m constantly encountering OS errors, such as ‘couldn’t find your file’ and ‘enonet’. I’ll attach my Python code here, and all suggestions are welcome. Please help me fix this.

Thanks & Regards
Aatman

import sensor
import image
import time
import tf
import pyb

# Load pretrained TFLite model for Red, Green, Blue classification
with open("model.tflite", "rb") as f:
    model_buf = f.read()

sensor.reset()
sensor.set_pixformat(sensor.RGB565)
sensor.set_framesize(sensor.QVGA)
sensor.skip_frames(time=2000)
sensor.set_auto_gain(False)       # Important for color constancy
sensor.set_auto_whitebal(False)   # Important for color constancy

net = tf.load(model_buf, load_to_fb=True)

# Initialize LEDs: Red = LED(1), Green = LED(2), Blue = LED(3)
led_red = pyb.LED(1)
led_green = pyb.LED(2)
led_blue = pyb.LED(3)

labels = ["Red", "Green", "Blue"]

clock = time.clock()

while True:
    clock.tick()
    img = sensor.snapshot()

    # Run classification on the full frame (or cropped ROI if desired)
    result = net.classify(img)

    if result:
        best = max(result, key=lambda x: x['confidence'])
        detected_color = best['label']
        confidence = best['confidence']

        print("Detected color:", detected_color, "Confidence:", confidence)

        # Turn off all LEDs first
        led_red.off()
        led_green.off()
        led_blue.off()

        # Turn on the corresponding LED
        if detected_color == "Red":
            led_red.on()
        elif detected_color == "Green":
            led_green.on()
        elif detected_color == "Blue":
            led_blue.on()

    else:
        # No detection, turn off all LEDs
        led_red.off()
        led_green.off()
        led_blue.off()

    print("FPS:", clock.fps())

What did I do to combat it? I made a Python byte array and tried to store it in OpenMV internal storage, but it didn’t work :frowning:

2

Hi, you’re on a really old API. We removed the tf module and now there’s a new ML module.

As for loading the model, you just pass the name to ml.Model("path", load_to_fb=True)

You’re going to need to update your firmware. Note, I am about to release a new IDE today.

3

Firstly, thanks for the prompt reply. Here’s what I did now.
I updated the firmware from the bottom right corner and wrote another code importing ml library, but I’m still encountering OS errors like Error 22(EINVAL). I couldn’t find the file even after specifying the path. I believe the error lies somewhere in the path specification, but I’m very new to this, so I would appreciate your help. Here’s the updated code.

import sensor
import time
import ml
from machine import LED

sensor.reset()
sensor.set_pixformat(sensor.RGB565)
sensor.set_framesize(sensor.QQVGA)
sensor.skip_frames(time=2000)

model = ml.Model("C:/Users/aatma/Downloads/model.tflite")
print("Model Loaded:", model)

labels = ["Red", "Green", "Blue"]

clock = time.clock()
led = LED("LED_BLUE")
while True:
    clock.tick()

    img = sensor.snapshot()
    prediction = model.predict([img])[0].flatten().tolist()

        # Pair each label with its confidence
    scores = sorted(zip(labels, prediction), key=lambda x: x[1], reverse=True)

        # Display top-1 result
    print(clock.fps(), "fps\t", "%s = %.2f" % (scores[0][0], scores[0][1]))

        # Optional: Show prediction on screen
    img.draw_string(10, 10, "%s: %.2f" % (scores[0][0], scores[0][1]), color=(255, 0, 0), scale=2)

The idea is to use my pre-trained model on Nicla Vision to identify objects that are red, green, or blue. I’ve been told that OpenMV can’t access files on its own, so what’s the fix now?

Thanks & Regards
Aatman

4

I opened the OpenMV Cam’s Drive Folder and it led me to D Drive (USB for Nicla Vision), where i uploaded the model.tflite file but it was too big to run. Also tried doing load_to_fb = True so that the heavier model can be run through frame buffer but it seems like its too big. So is the size of the model a real problem here or something else ?

5

model = ml.Model(“C:/Users/aatma/Downloads/model.tflite”)
print(“Model Loaded:”, model)

It’s got to be a path on the OpenMV Cam. This would be like “/flash/model.tflite”. The path the OS sees the file at is different from what the camera sees.

Or, just do “model.tflite” for a relative path.

6

So you’re saying that I need to store the .tflite file on the openmv cam folder and then just write “model.tflite” to access ?

7

Yes, becuase this is the file system local to the OpenMV Cam.

It can’t read files off your PC’s disk.