Hi @kwagyeman , I’m looking to do the same of @RichardW with Yolo8 on the N6 just arrived. I’ve seen from the examples that the file "yolov8n_192.txt "with the lables in the ROMFS contain just the lable “person”. To detect the remaining classes or select just some of them is enough update the file with all the 80 class names or must be change also the file with the Yolo8 model ? The lable file of Yolo if I’m not wrong have class names written in a specific sequence (person ,car,bicycle,motorcycle etc..) so if I want detect a class that is at specific postion do I’ve to include all lables till that position ? tks.
Hi Bob, the models that come with the camera only detect one class. If you’d like to detect more than one class, you have to train new models. The label file just has the classes the model detects in it.
This is what Roboflow is for, to train custom models.
You can generally run any of the models here on the N6: GitHub - STMicroelectronics/stm32ai-modelzoo: AI Model Zoo for STM32 devices · GitHub
However, each model needs post-processing to turn the output of the model into something interesting to use from generic vectors of floating-point values. This wouldn’t be too hard if things were documented; however, you basically have to go on an archaeological expedition to find all the details to write a post-processor. Unless you are very comfortable with working with models, I wouldn’t just use Roboflow’s method, which will output YOLOV8 models, for which we already have a post-processor. You just update the yolov8 example script with the new model and you are good.
@kwagyeman , ok now is clear. No I’m not so expert to write a post-processing or enter so deeply into trasfer learning etc. I’ll try to see what can I do with Roboflow . I’ve created a user but asn’t asked any specific promotional code as you provided by mail. Is it normal ? About pose detection do you thing to implement also the full body pode detection with mediapipe ?
Hi, I will be implementing full body pose detection soon. It just needs a post processor. Otherwise you can make it work by grabbing a mode from where I linked from STs Model Zoo.
As for the code, it will be needed when you need to pay for more credits to use their service.
@RichardW here’s a new release of the IDE with a new feature that allows you to tell the camera to enter the bootloader: Release Development Release · openmv/openmv-ide · GitHub
After you install it, see Tools→Force enter OpenMV Cam bootloader. Once you click that, while connected, the camera will enter its bootloader and not exit. From there you should be able to update the romfs without fear of the device not enumerating in time.
If you still have problems after this then please send me a picture of the windows device manager, I should be able to see the camera’s bootloader as a dfu device (or a warning sign as a non-enumerated device).
I’ve loaded the old version of the IDE onto my Surface laptop and have had success with editing and updating the ROMFS! I will also load up your new version later onto my desktop and see if that solves the problem I was experiencing on my desktop… thanks!
I’ve succeeded now in training a new model to detect 5 speific objects using Roboflow, deployed the Yolo8 model, uploaded the new model to my ROMFS and has tested the model on untrained images successfully. I’ve also tested the LMU gyroscope, accelerometer, pitch & roll and I can attest that your AE3 is performing as advertised! No issues so far - will now start testing the deepsleep states and wakeup using low power LMU and timers…
Okay, note that the current released deep sleep isn’t the best. The 80uA is still in upstream MicroPython that also supports wake-up from the user button. We need to get the v5.0.0 release done for that to get in. It’s still at 500uA right now and wake-up from I/O pins doesn’t work too.
Can u expand on this… are u saying that there is a hardware error with deep sleep or is it a software issue that you think can be solved with current AE3?
I’m planning to place AE3 into a deep sleep state with periodic wake up based on programming a variable hw timer countdown + motion detection. I suspect this will be a common practice with your user base.
If you belive this can work with current AE3 then could you provide sample code or describe if /when this capability would be ready?
The feature is completed and merged upstream. We just have to pull it into our downstream OpenMV firmware. The feature is in MicroPython master right now. We fork off of that and periodically pull releases of MicroPython master into our downstream firmware.
At the moment, you can develop your application without worrying about this. I am just saying that the current draw in sleep will improve from 500uA to 80uA once we pull the changes in.
We currently support wakeup via the RTC onboard. Wakeup via motion via the accel/gyro, ToF, or the user button being pressed will be possible once the changes upstream are pulled into our downstream firmware.
Perfect! Thanks…
I’ve used Edge Impulse (free version) to train an audio classification model. this was quick and simple to do. I deployed using the “Alif DevKit Gen2 HE core” and Inference Engine “TensorFlow Lite”. These seemed the closed fit - are these selections correct or do you recommend a better selection from the options?
I’m now trying to load the final model to my ROM FS but can you confirm that the file name should be the file type .lite (LITE file)?
OpenMV IDE will handle .lite or tflite. I’d recommend. TLFite though as we use that everywhere.
See our MicroSpeech example for audio classification. It does runs TensorFlows audio pre-processor network to generate the right data for the audio classification. However, this may not be what you want. But, you can study the code here for how to handle audio:
tried loading the .lite file to the ROM FS and have the error message:
Error: Reading input file ‘fileXYZ.lite’: Unsupported file extension. Only .tflite and .tosa files are supported.
I don’t see how to create .tflite files using Edge Impulse… any suggestions appreciated…
Just rename it to .tflite. It’s the same file type. That error is coming from arm vela compilier? Like, does that show in a dialog window? If so, then I need to make the IDE handle renaming that for you automatically.
the error came from the OpenMV IDE when trying to load the file to the ROM FS- by renaming the file to .tflite the error went away.
Now that I’ve been able to load the .tflite model into ROM FS by renaming the .lite files I have a new error I’m trying to understand. When I run a script on the AE3 the error message is ValueError: input shape does not match the model input shape.
I presume this means that I’ve selected incorrect deployment targets and inference engines?
From the Edge Impulse Deployment section I selected Deployment target: Alif Ai:ML Kit Gen2 HE core and the Infer once engine: TensorFlow Lite
That’s Alif’s official devkit not our board. EdgeImpulse doesn’t seem to support any of our new boards, so it will not work out of the box. The model itself should work fine, but you’ll need to write your own pre/post-processing functions, and figure out a way to feed the Audio samples to the model.
Ok - sounds way beyond my capabilities at present. Is there a plan to have Edge support the AE3? If so - any idea of timelines?
Hi Richard,
I can ask the EI team. They are aware of our new products.
What I can do right now is ask them for how to run this network. It might not be that hard to handle the pre-post/processing.