Model TFLite embedded in OpenMV Cam firmware

paolo · August 18, 2022, 7:47am

Hi, is it possible to embed a tflite model into the OpenMV Cam firmware? This will might speed up the inference and “hide” the model from the flash drive that is not protected.

iabdalkader · August 18, 2022, 2:17pm

Yes:

github.com

openmv/openmv/blob/master/src/lib/libtf/README.md#embedding-tensorflow-models-in-the-firmware

# Tensorflow Support
The OpenMV firmware supports loading quantized Tensorflow Lite models. The firmware supports loading external models that reside on the filesystem to memory (on boards with SDRAM), and internal models (embedded into the firmware) in place. To load an external Tensorflow model from the filesystem from Python use [`tf`](https://docs.openmv.io/library/omv.tf.html) Python module. For information on embedding Tensorflow models into the firmware, and loading them, please see the following sections.
  - [Training Tensorflow models](#training-tensorflow-models)
  - [Embedding Tensorflow models in the firmware](#embedding-tensorflow-models-in-the-firmware)
  - [Loading embedded Tensorflow models](#loading-embedded-tensorflow-models)

## Training Tensorflow models
You can use Tensorflow Lite with any high-level API, such as Keras or pytorch, to train and quantize models (for example, see [Training Mobilenet V2 for OpenMV with Keras](https://github.com/SingTown/openmv_tensorflow_training_scripts/blob/main/openmv_mobilenet_v2.ipynb)). However, the officially supported way to train models for OpenMV cameras is using EdgeImpulse. For more information on using EdgeImpulse with OpenMV cameras, please see the [EdgeImpulse tutorial for OpenMV](https://docs.edgeimpulse.com/docs/openmv-cam-h7-plus).

## Embedding Tensorflow models in the firmware
Tensorflow models can be embedded into the firmware for boards that don't have an SDRAM. The firmware ships with a default embedded model, that can be replaced by any other Tensorflow model. Additionally, more models can be embedded in the firmware, as many as the free flash size allows. Once you have a tflite model to replace the default model, or to add to the firmware, it can be embedded in the firmware by placing the model `.tflite` file and its `.txt` labels file in `src/libtf/models`. The model file and labels file must have the same basename, for example:

```bash
models/mask_detection.txt
models/mask_detection.tflite
```

The labels `.txt` file contains model output labels arranged 1 label per line, for example:
```
>>cat mask_detection.txt

This file has been truncated. show original

Topic		Replies	Views
Person Detection Model OpenMV Products	3	1327	April 7, 2023
Person detection model missing from example files OpenMV Products	7	561	February 10, 2023
Error while using tf.load on custom tflite model OpenMV Products	7	1968	May 5, 2022
Maximum size of tflite model on Cam H7 OpenMV Products	3	660	September 10, 2020
LSTM TFLite OpenMV OpenMV Products	3	166	January 19, 2024

Model TFLite embedded in OpenMV Cam firmware

Related Topics