# Can't use Tensorflow-Lite for OpenMV

**URL:** <https://forums.openmv.io/t/cant-use-tensorflow-lite-for-openmv/1688>\
**Category:** OpenMV Boards\
**Created:** [March 17, 2020, 12:46pm UTC](https://forums.openmv.io/t/cant-use-tensorflow-lite-for-openmv/1688 "2020-03-17T12:46:11Z")\
**Posts on this page:** 20\
**Page:** 2

<div class="post-metadata">

**Author:** ![kwagyeman](https://yyz1.discourse-cdn.com/flex029/user_avatar/forums.openmv.io/kwagyeman/32/4_2.png) [@kwagyeman](https://forums.openmv.io/u/kwagyeman)\
**Post date:** [April 17, 2020, 3:15pm UTC](https://forums.openmv.io/t/cant-use-tensorflow-lite-for-openmv/1688/21 "2020-04-17T15:15:19Z")

</div>

Your code is completely wrong… Please read what the network outputs. I will post an update with changes to your code in a it.

---

<div class="post-metadata">

**Author:** ![imlearning](https://avatars.discourse-cdn.com/v4/letter/i/f6c823/32.png) [@imlearning](https://forums.openmv.io/u/imlearning)\
**Post date:** [April 20, 2020, 7:10am UTC](https://forums.openmv.io/t/cant-use-tensorflow-lite-for-openmv/1688/22 "2020-04-20T07:10:33Z")

</div>

Hi,  
Yes, I had developed that code based on the network output I was getting.

For e.g. printing just the “out” variable in above code gets the following output like:

> [{“x”:0, “y”:0, “w”:128, “h”:128, “output”:[0.5294118, 0.8745098, 0.627451, 0.5803922, 0.5254902, 0.682353, 0.5372549, 0.6352942, 0.5058824, 0.5294118]}]

**Code with outputs is attached below**

But sure, I’ll be happy to learn from your updated code.  
Thanks in advance.

Regards,  
IM

 ![Screenshot from 2020-04-20 12-39-32.png](https://canada1.discourse-cdn.com/flex029/uploads/openmv1/original/1X/a90f7321d17c141b01bb5f0d2a4479eb984d7fda.png)

---

<div class="post-metadata">

**Author:** ![kwagyeman](https://yyz1.discourse-cdn.com/flex029/user_avatar/forums.openmv.io/kwagyeman/32/4_2.png) [@kwagyeman](https://forums.openmv.io/u/kwagyeman)\
**Post date:** [April 20, 2020, 7:25am UTC](https://forums.openmv.io/t/cant-use-tensorflow-lite-for-openmv/1688/23 "2020-04-20T07:25:58Z")

</div>

Hi, another user brought an issue up and the problem is likely related to the fact that I don’t do uint8\_t-\>int8\_t input/output conversion. Right now the image is fed to the net as a uint8\_t image.

Can you tell me what type of image data you trained on and what you expect the output to be? I.e is the image input -128 to 127? Or 0 to 127? And is the output -128 to 127? or 0 to 127? I have to fix this in the C code for things to work.

As for your code being wrong:

- out = net.classify(img.copy().binary([( 0, 1)], invert=True))  
score = max(out[0][4]) \* 100  
max\_idx = out[0][4].index(max(out[0][4]))  
if (score \< 50):  
score\_str = “??:??%”  
else:  
score\_str = "%s:%d%% "%(labels[max\_idx], score)  
img.draw\_string(0, 0, score\_str)

I was just thrown off by this. I think this is from some other code older code. The tensorflow example doesn’t show how to print class labels this way.

---

<div class="post-metadata">

**Author:** ![imlearning](https://avatars.discourse-cdn.com/v4/letter/i/f6c823/32.png) [@imlearning](https://forums.openmv.io/u/imlearning)\
**Post date:** [April 20, 2020, 12:03pm UTC](https://forums.openmv.io/t/cant-use-tensorflow-lite-for-openmv/1688/24 "2020-04-20T12:03:23Z")

</div>

Hi,  
I trained on the MNIST images(uint8) , then quantized it with full integer quantization, making the inputs and outputs as int8 type ( screenshot attached).

 ![Screenshot from 2020-04-20 17-13-03.png](https://canada1.discourse-cdn.com/flex029/uploads/openmv1/original/1X/fe9cf87faa2ffa5e462199cf5792fc0881833200.png)

---

<div class="post-metadata">

**Author:** ![kwagyeman](https://yyz1.discourse-cdn.com/flex029/user_avatar/forums.openmv.io/kwagyeman/32/4_2.png) [@kwagyeman](https://forums.openmv.io/u/kwagyeman)\
**Post date:** [April 20, 2020, 3:46pm UTC](https://forums.openmv.io/t/cant-use-tensorflow-lite-for-openmv/1688/25 "2020-04-20T15:46:50Z")

</div>

Okay, I’ll make a version of the firmware that fixes the offset and post it tonight. This should fix the problem.

The firmware will check the network input type and then fix the data signedness automatically.

…

That said, it would be helpful to know the range if possible. I’m going to assume I just need to subtract 128 on the input and add 128 on the output. I hope that is correct.

---

<div class="post-metadata">

**Author:** ![imlearning](https://avatars.discourse-cdn.com/v4/letter/i/f6c823/32.png) [@imlearning](https://forums.openmv.io/u/imlearning)\
**Post date:** [April 21, 2020, 6:27am UTC](https://forums.openmv.io/t/cant-use-tensorflow-lite-for-openmv/1688/26 "2020-04-21T06:27:36Z")

</div>

Hi,  
The scikit-image website says the range of int8 is -128 to 127.  
However, I think the your approach should work. Similar thing is suggested in this comment too: [Quantization problem while reproducing person detection example · Issue #37347 · tensorflow/tensorflow · GitHub](https://github.com/tensorflow/tensorflow/issues/37347#issuecomment-602962764)  
 ![Screenshot from 2020-04-21 09-33-29.png](https://canada1.discourse-cdn.com/flex029/uploads/openmv1/original/1X/60b8c3cbb27dc27c9031a04392a41e1ec2778204.png)

---

<div class="post-metadata">

**Author:** ![kwagyeman](https://yyz1.discourse-cdn.com/flex029/user_avatar/forums.openmv.io/kwagyeman/32/4_2.png) [@kwagyeman](https://forums.openmv.io/u/kwagyeman)\
**Post date:** [April 22, 2020, 6:30pm UTC](https://forums.openmv.io/t/cant-use-tensorflow-lite-for-openmv/1688/27 "2020-04-22T18:30:58Z")

</div>

Hi, try this binary out. It will subtract 128 from the input unsigned image data if the data is signed and add 128 to the output if signed to make it unsigned again. The input and output are done independently. So, you are free to mix and match.

I’ve verified that our unsigned person detector network works.

…

Please let me know if this works and then I will send in the branch to be merged.  
[firmware.zip](https://forums.openmv.io/uploads/short-url/uN3peMm2QZogWRHixhFbFeq3OQW.zip) (1.16 MB)

---

<div class="post-metadata">

**Author:** ![imlearning](https://avatars.discourse-cdn.com/v4/letter/i/f6c823/32.png) [@imlearning](https://forums.openmv.io/u/imlearning)\
**Post date:** [April 23, 2020, 7:05pm UTC](https://forums.openmv.io/t/cant-use-tensorflow-lite-for-openmv/1688/28 "2020-04-23T19:05:37Z")

</div>

Hi,  
Thanks for providing the binary. I gave it a try but the same situation still continues. The outputs are still following the same pattern as I mentioned earlier.

> the output is generating same result (say any particular number for e.g. ‘five’) no matter what the feed is…  
> So basically, the output changes with change of model files but is constant for a particular model with a slight variation in scores here & there.

I am attaching the full integer quantized model with this.  
[mnistconv\_int8.zip](https://forums.openmv.io/uploads/short-url/qG1BzPkw3sbi335db0Yr9W016Go.zip) (62.1 KB)

---

<div class="post-metadata">

**Author:** ![kwagyeman](https://yyz1.discourse-cdn.com/flex029/user_avatar/forums.openmv.io/kwagyeman/32/4_2.png) [@kwagyeman](https://forums.openmv.io/u/kwagyeman)\
**Post date:** [April 23, 2020, 7:12pm UTC](https://forums.openmv.io/t/cant-use-tensorflow-lite-for-openmv/1688/29 "2020-04-23T19:12:25Z")

</div>

Okay, I need this from you. Can you provide me the final model output and the training data set that you used for it. If I can load these images onto the camera via the SD card then I can actually debug this and get it working.

I won’t be able to debug your model unless I know what type of input it needs.

---

<div class="post-metadata">

**Author:** ![imlearning](https://avatars.discourse-cdn.com/v4/letter/i/f6c823/32.png) [@imlearning](https://forums.openmv.io/u/imlearning)\
**Post date:** [April 24, 2020, 7:47am UTC](https://forums.openmv.io/t/cant-use-tensorflow-lite-for-openmv/1688/30 "2020-04-24T07:47:02Z")

</div>

Okay, Here’s is a Jupyter notebook which trains the model and performs full integer quantization.

Open it up with Google Colaboratory and run in the “Playground” mode after that.

Running the entire notebook (by clicking on “Runtime” → “Run all” option in Colab ) will get you the Final Model (“.tflite”) output file which you can easily download from the “Files” section on left side of Colab.

The dataset is the inbuilt MNIST dataset that Tensorflow/Keras provide.

Link: [OpenMV\_tflite\_demo.ipynb - Google Drive](https://drive.google.com/file/d/1eiZlf4YO_lFwPf2rmefDd7buVrZLfQNZ/view?usp=sharing)

---

<div class="post-metadata">

**Author:** ![imlearning](https://avatars.discourse-cdn.com/v4/letter/i/f6c823/32.png) [@imlearning](https://forums.openmv.io/u/imlearning)\
**Post date:** [April 29, 2020, 6:11am UTC](https://forums.openmv.io/t/cant-use-tensorflow-lite-for-openmv/1688/31 "2020-04-29T06:11:14Z")

</div>

Hey,

Just circling back to see if there are any updates on this?

Regards,  
IM

---

<div class="post-metadata">

**Author:** ![kwagyeman](https://yyz1.discourse-cdn.com/flex029/user_avatar/forums.openmv.io/kwagyeman/32/4_2.png) [@kwagyeman](https://forums.openmv.io/u/kwagyeman)\
**Post date:** [April 29, 2020, 3:35pm UTC](https://forums.openmv.io/t/cant-use-tensorflow-lite-for-openmv/1688/32 "2020-04-29T15:35:58Z")

</div>

I’m focused on the interface library right now. Since the simple fix didn’t work this has to become a project for me now.

Question, did you try binarizing the image (using actual thresholds - not just calling to\_bitmap()) and feed that to the net? Previously, on our old CNN system you had to binarize the image and then invert the image to get things to work.

I.e. the network needs the background as white and the characters black. So, this requires and inverted binarize operation.

---

<div class="post-metadata">

**Author:** ![imlearning](https://avatars.discourse-cdn.com/v4/letter/i/f6c823/32.png) [@imlearning](https://forums.openmv.io/u/imlearning)\
**Post date:** [May 7, 2020, 1:29pm UTC](https://forums.openmv.io/t/cant-use-tensorflow-lite-for-openmv/1688/33 "2020-05-07T13:29:14Z")

</div>

Ahh alright, then it might take a while I assume?

And yes I did try the image with White Background & Black characters (& vice versa).

I feel maybe it’s not just about the input images or the dataset trained on because as soon as the video begins the same result [a particular number(class)] starts getting displayed and the result never changes no matter what the input is.

I hope you all are safe  
Wishing you good health,  
IM

---

<div class="post-metadata">

**Author:** ![kwagyeman](https://yyz1.discourse-cdn.com/flex029/user_avatar/forums.openmv.io/kwagyeman/32/4_2.png) [@kwagyeman](https://forums.openmv.io/u/kwagyeman)\
**Post date:** [May 7, 2020, 2:19pm UTC](https://forums.openmv.io/t/cant-use-tensorflow-lite-for-openmv/1688/34 "2020-05-07T14:19:07Z")

</div>

Hi, I’ll be able to start working on this next week. I have to finish the latest firmware release documentation and get that out to everyone.

We have a goal of integration with Edge Impulse soon for automatic deep learning training so everything with TensorFlow needs to get working for the release coming next month after the one we are about to do.

---

<div class="post-metadata">

**Author:** ![kwagyeman](https://yyz1.discourse-cdn.com/flex029/user_avatar/forums.openmv.io/kwagyeman/32/4_2.png) [@kwagyeman](https://forums.openmv.io/u/kwagyeman)\
**Post date:** [May 15, 2020, 1:00am UTC](https://forums.openmv.io/t/cant-use-tensorflow-lite-for-openmv/1688/35 "2020-05-15T01:00:12Z")

</div>

Hi, I’ve updated the firmware to support int8/uint8/float32.

The model still needs to be int8/uint8 quantized internally, but, float32 is supported as the input/output layer.

It was tested on Mobilenet (uint8), the person detector (uint8), another person detector (int8), and then a flower model (float32 with int8 internally).

This file is for the H7 Plus. Let me make one for the H7.  
[firmware.zip](https://forums.openmv.io/uploads/short-url/oBWv6f0HsMIq2q2ghPsFFGiyWdJ.zip) (1.18 MB)

---

<div class="post-metadata">

**Author:** ![kwagyeman](https://yyz1.discourse-cdn.com/flex029/user_avatar/forums.openmv.io/kwagyeman/32/4_2.png) [@kwagyeman](https://forums.openmv.io/u/kwagyeman)\
**Post date:** [May 15, 2020, 1:12am UTC](https://forums.openmv.io/t/cant-use-tensorflow-lite-for-openmv/1688/36 "2020-05-15T01:12:10Z")

</div>

H7 Firmware  
[firmware.zip](https://forums.openmv.io/uploads/short-url/5lHz5Npqt9z8UJyzwl7GS3HuOel.zip) (1.17 MB)

---

<div class="post-metadata">

**Author:** ![kwagyeman](https://yyz1.discourse-cdn.com/flex029/user_avatar/forums.openmv.io/kwagyeman/32/4_2.png) [@kwagyeman](https://forums.openmv.io/u/kwagyeman)\
**Post date:** [May 15, 2020, 1:13am UTC](https://forums.openmv.io/t/cant-use-tensorflow-lite-for-openmv/1688/37 "2020-05-15T01:13:44Z")

</div>

Regarding your model. Please try a dataset where the input data is a series of color images and verify that your model works okay before trying mnist.

We are getting things ready for Edge Impulse support. So, we expect TensorFlow support to be working smoothly moving forwards.

---

<div class="post-metadata">

**Author:** ![shawn.lee](https://avatars.discourse-cdn.com/v4/letter/s/848f3c/32.png) [@shawn.lee](https://forums.openmv.io/u/shawn.lee)\
**Post date:** [May 19, 2020, 11:27am UTC](https://forums.openmv.io/t/cant-use-tensorflow-lite-for-openmv/1688/38 "2020-05-19T11:27:51Z")

</div>

Hi,

I updated the H7 firmware that kwagyeman released on Fri May 15, 2020.

I follow the steps to prepare the tensorflow lite model for mnist.

> **[Post-training integer quantization  |  TensorFlow Lite](https://www.tensorflow.org/lite/performance/post_training_integer_quant)**

partial of training code as below.  
I modify some codes for channel dimention

```python
mnist = keras.datasets.mnist
(train_images, train_labels), (test_images, test_labels) = mnist.load_data()

# Normalize the input image so that each pixel value is between 0 to 1.
train_images = train_images / 255.0
test_images = test_images / 255.0

train_images = np.expand_dims(train_images, axis=3)
test_images = np.expand_dims(test_images, axis=3)

# Define the model architecture
model = tf.keras.Sequential([
  tf.keras.layers.InputLayer(input_shape=(28, 28, 1)),
  tf.keras.layers.Conv2D(filters=12, kernel_size=(3, 3), activation=tf.nn.relu),
  tf.keras.layers.MaxPooling2D(pool_size=(2, 2)),
  tf.keras.layers.Flatten(),
  tf.keras.layers.Dense(10, activation=tf.nn.softmax)
])

.......................................

```

Everything is fine on my PC.  
I try these models on openmv h7.  
What could i set image range? 0~255 or 0~1

```python
import sensor, image, time, os, tf

sensor.reset() # Reset and initialize the sensor.
sensor.set_contrast(3)
sensor.set_pixformat(sensor.GRAYSCALE) # Set pixel format to RGB565 (or GRAYSCALE)
sensor.set_framesize(sensor.QVGA) # Set frame size to QVGA (320x240)
sensor.set_windowing((28, 28)) # Set 128x128 window.
sensor.skip_frames(time=100)
sensor.set_auto_gain(False)
sensor.set_auto_exposure(False)

#net = tf.load('/mnist_model_quant.tflite')
net = tf.load('/mnist_model_quant_io.tflite')
clock = time.clock()

while (True):
    clock.tick()
    img = sensor.snapshot()
    out = net.classify(img.copy().binary([(0, 1)], invert=True))#?????????
    print(out)

```

Thanks.

---

<div class="post-metadata">

**Author:** ![kwagyeman](https://yyz1.discourse-cdn.com/flex029/user_avatar/forums.openmv.io/kwagyeman/32/4_2.png) [@kwagyeman](https://forums.openmv.io/u/kwagyeman)\
**Post date:** [May 20, 2020, 5:40am UTC](https://forums.openmv.io/t/cant-use-tensorflow-lite-for-openmv/1688/39 "2020-05-20T05:40:05Z")

</div>

Should be 0-255. However, you need to match the color of the binary data with MNIST. So, if it’s black on white then 0 and 255. If it’s white on black then 255 and 0.

---

<div class="post-metadata">

**Author:** ![shawn.lee](https://avatars.discourse-cdn.com/v4/letter/s/848f3c/32.png) [@shawn.lee](https://forums.openmv.io/u/shawn.lee)\
**Post date:** [May 20, 2020, 11:36am UTC](https://forums.openmv.io/t/cant-use-tensorflow-lite-for-openmv/1688/40 "2020-05-20T11:36:11Z")

</div>

Thanks for your reply.

test code on my PC

```python
img = np.ones((28,28,1),dtype=np.float)
white_img = img*255
test_image = np.expand_dims(white_img, axis=0).astype(np.float32)

tflite_model_quant_file = tflite_models_dir/"mnist_model_quant_io.tflite"
interpreter_quant = tf.lite.Interpreter(model_path=str(tflite_model_quant_file))
interpreter_quant.allocate_tensors()
input_index_quant = interpreter_quant.get_input_details()[0]["index"]
output_index_quant = interpreter_quant.get_output_details()[0]["index"]
interpreter_quant.set_tensor(input_index_quant, test_image)
interpreter_quant.invoke()
predictions = interpreter_quant.get_tensor(output_index_quant)
print('output:,',predictions[0])

```

output:, [0. 0. 0.0625 0.8515625 0. 0.078125 0.  
0. 0. 0. ]

test code on H7

```python
import sensor, image, time, os, tf

sensor.reset() # Reset and initialize the sensor.
sensor.set_contrast(3)
sensor.set_pixformat(sensor.GRAYSCALE) # Set pixel format to RGB565 (or GRAYSCALE)
sensor.set_framesize(sensor.QVGA) # Set frame size to QVGA (320x240)
sensor.set_windowing((28, 28)) # Set 128x128 window.
sensor.skip_frames(time=100)
sensor.set_auto_gain(False)
sensor.set_auto_exposure(False)

net = tf.load('/mnist_model_quant_io.tflite')
clock = time.clock()

while (True):
    clock.tick()
    myImage = image.Image("white.ppm", copy_to_fb = True)
    graytmp = myImage.to_grayscale(True, rgb_channel=0)
    out = net.classify(graytmp)
    print(out)

```

output as below:  
[{“x”:0, “y”:0, “w”:28, “h”:28, “output”:[0.5019608, 0.5019608, 0.5369792, 0.9202359, 0.5019608, 0.544761, 0.5019608, 0.5019608, 0.5019608, 0.5019608]}]  
[{“x”:0, “y”:0, “w”:28, “h”:28, “output”:[0.5019608, 0.5019608, 0.5369792, 0.9202359, 0.5019608, 0.544761, 0.5019608, 0.5019608, 0.5019608, 0.5019608]}]  
[{“x”:0, “y”:0, “w”:28, “h”:28, “output”:[0.5019608, 0.5019608, 0.5369792, 0.9202359, 0.5019608, 0.544761, 0.5019608, 0.5019608, 0.5019608, 0.5019608]}]  
[{“x”:0, “y”:0, “w”:28, “h”:28, “output”:[0.5019608, 0.5019608, 0.5369792, 0.9202359, 0.5019608, 0.544761, 0.5019608, 0.5019608, 0.5019608, 0.5019608]}]  
[{“x”:0, “y”:0, “w”:28, “h”:28, “output”:[0.5019608, 0.5019608, 0.5369792, 0.9202359, 0.5019608, 0.544761, 0.5019608, 0.5019608, 0.5019608, 0.5019608]}]  
[{“x”:0, “y”:0, “w”:28, “h”:28, “output”:[0.5019608, 0.5019608, 0.5369792, 0.9202359, 0.5019608, 0.544761, 0.5019608, 0.5019608, 0.5019608, 0.5019608]}]  
[{“x”:0, “y”:0, “w”:28, “h”:28, “output”:[0.5019608, 0.5019608, 0.5369792, 0.9202359, 0.5019608, 0.544761, 0.5019608, 0.5019608, 0.5019608, 0.5019608]}]  
[{“x”:0, “y”:0, “w”:28, “h”:28, “output”:[0.5019608, 0.5019608, 0.5369792, 0.9202359, 0.5019608, 0.544761, 0.5019608, 0.5019608, 0.5019608, 0.5019608]}]

why they are so different? 0.5019608 means?  
The sum should be 1 or not?  
[white.7z](https://forums.openmv.io/uploads/short-url/e6I0k8tU1vUwbdujRjoG93cf8Uh.7z) (207 Bytes)

[Previous page](https://forums.openmv.io/t/cant-use-tensorflow-lite-for-openmv/1688.md?page=1)

[Next page](https://forums.openmv.io/t/cant-use-tensorflow-lite-for-openmv/1688.md?page=3)