TensorFlow Lite model inference result is wrong

iizukak · April 19, 2020, 6:18am

Hi, I’m developing Facial Expression Recognition deep learning model for OpenMV.

I trained TensorFlow model for this task.
I converted Keras model to TensorFlow Lite quantized model.
The model is int8 quantized(both input and output).

My quantized model work well on PC,
But model dose not work on OpenMV H7.
It’s can loaded but inference result is always wrongz(guess wrong expression).

TFLite model is here.

Conversion script is here.

github.com

iizukak/fer-openmv/blob/master/src/tflite_converter.py#L31-L56


      
          def convert_quant(model):
              """
              Convert Keras model to quantized tflite
          
          
    reference:
              https://www.tensorflow.org/lite/performance/post_training_quantization
              """
              (x_train, _), (_, _), (_, _) = dataset.load_dataset(config.DATASET_PATH)
              x_train = x_train.astype('float32')
          
          
    # Calibration for 1 epoch
              def representative_dataset_gen():
                for i in range(len(x_train)):
                  # Get sample input data as a numpy array in a method of your choosing.
                  # Format NHW
                  yield [x_train[i][tf.newaxis, ..., tf.newaxis]]
          
          
    
              model_path = "trained_models/" + config.NETWORK + ".h5"
              converter = tf.compat.v1.lite.TFLiteConverter.from_keras_model_file(model_path)

This file has been truncated. show original

OpenMV script is here.

github.com

iizukak/fer-openmv/blob/master/src/openmv.py

# TensorFlow Lite Person Dection Example
#
# Google's Person Detection Model detects if a person is in view.
#
# In this example we slide the detector window over the image and get a list
# of activations. Note that use a CNN with a sliding window is extremely compute
# expensive so for an exhaustive search do not expect the CNN to be real-time.

import sensor, image, time, os, tf

sensor.reset()                         # Reset and initialize the sensor.
sensor.set_pixformat(sensor.GRAYSCALE) # Set pixel format to RGB565 (or GRAYSCALE)
sensor.set_framesize(sensor.QVGA)      # Set frame size to QVGA (320x240)
sensor.set_windowing((240, 240))       # Set 240x240 window.
sensor.skip_frames(time=2000)          # Let the camera adjust.

# Load the built-in person detection network (the network is in your OpenMV Cam's firmware).
net = tf.load('./mobilenet_small_quant.tflite')
labels = ["Angry", "Disgust", "Fear", "Happy", "Sad", "Surprise", "Neutral"]

This file has been truncated. show original

On OpenMV, using the firmware on this thread.

TFLite model input is int8 but Pixel Value is 255.
It’s no problem?

Do you have idea to fix inference result?

Thank you.

kwagyeman · April 19, 2020, 6:29am

Hi, if you trained for int and you use grayscale we’d just give you 0 to 255. This probably breaks your model. Our code just converts the source image into the target format for the CNN. It does not convert unsigned data into signed data.

What was the range of input you fed your model? 0 to 127? -128 to 127? What’s the mapping from 0 to 255 to what your model takes?

I can probably add something to make this automatic. But, information on the mapping would help.

…

Also, our output code expect uint8 too. Please look at the code here and let me know what needs more flexibility. I only just removed the check in our lib to allow int8 images in. I did not do anything to fix the signedness.

https://github.com/openmv/openmv/blob/master/src/omv/py/py_tf.c#L213 - Input

https://github.com/openmv/openmv/blob/master/src/omv/py/py_tf.c#L322 - Output

iizukak · April 19, 2020, 2:43pm

Thanks kwagyeman,

I tried int8(0-127) input and uint8 output.
Then I got this error message when running.

OSError: tensorflow/lite/micro/kernels/quantize.cc:47 input->type == kTfLiteFloat32 || input->type == kTfLiteInt16 was not true.

I have just one quantize node, after softmax.
This model is fully 8bit quantized and error says input should be float or int16…?
I don’t know what this mean crealy.

Anyway, I will try some experiment for some conditions.

Thank you.

kwagyeman · April 19, 2020, 5:03pm

That’s an error in the TensorFlow library. I cannot solve that.

Anyway, for your previous network. What was the range of the input data and expected output data?

I can just subtract 128 from all pixels going in and subtract 128 from the output and this might mix everything.

Please tell me the range of input data and output data on the desktop.

Topic		Replies	Views
H7 Plus for Quantization TFLite Model ？ OpenMV Products	1	761	October 18, 2020
Can't use Tensorflow-Lite for OpenMV OpenMV Products	50	10503	June 9, 2020
Training and Running TF Lite Models OpenMV Products	1	239	November 1, 2022
[ASK for help] issues with tf moudle on OpenMV H7 plus OpenMV Products	14	1513	July 24, 2021
Gender Recognition with OpenMV H7 Project Discussion	2	617	September 11, 2020

TensorFlow Lite model inference result is wrong

Related Topics