TFLite model load error

I have created and tested a fully uint8 quantized TFLite model on PC.

When loading the model in OpenMV with tfmodel = tf.load(“Model_UInt8_Quant_V8.tflite”, load_to_fb=True), I’m getting this error:

OSError: tensorflow/lite/micro/kernels/quantize.cc:68 input->type == kTfLiteFloat32 || input->type == kTfLiteInt16 || input->type == kTfLiteInt8 was not true.
Node QUANTIZE (number 0f) failed to prepare with status 1
AllocateTensors() failed!

One of the layers in your model is not quantized.

If possible please use Edge Impulse.

Both input and output layers of my quantized tflite model are uint8

TFLite input details [{‘name’: ‘rescaling_6_input’, ‘index’: 21, ‘shape’: array([ 1, 64, 64, 1]), ‘shape_signature’: array([-1, 64, 64, 1]), ‘dtype’: <class ‘numpy.uint8’>, ‘quantization’: (1.0, 0), ‘quantization_parameters’: {‘scales’: array([1.], dtype=float32), ‘zero_points’: array([0]), ‘quantized_dimension’: 0}, ‘sparsity_parameters’: {}}]

TFLite output details [{‘name’: ‘Identity’, ‘index’: 22, ‘shape’: array([1, 2]), ‘shape_signature’: array([-1, 2]), ‘dtype’: <class ‘numpy.uint8’>, ‘quantization’: (0.025388231500983238, 124), ‘quantization_parameters’: {‘scales’: array([0.03], dtype=float32), ‘zero_points’: array([124]), ‘quantized_dimension’: 0}, ‘sparsity_parameters’: {}}]

The tf.load error message seems to be that it is not accepting input type uint8.
Error message “input->type == kTfLiteFloat32 || input->type == kTfLiteInt16 || input->type == kTfLiteInt8 was not true”

Please try int8.

CMSISNN is not optimized for uint8. The mode will run slower even if it were passing that check.

Please try int8.

CMSISNN is not optimized for uint8. The mode will run slower even if it were passing that check.

Which version of tflite is used in openmv firmware?

We’re using this branch/commit: GitHub - openmv/tensorflow at 6a303b679d1a7e9978a05129d20c3eb578509fdb

1 Like

I got the int8 tflite model working in python on PC. Was missing the scaling step.

The example for tflite has a scaling step before sending to the image to the tflite model: Post-training integer quantization  |  TensorFlow Lite

rescale input data to uint8

input_scale, input_zero_point = input_details[“quantization”]
test_image = test_image / input_scale + input_zero_point

Does tf.classify perform this scaling step, or does it need to be done to the image before sending to tf.classify?

We convert the model from RGB888 to whatever the input layer format is. I.e. uint8, int8, or float. We don’t do rescaling to maximize the bounds however.

Sorry, I should have mentioned I’m working with grayscale. Does tf.classify also convert grayscale images to the tflite int8 input format?

Yes, also, just read the code if you have more questions:

https://github.com/openmv/openmv/blob/master/src/omv/py/py_tf.c#L221

Great, Thanks.
I have confirmed by confusion matrix of grayscale images that an int8 quantized tflite model on PC with tensorflow interpreter AND rescaling, matches the tf.classify on OpenMV WITHOUT rescaling.