I have created and tested a fully uint8 quantized TFLite model on PC.
When loading the model in OpenMV with tfmodel = tf.load(“Model_UInt8_Quant_V8.tflite”, load_to_fb=True), I’m getting this error:
OSError: tensorflow/lite/micro/kernels/quantize.cc:68 input->type == kTfLiteFloat32 || input->type == kTfLiteInt16 || input->type == kTfLiteInt8 was not true.
Node QUANTIZE (number 0f) failed to prepare with status 1
AllocateTensors() failed!
One of the layers in your model is not quantized.
If possible please use Edge Impulse.
ajacobs
November 5, 2020, 12:50am
3
Both input and output layers of my quantized tflite model are uint8
TFLite input details [{‘name’: ‘rescaling_6_input’, ‘index’: 21, ‘shape’: array([ 1, 64, 64, 1]), ‘shape_signature’: array([-1, 64, 64, 1]), ‘dtype’: <class ‘numpy.uint8’>, ‘quantization’: (1.0, 0), ‘quantization_parameters’: {‘scales’: array([1.], dtype=float32), ‘zero_points’: array([0]), ‘quantized_dimension’: 0}, ‘sparsity_parameters’: {}}]
TFLite output details [{‘name’: ‘Identity’, ‘index’: 22, ‘shape’: array([1, 2]), ‘shape_signature’: array([-1, 2]), ‘dtype’: <class ‘numpy.uint8’>, ‘quantization’: (0.025388231500983238, 124), ‘quantization_parameters’: {‘scales’: array([0.03], dtype=float32), ‘zero_points’: array([124]), ‘quantized_dimension’: 0}, ‘sparsity_parameters’: {}}]
The tf.load error message seems to be that it is not accepting input type uint8.
Error message “input->type == kTfLiteFloat32 || input->type == kTfLiteInt16 || input->type == kTfLiteInt8 was not true”
Please try int8.
CMSISNN is not optimized for uint8. The mode will run slower even if it were passing that check.
Please try int8.
CMSISNN is not optimized for uint8. The mode will run slower even if it were passing that check.
Which version of tflite is used in openmv firmware?
I got the int8 tflite model working in python on PC. Was missing the scaling step.
The example for tflite has a scaling step before sending to the image to the tflite model: Post-training integer quantization | TensorFlow Lite
rescale input data to uint8
input_scale, input_zero_point = input_details[“quantization”]
test_image = test_image / input_scale + input_zero_point
Does tf.classify perform this scaling step, or does it need to be done to the image before sending to tf.classify?
We convert the model from RGB888 to whatever the input layer format is. I.e. uint8, int8, or float. We don’t do rescaling to maximize the bounds however.
ajacobs
November 5, 2020, 11:40pm
10
Sorry, I should have mentioned I’m working with grayscale. Does tf.classify also convert grayscale images to the tflite int8 input format?
ajacobs
November 6, 2020, 4:35pm
12
Great, Thanks.
I have confirmed by confusion matrix of grayscale images that an int8 quantized tflite model on PC with tensorflow interpreter AND rescaling, matches the tf.classify on OpenMV WITHOUT rescaling.