Hi, I am using OpenMV firmware 4.5.9 and OpenMV IDE 4.2.0 to deploy a detection model. The quantized INT8 model has an output of type float32, which is different from the validation in Python scripts. I’ve checked my quant model detect_1000_int8.tflite, and the quantization seems successful.
For example, an identical image has different outputs in Python scripts validation and in OpenMV micropython scipts:
logits: array([-16.358, 15.7667], dtype=float32) # OpenMV logits: [[-67 57]] # Python
And I check the TFLite model tensor details, reassure that all operators are quantized to INT8 or INT16. I wonder why the output of quant model on OpenMV is float32, and why the results are so different from Tensorflow validation python scripts.
The followings are the scripts that I used:
# Validation on PC
interpreter = tf.lite.Interpreter(model_path='/home/user/detect_1000_hw100_int8.tflite')
interpreter.allocate_tensors()
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
print("Checking TFLite model tensor details...")
for detail in interpreter.get_tensor_details():
print(f"Tensor Name: {detail['name']}, Type: {detail['dtype']}")
test_dataset = []
for i in range(num_samples):
img = torch.tensor(extracted_images[i], dtype=torch.float32)
img = fivecrop_scale(img, crop_size=450) # scale
img_array = img.numpy()
scale, zero_point = input_details[0]['quantization']
img_int8 = np.round(img_array / scale + zero_point).astype(np.int8) # quant input to int8
img_int8 = np.expand_dims(img_int8, axis=0)
test_dataset.append(img_int8)
def run_inference_tflite(image_np):
interpreter.set_tensor(input_details[0]['index'], image_np)
interpreter.invoke()
output_data = interpreter.get_tensor(output_details[0]['index'])
return output_data
all_preds = []
i = 1
for img in test_dataset:
pred = run_inference_tflite(img)
print('index',i, 'Pred', math.ceil(1 / (1 + np.exp(-pred[0][1]))))
i = i + 1
all_preds.append(pred)
# Validation on OpenMV
model = ml.Model("detect_1000_hw100_int8.tflite", load_to_fb=True)
#print('model loaded')
predicted_class = []
num_iterations = 10
for _ in range(num_iterations):
if _ < 7:
print('collecting ECGdata')
samples = ECG_dataset.__getitem__()
for sample in samples:
input_list = []
input_list.append(samples[0])
logits = model.predict(input_list)
predicted_class.append(probability(logits))