Hello, digging up this topic to get an understanding of how these TF module functions work. It is awesome to read that the TF module handles a lot for us (but confusingly not documented in tf.classify help). Lately I have been using an EdgeImpulse-generated module to detect a black sponge and a green USB reader on a whiteboard. I classify images of variable resolutions and aspect ratios.
While under-scaled (relative to the model’s 160 px resolution) input images significantly reduced model performance (as expected) and over-sampled images had no detectable effect (as expected from the above), surprisingly, non-rescaled images (543 * 502 - close to square proportions) have terrible (null) performance (opposed to the assumption that the TF module handles everything), even though they are close to the resolution of the images that I re-scaled to 480*480, for instance.
Results above are based on samples of ~20 live image classifications of the target. The model usually has 100% accuracy for classifying the green USB reader. I used nearest neighbor interpolation. This is a sample image of what I was trying to classify (a green USB flash reader on a whiteboard):
I attach my script here, based on the one exported by Edgeimpulse:
# Edge Impulse - OpenMV Image Classification Example
import sensor, image, time, os, tf, uos, gc
sensor.reset() # Reset and initialize the sensor.
sensor.set_pixformat(sensor.RGB565) # Set pixel format to RGB565 (or GRAYSCALE)
sensor.set_framesize(sensor.WQXGA2) # Set frame size to QVGA (320x240)
#sensor.set_windowing((240, 240)) # Set 240x240 window.
sensor.skip_frames(time=2000) # Let the camera adjust.
net = None
labels = None
side_res = "original"
# load the model, alloc the model file on the heap if we have at least 64K free after loading
net = tf.load("trained.tflite", load_to_fb=uos.stat('trained.tflite') > (gc.mem_free() - (64*1024)))
except Exception as e:
raise Exception('Failed to load "trained.tflite", did you copy the .tflite and labels.txt file onto the mass-storage device? (' + str(e) + ')')
labels = [line.rstrip('\n') for line in open("labels.txt")]
except Exception as e:
raise Exception('Failed to load "labels.txt", did you copy the .tflite and labels.txt file onto the mass-storage device? (' + str(e) + ')')
clock = time.clock()
img = sensor.snapshot()
# default settings just do one detection... change them to search the image...
if (str(side_res) != "original"):
# MANUALLY comment/uncomment the following line for using original vs. re-scaled images
#for obj in net.classify(img,roi=(1340,1215,543,502), min_scale=1.0, scale_mul=0.8, x_overlap=0.5, y_overlap=0.5):
for obj in net.classify(img, min_scale=1.0, scale_mul=0.8, x_overlap=0.5, y_overlap=0.5):
print("**********\nPredictions at [x=%d,y=%d,w=%d,h=%d]" % obj.rect())
# This combines the labels and confidence values into a list of tuples
predictions_list = list(zip(labels, obj.output()))
with open('confidences.csv', 'a') as confidencelog:
confidencelog.write(str(predictions_list) + ',' + str(predictions_list) + ',' + str(side_res) + '\n')
for i in range(len(predictions_list)):
print("%s = %f" % (predictions_list[i], predictions_list[i]))