I’m currently using the find_blobs function with merge=True to find hot spots in a thermal image but in some circumstances it returns a boundary box within another boundary box inside. The temperature value displayed doesn’t match with the temperature value I send back to the raspberry pi to be stored in a database. Initially I thought setting merge=True would solve this but having looked at the documentation, is it the margin parameter that I need to optimize. More specifically should I be setting the margin parameter to a negative value to force it to merge boundary boxes that appear like this. The image I have attached shows the problem I’m having. If you look at the top and bottom lines of the boundary box you will notice they are quick thick. This is not some functionality I have implemented and I have checked this against other images I have taken where this does not happen. The boundary boxes are either slightly within one another or exactly on top of one another. The fact that multiple temperature strings are not shown is also puzzling.
I cannot debug this via the IDE as I am using the FLIR Lepton 3.5 on a raspberry pi 3B which I communicate with via the USB_VCP class. Could you point me to the source code for the find_blobs function if you are unsure what my issue might be caused by?
Just make the merge margin greater than 1. This will merge the blobs then. It’s quite weird for that to happen however since this would require a region of pixels in-between the blobs to fail being thresholded.
If you pass the thresholds to binary() you can see what the thresholded image looks like.
Thanks for the reply, Yes I was a bit unsure how it happened myself although I thought some implementation of the margin parameter would fix it. The image below would probably have been a better illustration of the issue I was having.
I am currently working on the device so I cannot attach an image yet as I will need to put it back in the server room to gather images. The code that generated that image are as follows:
import sensor, image, time, math, ustruct, pyb
from pyb import USB_VCP
# Color Tracking Thresholds (Grayscale Min, Grayscale Max)
min_rgb = 1
max_rgb = 255
threshold_list = [(min_rgb, max_rgb)]# only bright grey colours will get tracked.
# Set the target temp range here
min_temp_in_celsius = 15.0
max_temp_in_celsius = 95.0
pixel_thresh=200
area_thresh=200
# These settings are applied on reset
sensor.reset()
sensor.ioctl(sensor.IOCTL_LEPTON_SET_MEASUREMENT_MODE, True) #This command forces the lepton to return each pixel value as a temperature
sensor.ioctl(sensor.IOCTL_LEPTON_SET_MEASUREMENT_RANGE, min_temp_in_celsius, max_temp_in_celsius) #sets the temperature range in celcius for mapping
#image will then be clamped between these values and scaled to between 0 and 255
#print("Lepton Res (%dx%d)" % (sensor.ioctl(sensor.IOCTL_LEPTON_GET_WIDTH),sensor.ioctl(sensor.IOCTL_LEPTON_GET_HEIGHT)))
#print("Radiometry Available: " + ("Yes" if sensor.ioctl(sensor.IOCTL_LEPTON_GET_RADIOMETRY) else "No"))
sensor.set_pixformat(sensor.GRAYSCALE) #sets the number of bits per pixel, greyscale is 8
sensor.set_framesize(sensor.QQVGA) #sets the image frame size QQVGA is 160x120
sensor.skip_frames(time=5000) #camera is given 5000ms to stabilize
clock = time.clock()
usb = USB_VCP() #Create a USB virtual communication port object.
# Only blobs with more pixels than "pixel_threshold" and more area than "area_threshold" are
# returned by "find_blobs" below. Change "pixels_threshold" and "area_threshold" if you change the
# camera resolution. "merge=True" merges all overlapping blobs in the image.
def map_g_to_temp(g):
#this function takes a scaled pixel value and converts it back to a temperature value
return ((g * (max_temp_in_celsius - min_temp_in_celsius)) / 255.0) + min_temp_in_celsius
def get_temperatures(img):
temperatures = []
for blob in img.find_blobs(threshold_list, pixel_threshold=pixel_thresh, area_threshold=area_thresh, margin=5, merge=True):
stats = img.get_statistics(thresholds=threshold_list, roi=blob.rect()) #get statistics for each blob where the blob is the roi.
temperatures.append(map_g_to_temp(stats.mean()))
return temperatures
def add_details(img):
for blob in img.find_blobs(threshold_list, pixel_threshold=pixel_thresh, area_threshold=area_thresh, margin=5, merge=True):
img.draw_rectangle(blob.rect()) #draws a rectangle (x, y, w, h) around blob for bounding box
stats = img.get_statistics(thresholds=threshold_list, roi=blob.rect()) #get statistics for each blob where the blob is the roi.
img.draw_string(blob.x(), blob.y() - 10, "%.2f C" % map_g_to_temp(stats.mean()), mono_space=False)
return img
def send_temp(temperature_list):
cmd = usb.recv(4, timeout=5000)
if cmd == b'temp':
if len(temperature_list) >=1:
time.sleep(1)
usb.send(str(max(temperature_list))) #need to send string to avoid object buffer protocol error
else:
usb.send('-100') #if no hotspot is detected return -100 as the temperature
def campture_img():
img = sensor.snapshot() #take an image
temperature_list = get_temperatures(img)
img = add_details(img)
#img.to_rainbow(color_palette=sensor.PALETTE_IRONBOW) #show hot areas as colours
img = img.compress(quality=100) #compress the image keeping as much detail as possible
usb.send(ustruct.pack('<L', img.size()))
usb.send(img)
pyb.LED(3).off()
send_temp(temperature_list)
def configure(configuration):
global min_rgb
global max_rgb
global threshold_list
global min_temp_in_celsius
global max_temp_in_celsius
global pixel_thresh
global area_thresh
configs = configuration.split(':')[1]
values = configs.split(',')
min_rgb = int(values[0])
max_rgb = int(values[1])
threshold_list = [(min_rgb, max_rgb)]
pixel_thresh = int(values[2])
area_thresh = int(values[3])
min_temp_in_celsius = int(values[4])
max_temp_in_celsius = int(values[5])
sensor.ioctl(sensor.IOCTL_LEPTON_SET_MEASUREMENT_RANGE, min_temp_in_celsius, max_temp_in_celsius)
sensor.skip_frames(time=5000)
usb.send(b'Configured')
pyb.LED(2).off()
while(True):
clock.tick()
data = usb.read() #Read all available bytes
if data != None: #check that data is not a None object if the read function returned nothing
#data = data.decode('utf-8')
if len(data) >= 4: # NEED TO HANDLE AT Commands
if data[:4] == b'snap':
pyb.LED(3).on()
campture_img()
if data[:4] == b'conf':
data = data.decode('utf-8')
pyb.LED(2).on()
configure(data)
else:
continue
else:
continue
else:
continue
I have since removed the get_temperatures function and incorporated it into the add_details function. Also is there a way to force synchronisation when communicating over usb_vcp? I was having another issue where on occasion the maximum temperature returned by the get_temperatures function did not match the maximum temperature denoted on any of the bounding boxes. It was like it had gone out of sync but was not generating an error make it quite difficult to debug.
No you’ll have to implement that somehow. When you attach the image I’ll run the same blob detection and drawing code and see if this issue can be reproduced.
