I am working on the FLIR® Lepton® Camera Sensor to measure the temperature in a manufacture process. I set the measurement mode :
sensor.ioctl(sensor.IOCTL_LEPTON_SET_MEASUREMENT_MODE, True, True)
to work in high temperature mode to get the real temperature data. I set the measure range from 160-240 degree in the range of the high temperature mode.
However, the output image comes with a lot of noise. I think part of the noise comes from the saturation or vacuum in measure, but there are some noise in the part with temperature as well.
The picture is two curves with different temperatures, which you can see in the another thermal sensor, but in the images from lepton, there are quite a lot of noise spikes in the pictures.
I think it could be the problem the auto gain mode is turn off for real temperature measure, and the fixed gain of the thermal camera is too large and brings too much noise when measuring. I test with lower temperature range from 20 to 35, and there aren’t that much noise.
Are there any ways to solve the noise problem? like setting a proper fixed gain of the thermal camera?
Hi, you’d need to look into the FLIR Lepton API. We allow you to execute API commands from Python by serializing byte structs that you can use to create commands.
Alternatively, do deal with the noise, you may wish to use some of the machine vision features onboard like the median() filter.
From the image you posted, I guess the first one is the output from the system. Why is it black and white, though? The image kinda looks like it’s clipping or something. Can you post the code snippet that shows the error?
Hi, thank you so much for your reply, and the first one is the output of the OpenMV system, here is the code snippet:
import sensor
import time
# Set the target temp range here
min_temp_in_celsius = 160.0
max_temp_in_celsius = 240.0
print("Resetting Lepton...")
# These settings are applied on reset
sensor.reset()
sensor.ioctl(sensor.IOCTL_LEPTON_SET_MEASUREMENT_MODE, True, True)
sensor.ioctl(
sensor.IOCTL_LEPTON_SET_MEASUREMENT_RANGE, min_temp_in_celsius, max_temp_in_celsius
)
print(
"Lepton Res (%dx%d)"
% (
sensor.ioctl(sensor.IOCTL_LEPTON_GET_WIDTH),
sensor.ioctl(sensor.IOCTL_LEPTON_GET_HEIGHT),
)
)
print(
"Radiometry Available: "
+ ("Yes" if sensor.ioctl(sensor.IOCTL_LEPTON_GET_RADIOMETRY) else "No")
)
print("temperature range:", sensor.ioctl(sensor.IOCTL_LEPTON_GET_MEASUREMENT_RANGE))
print("Measure mode:", sensor.ioctl(sensor.IOCTL_LEPTON_GET_MEASUREMENT_MODE))
sensor.set_pixformat(sensor.GRAYSCALE)
sensor.set_framesize(sensor.QQVGA)
#sensor.set_vflip(True)
sensor.skip_frames(time=5000)
clock = time.clock()
while True:
clock.tick() # Update the FPS clock.
img = sensor.snapshot() # Take a picture and return the image.
print(clock.fps()) # Note: OpenMV Cam runs about half as fast when connected
# to the IDE. The FPS should increase once disconnected.
The frame size is QQVGA, the same as the resolution of the FILR’s highest resolution (160x120). I am wondering what caused the black and white in the image as well.
The upper and lower black white may be caused by vacuum for perceive, the remote background doesn’t have any right measure value, and it shows in black and white, but it doesn’t make sense that there are black and white noise in the area with temperature. I am not sure whether it is caused by the sensor defect or wrong setup of the sensor.
I tried to take a thermal picture in the range of 25-40 degree as well, and the image(the first one below) in the low temperature mode ( with set of ‘‘sensor.ioctl(sensor.IOCTL_LEPTON_SET_MEASUREMENT_MODE, True)’’ ) looks better than the one in the high temperature mode (the second one below)
low temp mode.bmp (56.3 KB)
high temp mode.bmp (56.3 KB)
The image in high temperature mode shows a lot of black and white in the area of the clothes, maybe it is caused by the gain problem.
Thank you for your help in advance!
Hi, just to be clear. When you set the min temp of 160 it can’t see anything below 160 in measurement mode.
When I run your code in low-temp mode I get a black image… which is expected.
When I turn on high temp mode I get a lot of noise pixels:
I can also make out an outline of myself.
…
Default settings are -10C to 140C. So, you need to turn on high temp mode to access -10C to 450C.
I’m not sure why pixels are appearing white in the image. It should be all black. There’s a bug here that will need to be debugged. The high-temp mode option was user-contributed. This was not supported initial by me. It appears it only works in AGC mode. I’ll have to debug why it doesn’t work in linear measurement mode.
Thank you so much for your help. The white noise doesn’t make much sense and also add noise in the measure. Although there are some usable measurement, it would be better to debug the noise.
I really appreciate your help and debug, and look forward to the debug results!
Hi, sorry for the delay on this. I’ll be debugging this tomorrow.
No worries, thanks a lot for debugging, look forwards to good news! Good luck!
I was just running your script and the example script on an H7 Plus with the FLIR Lepton 3.5 and I don’t notice any issues anymore:
This code works for either low-gain mode or high-gain mode. I used a lighter to test the system seeing a flame. There was no noise anymore.
import sensor
import time
# Color Tracking Thresholds (Grayscale Min, Grayscale Max)
threshold_list = [(100, 255)] # track very hot objects
# Set the target temp range here
# 500C is the maximum the Lepton 3.5 sensor can measure
# At room temperature it's max is ~380C
min_temp_in_celsius = 0.0
max_temp_in_celsius = 400.0
print("Resetting Lepton...")
# These settings are applied on reset
sensor.reset()
# Enable measurement mode with high temp
sensor.ioctl(sensor.IOCTL_LEPTON_SET_MODE, True, True)
sensor.ioctl(
sensor.IOCTL_LEPTON_SET_RANGE, min_temp_in_celsius, max_temp_in_celsius
)
print(
"Lepton Res (%dx%d)"
% (
sensor.ioctl(sensor.IOCTL_LEPTON_GET_WIDTH),
sensor.ioctl(sensor.IOCTL_LEPTON_GET_HEIGHT),
)
)
print(
"Radiometry Available: "
+ ("Yes" if sensor.ioctl(sensor.IOCTL_LEPTON_GET_RADIOMETRY) else "No")
)
sensor.set_pixformat(sensor.GRAYSCALE)
sensor.set_framesize(sensor.QQVGA)
sensor.skip_frames(time=5000)
clock = time.clock()
# Only blobs that with more pixels than "pixel_threshold" and more area than "area_threshold" are
# returned by "find_blobs" below. Change "pixels_threshold" and "area_threshold" if you change the
# camera resolution. "merge=True" merges all overlapping blobs in the image.
def map_g_to_temp(g):
return (
(g * (max_temp_in_celsius - min_temp_in_celsius)) / 255.0
) + min_temp_in_celsius
while True:
clock.tick()
img = sensor.snapshot()
for blob in img.find_blobs(
threshold_list, pixels_threshold=200, area_threshold=200, merge=True
):
stats = img.get_statistics(thresholds=threshold_list, roi=blob.rect())
img.draw_rectangle(blob.rect())
img.draw_cross(blob.cx(), blob.cy())
img.draw_string(
blob.x(),
blob.y() - 10,
"%.2f C" % map_g_to_temp(stats.mean()),
mono_space=False,
)
print(
"FPS %f - Lepton Temp: %f C"
% (clock.fps(), sensor.ioctl(sensor.IOCTL_LEPTON_GET_FPA_TEMP))
)
Not sure what has changed. Note, I’m using the latest firmware release on an H7 Plus. There was some API refactoring that made the IOCTL names shorter. This will hit in v4.6.0.
It appears the issue is the RT1062. I get a perfect image on the H7 Plus. When I use the RT1062 in measurement mode I get noise in the image.
This is very odd, AGC mode works fine on the RT1062. So, the voltage regulators and etc. must be okay. Otherwise that would be highly broken.
Must be an error with the master clock to the module or SPI bus data. The SPI bus has control packets embedded into it though, if there were major issues with it then you wouldn’t get video in AGC mode.
I’ll compare the two boards.
It looks like there’s something funny going on with bit15 received by the SPI bus driver… if I mask that out then everything more or less is fixed. Images look sane. However, since the FLIR lepton 3.5 has radiometry that bit is actually needed as values will climb such that you need all 16-bits.
Bit 15 is also the only bit not used in the VOSPI framing packets, so it being off doesn’t break the video stream… this is why AGC mode works. You only notice it being off in non-AGC mode.
Figured out the issue. It’s just wrong SPI settings that appeared to be working. It will take a bit to short this out in a PR.
firmware.zip (1.7 MB)
Attached is a RT1062 firmware that works. Note that you need to use the shorter IOCTL names given the refactoring.
Thanks a lot for your efforts, and it is great that the bug is fixed, and I really appreciate it.
Could you share some guidance or tips to use the firmware you share?
and I am not that sure about what the shorter IOCTL names means, could you explain it a bit more? Thank you so much!
Tools->Run Bootloader. Select the firmware.bin file to load it.
The shorter IOCTL names mean they were renamed. When you try to run previous code you will get a failure. Use the new shorter names in the script I posted.
