How can I access LEPTON API using OpenMV IDE?

Hello,

I got my new kit of OpenMV with Lepton 3.5 and accessories. I ran the example codes and it works fine. I needed to calibrate the thermal camera to obtain accurate temperature data i.e change it’s measurement parameters; how could I access the Lepton API through OpenMV IDE.

Thanks,
~Jubin

Hi, you have to manually create byte structs using ustruct with the pack operation and then send the structs via the flir ioctls:

# Lepton Get Object Temp Example
#
# This example shows off how to get an object's temperature using color tracking.
 
# By turning the AGC off and setting a max and min temperature range you can make the lepton into
# a great sensor for seeing objects of a particular temperature. That said, the FLIR lepton is a
# microblobometer and not a thermophile. So, it needs to re-calibrate itself often (which is called
# flat-field-correction - FFC). Additionally, microblobmeter devices require pprocessing support
# onboard to deal with the effects of temperature drift which is called radiometry support.
 
# FLIR Lepton Shutter Note: FLIR Leptons with radiometry and a shutter will pause the video often
# as they heatup to re-calibrate. This will happen less and less often as the sensor temperature
# stablizes. You can force the re-calibration to not happen if you need to via the lepton API.
# However, it is not recommended because the image will degrade overtime.
 
# If you are using a LEPTON other than the Lepton 3.5 this script may not work perfectly as other
# leptons don't have radiometry support or they don't activate their calibration process often
# enough to deal with temperature changes (FLIR 2.5).
 
import sensor, image, time, math
 
# Color Tracking Thresholds (Grayscale Min, Grayscale Max)
threshold_list = [(200, 255)]
 
# Set the target temp range here
min_temp_in_celsius = 20.0
max_temp_in_celsius = 35.0
 
print("Resetting Lepton...")
# These settings are applied on reset
sensor.reset()
sensor.ioctl(sensor.IOCTL_LEPTON_SET_MEASUREMENT_MODE, True)
sensor.ioctl(sensor.IOCTL_LEPTON_SET_MEASUREMENT_RANGE, min_temp_in_celsius, max_temp_in_celsius)
print("Lepton Res (%dx%d)" % (sensor.ioctl(sensor.IOCTL_LEPTON_GET_WIDTH),
                              sensor.ioctl(sensor.IOCTL_LEPTON_GET_HEIGHT)))
print("Radiometry Available: " + ("Yes" if sensor.ioctl(sensor.IOCTL_LEPTON_GET_RADIOMETRY) else "No"))
 
sensor.set_pixformat(sensor.GRAYSCALE)
sensor.set_framesize(sensor.QQVGA)
sensor.skip_frames(time=5000)
clock = time.clock()
 
# https://www.flir.com/globalassets/imported-assets/document/flir-lepton-software-interface-description-document.pdf
 
import ustruct
 
def start_ffc():
    sensor.ioctl(sensor.IOCTL_LEPTON_RUN_COMMAND, 0x0242)
 
def get_ffc_running():
    state = ustruct.unpack("<I", sensor.ioctl(sensor.IOCTL_LEPTON_GET_ATTRIBUTE, 0x0244, 2))[0]
    # LEP_SYS_STATUS_WRITE_ERROR == -2
    # LEP_SYS_STATUS_ERROR == -1
    # LEP_SYS_STATUS_READY == 0
    # LEP_SYS_STATUS_BUSY == 1
    # LEP_SYS_FRAME_AVERAGE_COLLECTING_FRAMES == 2
    return state
    
# Only blobs that with more pixels than "pixel_threshold" and more area than "area_threshold" are
# returned by "find_blobs" below. Change "pixels_threshold" and "area_threshold" if you change the
# camera resolution. "merge=True" merges all overlapping blobs in the image.
 
def map_g_to_temp(g):
    return ((g * (max_temp_in_celsius - min_temp_in_celsius)) / 255.0) + min_temp_in_celsius
 
i = 0
 
while(True):
    clock.tick()
    img = sensor.snapshot()
    for blob in img.find_blobs(threshold_list, pixels_threshold=200, area_threshold=200, merge=True):
        img.draw_rectangle(blob.rect())
        img.draw_cross(blob.cx(), blob.cy())
        stats = img.get_statistics(thresholds=threshold_list, roi=blob.rect())
        img.draw_string(blob.x(), blob.y() - 10, "%.2f C" % map_g_to_temp(stats.mean()), mono_space=False)
    print("FPS %f - Lepton Temp: %f C - FFC %d" % (clock.fps(), sensor.ioctl(sensor.IOCTL_LEPTON_GET_FPA_TEMPERATURE), get_ffc_running()))
    
    i+=1
    if i % 100 == 0:
        start_ffc()

There were way too many flir methods to wrap in python api calls. So, this allows you to create a byte struct, and send it with the right command to the lepton. It’s quite low level to program this way but it works. You will need to reference the flir lepton api when writing the code.

Thank you for the prompt reply. I will look into the documentations and try accessing the LEPTON API as you suggested.

I am trying to add a spot feature to the lepton camera interface! This would enable us to read the spot temperature. As of now with the default settings the camera is not reading accurate temperature values, i need to calibrate it.

~Jubin

Hi,

I am sorry to pose the same questions again. I am new to programming and is having a hard time to figure out the example you showed. Is it possible for you to show one more similar method to access LEPTON SDK in OpenMV IDE. I am trying to use RAD T-Linear Enable State (4.8.9 Pg 149) from the Lepton IDD file. How do I write the code for this or any other examples? I am trying to understand the syntax being followed.

Thank you
~Jubin

First… I recommend looking at the API code for for the lepton too. It helps:

https://github.com/openmv/openmv/tree/master/src/lepton/src
https://github.com/openmv/openmv/tree/master/src/lepton/include

Then, here’s the code for that:

import sensor, struct
def read_enable(en):
    sensor.ioctl(sensor.IOCTL_LEPTON_SET_ATTRIBUTE, 0x4E11, struct.pack("<I", 1 if en else 0))

If you look at the API… all the OpenMV Cam code is doing is just executing a FLIR Lepton set attribute command, passing the attribute number to that set command, and then some byte struct. If you check the API for this command it says it expects two shorts (4 bytes) which is the enable enum (32-bits) which can have values 0 or 1.

For various reasons the lengths in the FLIR API are all in shorts (16-bits), that’s what the IDD means when it says command get or set length.

Anyway, for setting things you want to pass a byte struct created by struct.pack(). Then you create the byte struct and pass that to the OpenMV Cam IOCTL code.

And this example before shows grabbing data from the lepton:

import sensor, struct
data = sensor.ioctl(sensor.IOCTL_LEPTON_GET_ATTRIBUTE, 0x4ED0, 4)
radSpotmeterValue, radSpotmeterMaxValue, radSpotmeterMinValue, radSpotmeterPopulation = struct.unpack("<HHHH", data)

As you can see above… the command will do a get operation for opcode 0x4ED0 to the lepton and get 4 shorts back which I then unpack into the 4 values above.

Thank you for the inputs by showing the example. I have understood it better. I wanted to know how I could read the values in OpenMV IDE of the below code u wrote?

import sensor, struct
data = sensor.ioctl(sensor.IOCTL_LEPTON_GET_ATTRIBUTE, 0x4ED0, 4)
radSpotmeterValue, radSpotmeterMaxValue, radSpotmeterMinValue, radSpotmeterPopulation = struct.unpack(“<HHHH”, data)

Print() the values. They will then appear in the terminal in the IDE.

The OpenMV Cam runs MicroPython onboard. You can do a great many things in python onboard it to manipulate data and print it.