Page 1 of 1

Temperature value of each pixel captured by LEPTON

Posted: Mon Dec 30, 2019 4:46 pm
by jubi5312
Hi,

I use Lepton 3.5 with OpenMV board. I wanted to get the temperature values of each pixel of the frame captured(160*120). Are there any available codes for this. Could the pixel data be accessed in excel?

~Jubin

Re: Temperature value of each pixel captured by LEPTON

Posted: Mon Dec 30, 2019 5:34 pm
by kwagyeman
Hi, you need to put the camera into a target temperature range mode (see the flir example scripts) and then comment out the code in the example scripts that markup the image with blob locations.

You can then save the image to an SD card or save the image in OpenMV IDE by right clicking on the screen. Please make sure to keep the FLIR Lepton in grayscale mode when doing this do that you have a linear mapping of pixels with value 0-255 as the min/max temperature range you set the FLIR lepton to look for.

Re: Temperature value of each pixel captured by LEPTON

Posted: Fri Jan 03, 2020 2:04 pm
by jubi5312
Hi,

I wanted to add the spot-meter feature and read the temperature value of a precise spot(roi) under measurement. I accessed the FLIR LEPTON IDD documentation and it has an example SDK interface code for obtaining spot-meter value, I have attached the snapshot of this). Could you kindly guide me on how to use this code in the OpenMV IDE for the OpenMV Lepton module to work with spot-meter feature. The spot-meter ability on an IR camera enables to measure specific temperature values of an object under study!

Thanking you,
~Jubin

Re: Temperature value of each pixel captured by LEPTON

Posted: Fri Jan 03, 2020 7:44 pm
by jubi5312
Hi,

The saved image does not give temperature for each pixel. Is there any way to get temperature for each pixel in the frame?

Thanking you,
Jubin

Re: Temperature value of each pixel captured by LEPTON

Posted: Fri Jan 03, 2020 8:10 pm
by kwagyeman
Hi, do something like this to get the SPOT-O-Meter value:

Code: Select all

import sensor, struct
data = sensor.ioctl(sensor.IOCTL_LEPTON_GET_ATTRIBUTE, 0x4ED0, 4)
radSpotmeterValue, radSpotmeterMaxValue, radSpotmeterMinValue, radSpotmeterPopulation = struct.unpack("<HHHH", data)
For other commands, you just need to pass the ID hex number and then the size in 16-bit shorts that are read from the device. Then you just have to unpack the data which will be in the order of the struct for the FLIR SDK. For the set operation you'd do a similar thing where you'd call the set attribute and then pass the SDK ID hex number along with a byte array of packed bytes which are the struct in the SDK document to send.

Re: Temperature value of each pixel captured by LEPTON

Posted: Fri Jan 03, 2020 8:12 pm
by kwagyeman
If you want to get the temperature per pixel please run the "lepton target temp hot spot color tracking" example under the FLIR lepton examples. You can set the min and max range which all pixels are mapped to in the image. Then pixels will go from 0-255 within your range you set. To get the temp per pixel you can look at the code in the example.

Re: Temperature value of each pixel captured by LEPTON

Posted: Tue Feb 25, 2020 7:31 am
by GL-ITC
Hi,

I have a question regarding the colour thresholds and the min and max temperature range. If I set the colour threshold to (120, 255), the minimum temperature to 25 degrees and the maximum temperature to 45 degrees does this mean my minimum temperature is mapped to the 120 greyscale colour or is the minimum temperature mapped to a greyscale value of 0?

Re: Temperature value of each pixel captured by LEPTON

Posted: Tue Feb 25, 2020 2:30 pm
by kwagyeman
It would be 0. The thresholds are applied on the mapped image. You set a min and max temperature range which maps all pixels on the image to 0 to 255 which represent the min and max temperature range. This is just simple scaling...

Re: Temperature value of each pixel captured by LEPTON

Posted: Tue Feb 25, 2020 2:31 pm
by GL-ITC
Ok thank you

Re: Temperature value of each pixel captured by LEPTON

Posted: Thu Mar 26, 2020 12:47 pm
by matiasandina
Hello,

Just a follow up question.

My script saves .jpg images. If I understood correctly, the max and min temps I selected will be mapped to 0 and 255 into an rgb image. Does this mean that the same rgb image could be actually encoding different temperatures if taken on two sessions with different max and min parameters ?

I think that I will add saving of parameters to be able to rescale everything to a common scale later.

Re: Temperature value of each pixel captured by LEPTON

Posted: Thu Mar 26, 2020 1:32 pm
by kwagyeman
Yes, I would open a file in python. Just use standard python code to open a text file and write out the file name and the temperature scale you used to that text file log. You can open the file in append mode so that you just append to it.

Re: Temperature value of each pixel captured by LEPTON

Posted: Sun Jun 28, 2020 10:11 pm
by byrdman12
I understand you can simply get temperature values with Lepton 2.5, where the 14-bit value you get out is the temperature value in Kelvin * 100. How do I do this with the OpenMV?

Re: Temperature value of each pixel captured by LEPTON

Posted: Sun Jun 28, 2020 11:08 pm
by kwagyeman
You need to modify the C firmware if you want the RAW 14-bit value. We have no paths in our code to handle 14-bit data.

Re: Temperature value of each pixel captured by LEPTON

Posted: Mon Jun 29, 2020 7:45 pm
by byrdman12
Can you get raw temperature data using the Rad Spotmeter commands (for individual pixels or groups of pixels)?

Re: Temperature value of each pixel captured by LEPTON

Posted: Mon Jun 29, 2020 9:16 pm
by kwagyeman
Yes, you can. You need to use our IOCTLs to issue FLIR API calls.

Re: Temperature value of each pixel captured by LEPTON

Posted: Tue Jun 30, 2020 2:24 pm
by byrdman12
I'm trying to use this sample code you posted, but I'm not sure how to read/display the values such as radSpotmeterValue or radSpotmeterMaxValue

Code: Select all

import sensor, struct data = sensor.ioctl(sensor.IOCTL_LEPTON_GET_ATTRIBUTE, 0x4ED0, 4)
radSpotmeterValue, radSpotmeterMaxValue, radSpotmeterMinValue, radSpotmeterPopulation = struct.unpack("<HHHH", data)

Re: Temperature value of each pixel captured by LEPTON

Posted: Tue Jun 30, 2020 5:10 pm
by kwagyeman
The values are 16 bit unsigned ints. Since that fits inside of a native python int you can treat them as regular ints. So, just print() them.

Re: Temperature value of each pixel captured by LEPTON

Posted: Mon Jul 06, 2020 10:27 pm
by byrdman12
I think I need to first define the ROI, is there a command for that?

Re: Temperature value of each pixel captured by LEPTON

Posted: Mon Jul 06, 2020 11:06 pm
by kwagyeman
You have to read the lepton API and lookup the command for that and determine what struct to pass.

There are a lot of functions in the FLIR Lepton API. The python ioctls I expose allow you to access them all. It was not possible to code a wrapper in python for each one.

Re: Temperature value of each pixel captured by LEPTON

Posted: Fri Jul 10, 2020 7:44 am
by iseeinir23
Hi Kwagyeman,

How are you? I am trying to do exactly what byrdman12 is trying to do. I would like to use radSpotmeterRoi and radSpotmeterValue in conjunction to obtain a temperature value for any given pixel. I saw your example code from a previous post:

Code: Select all

data = sensor.ioctl(sensor.IOCTL_LEPTON_GET_ATTRIBUTE, 0x4ED0, 4)
. You did this for radSpotmeterValue. I understand how you calculated the command id for this one to yield 0x4ED0. The Command ID is module ID + command ID Base + command type + protection bit value. I am now trying to do the same thing for radSpotmeterRoi. However, the command id base for radSpotmeterRoi is 0xcc instead of 0x00. Does this mean that the command id will for radSpotmeterRoi will be 0x4Ecco or 0x4eco?

Thank you in advance.

Re: Temperature value of each pixel captured by LEPTON

Posted: Fri Jul 10, 2020 11:15 am
by kwagyeman
It's easier to just look at the C code API here:

https://github.com/openmv/openmv/blob/7 ... RAD.h#L133

Re: Temperature value of each pixel captured by LEPTON

Posted: Fri Jul 10, 2020 11:16 am
by kwagyeman
Notice how there are so many functions. This is why I did the IOCTLs. Because, it is not possible to do a wrapper for each one.

Re: Temperature value of each pixel captured by LEPTON

Posted: Fri Jul 10, 2020 10:54 pm
by iseeinir23
Hi! Made some progress here. Please see code below. I've initialized the sensor. Then, I used radspotmeterroi to create a rectangle roi with coord. 0,0,10,10. Then I want to use the set_attribute command send this information to the Lepton and to set this attribute. I am getting an error at line 14 though (can't convert 'int' object to str implicitly). Am I going about this the right way?

Code: Select all

import sensor, image, time, math, struct

print("Resetting Lepton...")
# These settings are applied on reset
sensor.reset()
sensor.ioctl(sensor.IOCTL_LEPTON_SET_MEASUREMENT_MODE, True)

#radspotmeter ROI

data1 = struct.pack(">HHHH", 0,0,10,10) # defining rectangle coordinates at 0,0,10,10 and putting in structure
print(data1)
sensor.ioctl(sensor.IOCTL_LEPTON_SET_ATTRIBUTE, 0x4ECD, 4)


#radspotmeter value
data2 = sensor.ioctl(sensor.IOCTL_LEPTON_GET_ATTRIBUTE, 0x4ED0, 4)
radSpotmeterValue, radSpotmeterMaxValue, radSpotmeterMinValue, radSpotmeterPopulation = struct.unpack("<HHHH", data2)

Re: Temperature value of each pixel captured by LEPTON

Posted: Sat Jul 11, 2020 12:37 am
by kwagyeman
sensor.ioctl(sensor.IOCTL_LEPTON_SET_ATTRIBUTE

Should have data1 I think instead of 4.

Re: Temperature value of each pixel captured by LEPTON

Posted: Sat Jul 11, 2020 8:58 pm
by iseeinir23
Okay, I changed "4" to "data1" for radspotmeterroi, but I received an error when running the code:

Code: Select all

sensor control failed!
By the way, thank you for helping and being so quick with responses. You are very helpful.

Re: Temperature value of each pixel captured by LEPTON

Posted: Sat Jul 11, 2020 11:30 pm
by kwagyeman
Oh, the lepton rejected the command then. What lepton module do you have? The command needs to be supported by the module.

Re: Temperature value of each pixel captured by LEPTON

Posted: Sat Jul 11, 2020 11:31 pm
by kwagyeman
The ROI command should be 0x4ECC.

Re: Temperature value of each pixel captured by LEPTON

Posted: Sat Jul 11, 2020 11:32 pm
by kwagyeman
The result index is 0x4ED0.

Please get the command values from the C code I linked to.

Re: Temperature value of each pixel captured by LEPTON

Posted: Sun Jul 19, 2020 11:32 am
by iseeinir23
Hello,

We were able to get the code running using the radspotmeterRoi and radspotmeterValue commands taken from the FLIR Lepton SDK. Now, we are able to create a rectangle on the screen and get temperature measurements back. However, the temps are a bit off. For example, pointing at my hand as reference, I get 104F back. Pointing at the wall inside my room, I get 93F back. These are obviously wrong. Do you perhaps have some insight as to why this is happening?

Thank you in advance.

Re: Temperature value of each pixel captured by LEPTON

Posted: Sun Jul 19, 2020 1:06 pm
by kwagyeman
Are you using a FLIR Lepton 3.5? Other leptons don't really return right temperature readings.

I've never used the rad-spot-o-meter command. I just know it exists. I can't quite say what's wrong.

Re: Temperature value of each pixel captured by LEPTON

Posted: Mon Jul 20, 2020 4:11 pm
by GL-ITC
Hi Iseeinir23,

I'm definitely not an expert but it sounds like you need to calibrate the lepton off a black body source. The camera should be doing FFC automatically so the core temperature of the camera shouldn't be an issue. The lepton is calibrated at factory for a scence with 100% emissivity in close proximity to the camera which is probably not realistic in the real world. This Flir guide describes how to calibrate the lepton https://www.flir.com/globalassets/impor ... -guide.pdf

Re: Temperature value of each pixel captured by LEPTON

Posted: Thu Jul 23, 2020 10:18 am
by GL-ITC
Hi,

When I issue the rad spot-o-meter set command with the following values I get a sensor control failed error. The code I use is:

Code: Select all

roi = struct.pack("<HHHH", 0,0,159,119) # defining rectangle coordinates and putting in structure
sensor.ioctl(sensor.IOCTL_LEPTON_SET_ATTRIBUTE, 0x4ECD, roi)
    

data = sensor.ioctl(sensor.IOCTL_LEPTON_GET_ATTRIBUTE, 0x4ED0, 4)
radSpotmeterValue, radSpotmeterMaxValue, radSpotmeterMinValue, radSpotmeterPopulation = struct.unpack("<HHHH", data)
    
I'm trying to set the ROI to cover the entire image but these values don't seem to work, if I change them to 0,0,10,10 it works fine. From my reading of their SDK document it can be done because the maximum value the radSpotmeterPopulation variable can hold is 19200 which is the entire pixel array. Any ideas?

The result of the get operation does not seem to be based on the values I set on the occasions that it works.

Thanks

Re: Temperature value of each pixel captured by LEPTON

Posted: Thu Jul 23, 2020 11:04 am
by kwagyeman
Do you have a FLIR Lepton 3/3.5 or less? Only the 3 accepts the 160x120.

Sorry I can't be more help here... We just supply the I/O layer. Getting the commands to work takes quite a bit of effort.

During the driver development we spent countless hours fighting the VOSPI bus which locks up unless it's clocked constantly.

Re: Temperature value of each pixel captured by LEPTON

Posted: Thu Jul 23, 2020 11:42 am
by GL-ITC
Hi,

I have the 3.5. I can set the coordinates and get the coordinates to make sure they are set to mine but when I query the ROI for min, max, avg temp and population it seems to be based off of other coordinates as far as I can see.

Re: Temperature value of each pixel captured by LEPTON

Posted: Thu Jul 23, 2020 2:02 pm
by kwagyeman
I don't know. Maybe there's more setup that's needed. Check the API document and try calling other RAD commands.

https://github.com/openmv/openmv/blob/m ... _RAD.h#L80

FLIR doesn't really have documentation on anything other than generating pretty images. I think they want to keep how to use temperature values close to their chest.

Re: Temperature value of each pixel captured by LEPTON

Posted: Thu Jul 23, 2020 4:43 pm
by GL-ITC
Yes I'll have a look. I also agree that the Flir documentation is quite poor given the complexity of the subject matter. Thank you.

Re: Temperature value of each pixel captured by LEPTON

Posted: Fri Jul 24, 2020 5:56 am
by GL-ITC
Hi,

Just to let you know, and for anyone else that may need it, I got the spotmeter stuff working. There were two things happening. I had to put a time.sleep(500) between the setting of the ROI and the getting of the ROI to allow the changes to take effect. The other thing I wanted was to set the ROI to the entire image. I had just interpreted the order of the coordinates wrong. From my understanding you have 160 pixels across and and 120 pixels down. This would mean that the maximum value for the columns is 119 and the maximum value for the row is 159 which is not what is shown in the image.

To get the spotmeter working I did the following:

Code: Select all

roi = struct.pack("<HHHH", 0,0,119,159) # defining rectangle coordinates and putting in structure
sensor.ioctl(sensor.IOCTL_LEPTON_SET_ATTRIBUTE, 0x4ECD, roi)
    
time.sleep(500)

data = sensor.ioctl(sensor.IOCTL_LEPTON_GET_ATTRIBUTE, 0x4ED0, 4)
radSpotmeterValue, radSpotmeterMaxValue, radSpotmeterMinValue, radSpotmeterPopulation = struct.unpack("<HHHH", data)

print('Number of pixels used: ', radSpotmeterPopulation, '  ', radSpotmeterValue)

Re: Temperature value of each pixel captured by LEPTON

Posted: Fri Jul 24, 2020 10:58 pm
by kwagyeman
Thanks for the note!

160 -> columns (x or width)
120 -> rows (y or height)

Re: Temperature value of each pixel captured by LEPTON

Posted: Thu Jul 30, 2020 6:41 am
by GL-ITC
Hi,

I thought this might be useful if someone like me wants the raw data or the reassurance that the 8 bit conversion the OpenMV board does is still quite accurate. I done two comparisons. The top comparison uses the Flir radiometric ROI SDK function to return the average temperature in Kelvin scaled by a factor of 100. I convert this temperature to Celsius. The key point is I set the ROI to the entire image so I am getting an average temperature for the entire image.

I then modified the firmware so the snapshot function returned an array of raw 16 bit temperature values which again are in Kelvin and scaled by a factor of 100. Again I convert these values to Celsius and get an average for the entire image. These values were used to generate the first dataset so I could compare the data from the Flir SDK versus the 16 bit raw data I got from the OpenMV. Obviously these lines are almost identical as I would expect as Im comparing raw values with raw values in essence albeit from potentially different frames.

I then change the firmware back to its original state.

The other comparison I implemented is slightly more useful. Again I used the Flir radiometric ROI SDK function to return the average temperature in Kelvin scaled by a factor of 100 which I convert to Celsius. Then I get the average grayscale value for the entire image and convert it to Celsius using the function provided by OpenMV. These values were used to generate the second dataset. As you can see the loss of precision when the openmv firmware converts the raw data to 8 bit data is minimal. In fact both datasets in my case had 500 samples and only 1 sample in each dataset had a temperature difference greater than 0.2 degrees Celsius which I believe was caused by the camera running its FFC algorithm.

Long storey short a 16 bit image would be very nice but the 8 bit grayscale is probably enough as Kwabena tried to explain to me numerous times :D

Re: Temperature value of each pixel captured by LEPTON

Posted: Thu Jul 30, 2020 10:23 am
by kwagyeman
Thanks for posting about this! :)