Temperature value of each pixel captured by LEPTON

Discussion related to "under the hood" OpenMV topics.
jubi5312
Posts: 19
Joined: Tue Jul 09, 2019 2:05 pm

Temperature value of each pixel captured by LEPTON

Postby jubi5312 » Mon Dec 30, 2019 4:46 pm

Hi,

I use Lepton 3.5 with OpenMV board. I wanted to get the temperature values of each pixel of the frame captured(160*120). Are there any available codes for this. Could the pixel data be accessed in excel?

~Jubin
User avatar
kwagyeman
Posts: 4551
Joined: Sun May 24, 2015 2:10 pm

Re: Temperature value of each pixel captured by LEPTON

Postby kwagyeman » Mon Dec 30, 2019 5:34 pm

Hi, you need to put the camera into a target temperature range mode (see the flir example scripts) and then comment out the code in the example scripts that markup the image with blob locations.

You can then save the image to an SD card or save the image in OpenMV IDE by right clicking on the screen. Please make sure to keep the FLIR Lepton in grayscale mode when doing this do that you have a linear mapping of pixels with value 0-255 as the min/max temperature range you set the FLIR lepton to look for.
Nyamekye,
jubi5312
Posts: 19
Joined: Tue Jul 09, 2019 2:05 pm

Re: Temperature value of each pixel captured by LEPTON

Postby jubi5312 » Fri Jan 03, 2020 2:04 pm

Hi,

I wanted to add the spot-meter feature and read the temperature value of a precise spot(roi) under measurement. I accessed the FLIR LEPTON IDD documentation and it has an example SDK interface code for obtaining spot-meter value, I have attached the snapshot of this). Could you kindly guide me on how to use this code in the OpenMV IDE for the OpenMV Lepton module to work with spot-meter feature. The spot-meter ability on an IR camera enables to measure specific temperature values of an object under study!

Thanking you,
~Jubin
Attachments
Spotmeter C SDK Interface.JPG
jubi5312
Posts: 19
Joined: Tue Jul 09, 2019 2:05 pm

Re: Temperature value of each pixel captured by LEPTON

Postby jubi5312 » Fri Jan 03, 2020 7:44 pm

Hi,

The saved image does not give temperature for each pixel. Is there any way to get temperature for each pixel in the frame?

Thanking you,
Jubin
User avatar
kwagyeman
Posts: 4551
Joined: Sun May 24, 2015 2:10 pm

Re: Temperature value of each pixel captured by LEPTON

Postby kwagyeman » Fri Jan 03, 2020 8:10 pm

Hi, do something like this to get the SPOT-O-Meter value:

Code: Select all

import sensor, struct
data = sensor.ioctl(sensor.IOCTL_LEPTON_GET_ATTRIBUTE, 0x4ED0, 4)
radSpotmeterValue, radSpotmeterMaxValue, radSpotmeterMinValue, radSpotmeterPopulation = struct.unpack("<HHHH", data)
For other commands, you just need to pass the ID hex number and then the size in 16-bit shorts that are read from the device. Then you just have to unpack the data which will be in the order of the struct for the FLIR SDK. For the set operation you'd do a similar thing where you'd call the set attribute and then pass the SDK ID hex number along with a byte array of packed bytes which are the struct in the SDK document to send.
Nyamekye,
User avatar
kwagyeman
Posts: 4551
Joined: Sun May 24, 2015 2:10 pm

Re: Temperature value of each pixel captured by LEPTON

Postby kwagyeman » Fri Jan 03, 2020 8:12 pm

If you want to get the temperature per pixel please run the "lepton target temp hot spot color tracking" example under the FLIR lepton examples. You can set the min and max range which all pixels are mapped to in the image. Then pixels will go from 0-255 within your range you set. To get the temp per pixel you can look at the code in the example.
Nyamekye,
GL-ITC
Posts: 76
Joined: Tue Feb 25, 2020 7:09 am

Re: Temperature value of each pixel captured by LEPTON

Postby GL-ITC » Tue Feb 25, 2020 7:31 am

Hi,

I have a question regarding the colour thresholds and the min and max temperature range. If I set the colour threshold to (120, 255), the minimum temperature to 25 degrees and the maximum temperature to 45 degrees does this mean my minimum temperature is mapped to the 120 greyscale colour or is the minimum temperature mapped to a greyscale value of 0?
User avatar
kwagyeman
Posts: 4551
Joined: Sun May 24, 2015 2:10 pm

Re: Temperature value of each pixel captured by LEPTON

Postby kwagyeman » Tue Feb 25, 2020 2:30 pm

It would be 0. The thresholds are applied on the mapped image. You set a min and max temperature range which maps all pixels on the image to 0 to 255 which represent the min and max temperature range. This is just simple scaling...
Nyamekye,
GL-ITC
Posts: 76
Joined: Tue Feb 25, 2020 7:09 am

Re: Temperature value of each pixel captured by LEPTON

Postby GL-ITC » Tue Feb 25, 2020 2:31 pm

Ok thank you
matiasandina
Posts: 14
Joined: Mon Mar 02, 2020 1:33 pm

Re: Temperature value of each pixel captured by LEPTON

Postby matiasandina » Thu Mar 26, 2020 12:47 pm

Hello,

Just a follow up question.

My script saves .jpg images. If I understood correctly, the max and min temps I selected will be mapped to 0 and 255 into an rgb image. Does this mean that the same rgb image could be actually encoding different temperatures if taken on two sessions with different max and min parameters ?

I think that I will add saving of parameters to be able to rescale everything to a common scale later.
User avatar
kwagyeman
Posts: 4551
Joined: Sun May 24, 2015 2:10 pm

Re: Temperature value of each pixel captured by LEPTON

Postby kwagyeman » Thu Mar 26, 2020 1:32 pm

Yes, I would open a file in python. Just use standard python code to open a text file and write out the file name and the temperature scale you used to that text file log. You can open the file in append mode so that you just append to it.
Nyamekye,
byrdman12
Posts: 4
Joined: Sun Jun 28, 2020 10:02 pm

Re: Temperature value of each pixel captured by LEPTON

Postby byrdman12 » Sun Jun 28, 2020 10:11 pm

I understand you can simply get temperature values with Lepton 2.5, where the 14-bit value you get out is the temperature value in Kelvin * 100. How do I do this with the OpenMV?
User avatar
kwagyeman
Posts: 4551
Joined: Sun May 24, 2015 2:10 pm

Re: Temperature value of each pixel captured by LEPTON

Postby kwagyeman » Sun Jun 28, 2020 11:08 pm

You need to modify the C firmware if you want the RAW 14-bit value. We have no paths in our code to handle 14-bit data.
Nyamekye,
byrdman12
Posts: 4
Joined: Sun Jun 28, 2020 10:02 pm

Re: Temperature value of each pixel captured by LEPTON

Postby byrdman12 » Mon Jun 29, 2020 7:45 pm

Can you get raw temperature data using the Rad Spotmeter commands (for individual pixels or groups of pixels)?
User avatar
kwagyeman
Posts: 4551
Joined: Sun May 24, 2015 2:10 pm

Re: Temperature value of each pixel captured by LEPTON

Postby kwagyeman » Mon Jun 29, 2020 9:16 pm

Yes, you can. You need to use our IOCTLs to issue FLIR API calls.
Nyamekye,
byrdman12
Posts: 4
Joined: Sun Jun 28, 2020 10:02 pm

Re: Temperature value of each pixel captured by LEPTON

Postby byrdman12 » Tue Jun 30, 2020 2:24 pm

I'm trying to use this sample code you posted, but I'm not sure how to read/display the values such as radSpotmeterValue or radSpotmeterMaxValue

Code: Select all

import sensor, struct data = sensor.ioctl(sensor.IOCTL_LEPTON_GET_ATTRIBUTE, 0x4ED0, 4)
radSpotmeterValue, radSpotmeterMaxValue, radSpotmeterMinValue, radSpotmeterPopulation = struct.unpack("<HHHH", data)
User avatar
kwagyeman
Posts: 4551
Joined: Sun May 24, 2015 2:10 pm

Re: Temperature value of each pixel captured by LEPTON

Postby kwagyeman » Tue Jun 30, 2020 5:10 pm

The values are 16 bit unsigned ints. Since that fits inside of a native python int you can treat them as regular ints. So, just print() them.
Nyamekye,
byrdman12
Posts: 4
Joined: Sun Jun 28, 2020 10:02 pm

Re: Temperature value of each pixel captured by LEPTON

Postby byrdman12 » Mon Jul 06, 2020 10:27 pm

I think I need to first define the ROI, is there a command for that?
User avatar
kwagyeman
Posts: 4551
Joined: Sun May 24, 2015 2:10 pm

Re: Temperature value of each pixel captured by LEPTON

Postby kwagyeman » Mon Jul 06, 2020 11:06 pm

You have to read the lepton API and lookup the command for that and determine what struct to pass.

There are a lot of functions in the FLIR Lepton API. The python ioctls I expose allow you to access them all. It was not possible to code a wrapper in python for each one.
Nyamekye,
iseeinir23
Posts: 4
Joined: Fri Jul 10, 2020 5:31 am

Re: Temperature value of each pixel captured by LEPTON

Postby iseeinir23 » Fri Jul 10, 2020 7:44 am

Hi Kwagyeman,

How are you? I am trying to do exactly what byrdman12 is trying to do. I would like to use radSpotmeterRoi and radSpotmeterValue in conjunction to obtain a temperature value for any given pixel. I saw your example code from a previous post:

Code: Select all

data = sensor.ioctl(sensor.IOCTL_LEPTON_GET_ATTRIBUTE, 0x4ED0, 4)
. You did this for radSpotmeterValue. I understand how you calculated the command id for this one to yield 0x4ED0. The Command ID is module ID + command ID Base + command type + protection bit value. I am now trying to do the same thing for radSpotmeterRoi. However, the command id base for radSpotmeterRoi is 0xcc instead of 0x00. Does this mean that the command id will for radSpotmeterRoi will be 0x4Ecco or 0x4eco?

Thank you in advance.
User avatar
kwagyeman
Posts: 4551
Joined: Sun May 24, 2015 2:10 pm

Re: Temperature value of each pixel captured by LEPTON

Postby kwagyeman » Fri Jul 10, 2020 11:15 am

It's easier to just look at the C code API here:

https://github.com/openmv/openmv/blob/7 ... RAD.h#L133
Nyamekye,
User avatar
kwagyeman
Posts: 4551
Joined: Sun May 24, 2015 2:10 pm

Re: Temperature value of each pixel captured by LEPTON

Postby kwagyeman » Fri Jul 10, 2020 11:16 am

Notice how there are so many functions. This is why I did the IOCTLs. Because, it is not possible to do a wrapper for each one.
Nyamekye,
iseeinir23
Posts: 4
Joined: Fri Jul 10, 2020 5:31 am

Re: Temperature value of each pixel captured by LEPTON

Postby iseeinir23 » Fri Jul 10, 2020 10:54 pm

Hi! Made some progress here. Please see code below. I've initialized the sensor. Then, I used radspotmeterroi to create a rectangle roi with coord. 0,0,10,10. Then I want to use the set_attribute command send this information to the Lepton and to set this attribute. I am getting an error at line 14 though (can't convert 'int' object to str implicitly). Am I going about this the right way?

Code: Select all

import sensor, image, time, math, struct

print("Resetting Lepton...")
# These settings are applied on reset
sensor.reset()
sensor.ioctl(sensor.IOCTL_LEPTON_SET_MEASUREMENT_MODE, True)

#radspotmeter ROI

data1 = struct.pack(">HHHH", 0,0,10,10) # defining rectangle coordinates at 0,0,10,10 and putting in structure
print(data1)
sensor.ioctl(sensor.IOCTL_LEPTON_SET_ATTRIBUTE, 0x4ECD, 4)


#radspotmeter value
data2 = sensor.ioctl(sensor.IOCTL_LEPTON_GET_ATTRIBUTE, 0x4ED0, 4)
radSpotmeterValue, radSpotmeterMaxValue, radSpotmeterMinValue, radSpotmeterPopulation = struct.unpack("<HHHH", data2)
User avatar
kwagyeman
Posts: 4551
Joined: Sun May 24, 2015 2:10 pm

Re: Temperature value of each pixel captured by LEPTON

Postby kwagyeman » Sat Jul 11, 2020 12:37 am

sensor.ioctl(sensor.IOCTL_LEPTON_SET_ATTRIBUTE

Should have data1 I think instead of 4.
Nyamekye,
iseeinir23
Posts: 4
Joined: Fri Jul 10, 2020 5:31 am

Re: Temperature value of each pixel captured by LEPTON

Postby iseeinir23 » Sat Jul 11, 2020 8:58 pm

Okay, I changed "4" to "data1" for radspotmeterroi, but I received an error when running the code:

Code: Select all

sensor control failed!
By the way, thank you for helping and being so quick with responses. You are very helpful.
User avatar
kwagyeman
Posts: 4551
Joined: Sun May 24, 2015 2:10 pm

Re: Temperature value of each pixel captured by LEPTON

Postby kwagyeman » Sat Jul 11, 2020 11:30 pm

Oh, the lepton rejected the command then. What lepton module do you have? The command needs to be supported by the module.
Nyamekye,
User avatar
kwagyeman
Posts: 4551
Joined: Sun May 24, 2015 2:10 pm

Re: Temperature value of each pixel captured by LEPTON

Postby kwagyeman » Sat Jul 11, 2020 11:31 pm

The ROI command should be 0x4ECC.
Nyamekye,
User avatar
kwagyeman
Posts: 4551
Joined: Sun May 24, 2015 2:10 pm

Re: Temperature value of each pixel captured by LEPTON

Postby kwagyeman » Sat Jul 11, 2020 11:32 pm

The result index is 0x4ED0.

Please get the command values from the C code I linked to.
Nyamekye,
iseeinir23
Posts: 4
Joined: Fri Jul 10, 2020 5:31 am

Re: Temperature value of each pixel captured by LEPTON

Postby iseeinir23 » Sun Jul 19, 2020 11:32 am

Hello,

We were able to get the code running using the radspotmeterRoi and radspotmeterValue commands taken from the FLIR Lepton SDK. Now, we are able to create a rectangle on the screen and get temperature measurements back. However, the temps are a bit off. For example, pointing at my hand as reference, I get 104F back. Pointing at the wall inside my room, I get 93F back. These are obviously wrong. Do you perhaps have some insight as to why this is happening?

Thank you in advance.
User avatar
kwagyeman
Posts: 4551
Joined: Sun May 24, 2015 2:10 pm

Re: Temperature value of each pixel captured by LEPTON

Postby kwagyeman » Sun Jul 19, 2020 1:06 pm

Are you using a FLIR Lepton 3.5? Other leptons don't really return right temperature readings.

I've never used the rad-spot-o-meter command. I just know it exists. I can't quite say what's wrong.
Nyamekye,
GL-ITC
Posts: 76
Joined: Tue Feb 25, 2020 7:09 am

Re: Temperature value of each pixel captured by LEPTON

Postby GL-ITC » Mon Jul 20, 2020 4:11 pm

Hi Iseeinir23,

I'm definitely not an expert but it sounds like you need to calibrate the lepton off a black body source. The camera should be doing FFC automatically so the core temperature of the camera shouldn't be an issue. The lepton is calibrated at factory for a scence with 100% emissivity in close proximity to the camera which is probably not realistic in the real world. This Flir guide describes how to calibrate the lepton https://www.flir.com/globalassets/impor ... -guide.pdf
GL-ITC
Posts: 76
Joined: Tue Feb 25, 2020 7:09 am

Re: Temperature value of each pixel captured by LEPTON

Postby GL-ITC » Thu Jul 23, 2020 10:18 am

Hi,

When I issue the rad spot-o-meter set command with the following values I get a sensor control failed error. The code I use is:

Code: Select all

roi = struct.pack("<HHHH", 0,0,159,119) # defining rectangle coordinates and putting in structure
sensor.ioctl(sensor.IOCTL_LEPTON_SET_ATTRIBUTE, 0x4ECD, roi)
    

data = sensor.ioctl(sensor.IOCTL_LEPTON_GET_ATTRIBUTE, 0x4ED0, 4)
radSpotmeterValue, radSpotmeterMaxValue, radSpotmeterMinValue, radSpotmeterPopulation = struct.unpack("<HHHH", data)
    
I'm trying to set the ROI to cover the entire image but these values don't seem to work, if I change them to 0,0,10,10 it works fine. From my reading of their SDK document it can be done because the maximum value the radSpotmeterPopulation variable can hold is 19200 which is the entire pixel array. Any ideas?

The result of the get operation does not seem to be based on the values I set on the occasions that it works.

Thanks
User avatar
kwagyeman
Posts: 4551
Joined: Sun May 24, 2015 2:10 pm

Re: Temperature value of each pixel captured by LEPTON

Postby kwagyeman » Thu Jul 23, 2020 11:04 am

Do you have a FLIR Lepton 3/3.5 or less? Only the 3 accepts the 160x120.

Sorry I can't be more help here... We just supply the I/O layer. Getting the commands to work takes quite a bit of effort.

During the driver development we spent countless hours fighting the VOSPI bus which locks up unless it's clocked constantly.
Nyamekye,
GL-ITC
Posts: 76
Joined: Tue Feb 25, 2020 7:09 am

Re: Temperature value of each pixel captured by LEPTON

Postby GL-ITC » Thu Jul 23, 2020 11:42 am

Hi,

I have the 3.5. I can set the coordinates and get the coordinates to make sure they are set to mine but when I query the ROI for min, max, avg temp and population it seems to be based off of other coordinates as far as I can see.
User avatar
kwagyeman
Posts: 4551
Joined: Sun May 24, 2015 2:10 pm

Re: Temperature value of each pixel captured by LEPTON

Postby kwagyeman » Thu Jul 23, 2020 2:02 pm

I don't know. Maybe there's more setup that's needed. Check the API document and try calling other RAD commands.

https://github.com/openmv/openmv/blob/m ... _RAD.h#L80

FLIR doesn't really have documentation on anything other than generating pretty images. I think they want to keep how to use temperature values close to their chest.
Nyamekye,
GL-ITC
Posts: 76
Joined: Tue Feb 25, 2020 7:09 am

Re: Temperature value of each pixel captured by LEPTON

Postby GL-ITC » Thu Jul 23, 2020 4:43 pm

Yes I'll have a look. I also agree that the Flir documentation is quite poor given the complexity of the subject matter. Thank you.
GL-ITC
Posts: 76
Joined: Tue Feb 25, 2020 7:09 am

Re: Temperature value of each pixel captured by LEPTON

Postby GL-ITC » Fri Jul 24, 2020 5:56 am

Hi,

Just to let you know, and for anyone else that may need it, I got the spotmeter stuff working. There were two things happening. I had to put a time.sleep(500) between the setting of the ROI and the getting of the ROI to allow the changes to take effect. The other thing I wanted was to set the ROI to the entire image. I had just interpreted the order of the coordinates wrong. From my understanding you have 160 pixels across and and 120 pixels down. This would mean that the maximum value for the columns is 119 and the maximum value for the row is 159 which is not what is shown in the image.

To get the spotmeter working I did the following:

Code: Select all

roi = struct.pack("<HHHH", 0,0,119,159) # defining rectangle coordinates and putting in structure
sensor.ioctl(sensor.IOCTL_LEPTON_SET_ATTRIBUTE, 0x4ECD, roi)
    
time.sleep(500)

data = sensor.ioctl(sensor.IOCTL_LEPTON_GET_ATTRIBUTE, 0x4ED0, 4)
radSpotmeterValue, radSpotmeterMaxValue, radSpotmeterMinValue, radSpotmeterPopulation = struct.unpack("<HHHH", data)

print('Number of pixels used: ', radSpotmeterPopulation, '  ', radSpotmeterValue)
Attachments
ROI TABLE.PNG
User avatar
kwagyeman
Posts: 4551
Joined: Sun May 24, 2015 2:10 pm

Re: Temperature value of each pixel captured by LEPTON

Postby kwagyeman » Fri Jul 24, 2020 10:58 pm

Thanks for the note!

160 -> columns (x or width)
120 -> rows (y or height)
Nyamekye,
GL-ITC
Posts: 76
Joined: Tue Feb 25, 2020 7:09 am

Re: Temperature value of each pixel captured by LEPTON

Postby GL-ITC » Thu Jul 30, 2020 6:41 am

Hi,

I thought this might be useful if someone like me wants the raw data or the reassurance that the 8 bit conversion the OpenMV board does is still quite accurate. I done two comparisons. The top comparison uses the Flir radiometric ROI SDK function to return the average temperature in Kelvin scaled by a factor of 100. I convert this temperature to Celsius. The key point is I set the ROI to the entire image so I am getting an average temperature for the entire image.

I then modified the firmware so the snapshot function returned an array of raw 16 bit temperature values which again are in Kelvin and scaled by a factor of 100. Again I convert these values to Celsius and get an average for the entire image. These values were used to generate the first dataset so I could compare the data from the Flir SDK versus the 16 bit raw data I got from the OpenMV. Obviously these lines are almost identical as I would expect as Im comparing raw values with raw values in essence albeit from potentially different frames.

I then change the firmware back to its original state.

The other comparison I implemented is slightly more useful. Again I used the Flir radiometric ROI SDK function to return the average temperature in Kelvin scaled by a factor of 100 which I convert to Celsius. Then I get the average grayscale value for the entire image and convert it to Celsius using the function provided by OpenMV. These values were used to generate the second dataset. As you can see the loss of precision when the openmv firmware converts the raw data to 8 bit data is minimal. In fact both datasets in my case had 500 samples and only 1 sample in each dataset had a temperature difference greater than 0.2 degrees Celsius which I believe was caused by the camera running its FFC algorithm.

Long storey short a 16 bit image would be very nice but the 8 bit grayscale is probably enough as Kwabena tried to explain to me numerous times :D
Attachments
Radiometric vs Grayscale.png
User avatar
kwagyeman
Posts: 4551
Joined: Sun May 24, 2015 2:10 pm

Re: Temperature value of each pixel captured by LEPTON

Postby kwagyeman » Thu Jul 30, 2020 10:23 am

Thanks for posting about this! :)
Nyamekye,

Return to “Technical Discussion”

Who is online

Users browsing this forum: No registered users and 1 guest