Radiometric vs Non-Radiometric Leptons

Discussion related to "under the hood" OpenMV topics.
GL-ITC
Posts: 61
Joined: Tue Feb 25, 2020 7:09 am

Radiometric vs Non-Radiometric Leptons

Postby GL-ITC » Thu Jun 18, 2020 9:21 am

Hi,

Its probably not the right place to be asking the question but I'm struggling to find a specific answer, how does a radiometric lepton (3.5) differ from a non-radiometric. More specifically when reading about radiometric measurement it says each pixel can be interpreted as a temperature value. What is a pixel from a non-radiometric lepton considered then, surely it could also be mapped back to a temperature as well? In which case how are they different.

I have been trying to understand where the object detection examples you provide actually make use of the radiometric aspect of the Lepton. My understanding is that a radiometric camera will give each pixel a temperature value and this data is stored in the image metadata. So why are the object detection examples converting a color to a temperature using the map_g_to_temp() function and not using the raw temperature values that a radiometric camera is supposed to provide.

My other question is how do I maintain the image as a radiometric image when sending it between devices because it currently seems like I am sending a standard .jpg file instead of a radiometric .jpg file which would have the temperature data in the images metadata. My current compressed .jpg images don't seem to contain this information which is critical if your doing post processing.

Thanks,
Gar
User avatar
kwagyeman
Posts: 4430
Joined: Sun May 24, 2015 2:10 pm

Re: Radiometric vs Non-Radiometric Leptons

Postby kwagyeman » Thu Jun 18, 2020 3:25 pm

The Lepton outputs 14-bit data. We are an 8-bit device. So, given the min and max temp we scale that 14-bit data to 8-bits.

If you want to edit the c code on the device to process 14-bit data you can.

Radiometric means that the Lepton will try to output valid calibrated temperature readings. Without this feature the thermal image just looks pretty but isn't accurate in any way shape or form. For like a hot spot detector this is fine. The thermal image doesn't need to be calibrated.

The Lepton has a "hot spot" API call you can do to get the temperature of what's in the center of the image. We don't implement this feature in our firmware currently... But, it would allow you to get a 14-bit temperature reading given a region of interest. This hotspot reading is valid for any Lepton.
Nyamekye,
GL-ITC
Posts: 61
Joined: Tue Feb 25, 2020 7:09 am

Re: Radiometric vs Non-Radiometric Leptons

Postby GL-ITC » Thu Jun 18, 2020 3:39 pm

Hi,

By converting from 14 bit to 8 bit is the radiometric aspect lost?

So really in the current implementation of the Lepton code you provide, it doesn't make use of the radiometric aspect of the camera? I'm not sure I fully understand your explanation. Does that mean then the map_g_to_temp() function is not providing me with a valid calibrated temperature reading in which case what does that temperature actually represent?

This page https://flir.custhelp.com/app/answers/d ... /related/1 references radiometric data which I'm trying to understand if I can access on the H7/ ideally pass the radiometric jpeg to another device for post processing. It seems like the example code you have for the lepton is treating it as a "normal" jpeg which is different.

This line from your docs would suggest it should be giving an array of temperature values:
sensor.IOCTL_LEPTON_SET_MEASUREMENT_MODE - Pass this followed by True or False to turn off automatic gain control on the FLIR Lepton and force it to output an image where each pixel value represents an exact temperature value in celsius.
User avatar
kwagyeman
Posts: 4430
Joined: Sun May 24, 2015 2:10 pm

Re: Radiometric vs Non-Radiometric Leptons

Postby kwagyeman » Thu Jun 18, 2020 6:52 pm

You are confusing yourself by thinking this is more complex than it is. All we are doing is scaling the output to 8-bits. The 8-bit value just looses precision - not accuracy. Meaning that instead of knowing the temperature to two decimal places you only know it to one.

Non-radiometic units might have precision but not accuracy. I.e. it will tell you temperature readings but they are wrong.

In non measurement mode the sensor runs an algorithm called AGC which adaptively scales the 14-bits to 8-bits using a method that changes per area of the image. This gives visually pleasing results but you will be unable to extract the temperature information from the image.
Nyamekye,
User avatar
kwagyeman
Posts: 4430
Joined: Sun May 24, 2015 2:10 pm

Re: Radiometric vs Non-Radiometric Leptons

Postby kwagyeman » Thu Jun 18, 2020 6:54 pm

With jpeg compression on a thermal image you will effectively further comprise the precision depending on the jpeg quality. That may or may not be okay for you. Jpeg compression will probably result in 1/2-bits of precision loss effectively.
Nyamekye,
User avatar
kwagyeman
Posts: 4430
Joined: Sun May 24, 2015 2:10 pm

Re: Radiometric vs Non-Radiometric Leptons

Postby kwagyeman » Thu Jun 18, 2020 6:54 pm

You can save images as BMP files to avoid this
Nyamekye,
GL-ITC
Posts: 61
Joined: Tue Feb 25, 2020 7:09 am

Re: Radiometric vs Non-Radiometric Leptons

Postby GL-ITC » Tue Jul 07, 2020 11:10 am

Hi,

Could you provide me with a link to the file that scales the data from 14 bit to 8 bit please? I would like to look at the source code as it seems I will need to edit it to get the radiometric data that I require.
User avatar
kwagyeman
Posts: 4430
Joined: Sun May 24, 2015 2:10 pm

Re: Radiometric vs Non-Radiometric Leptons

Postby kwagyeman » Tue Jul 07, 2020 11:57 am

Nyamekye,
GL-ITC
Posts: 61
Joined: Tue Feb 25, 2020 7:09 am

Re: Radiometric vs Non-Radiometric Leptons

Postby GL-ITC » Mon Jul 13, 2020 6:43 am

Hi,

Is the logical AND of value and 0xff in the switch statement where the 14 bit to 8 bit conversion is actually happening. I had a quick look at the code and specifically looked up the 0xff operator which gives you the last 8 bits of the value. The code I'm looking at is here:

Code: Select all

IMAGE_PUT_GRAYSCALE_PIXEL(image, t_x, t_y, value & 0xFF);
If that is the case, would a quick approach to get the raw 14 bit data be to remove this AND operation as well as the temperature conversion code:

Code: Select all

float celsius = (value * 0.01f) - 273.15f;
celsius = IM_MAX(IM_MIN(celsius, max_temp), min_temp);
value = IM_MAX(IM_MIN(IM_DIV(((celsius - min_temp) * 255), (max_temp - min_temp)), 255), 0);
User avatar
kwagyeman
Posts: 4430
Joined: Sun May 24, 2015 2:10 pm

Re: Radiometric vs Non-Radiometric Leptons

Postby kwagyeman » Mon Jul 13, 2020 11:13 am

Hi, the camera has no way to display 14-bit data. None of our code works on that pixel depth.

If you want to shove a 14-bit grayscale output into a 16-bit RGB565 image you can do that. However, the only thing you can do then is just write the image to disk as a binary file. None of our CV methods can be used on it. But, you'd be able to save the raw 14-bit images.
Nyamekye,
GL-ITC
Posts: 61
Joined: Tue Feb 25, 2020 7:09 am

Re: Radiometric vs Non-Radiometric Leptons

Postby GL-ITC » Mon Jul 13, 2020 11:25 am

Hi

I would like to access the raw temperature data the camera produces I understand that I would still need to convert it to 8 bit or whatever for display purposes. So in short I would like to have an array of raw temperature values and the 8 bit greyscale image. I understand what you mean about converting to 16 bit and its limitations.

Maybe the way you have it implemented is enough for what i need without having to change the firmware. We are trying to use the lepton to find hot spots on server racks and then direct the cool air towards that hot spot. We wanted to have the raw temperature data the camera produces as well as the image for any post processing we do.
User avatar
kwagyeman
Posts: 4430
Joined: Sun May 24, 2015 2:10 pm

Re: Radiometric vs Non-Radiometric Leptons

Postby kwagyeman » Mon Jul 13, 2020 1:02 pm

There's another thread about using the RAD spot-o-mter. This is what you want to use. It lets you get the temperature of an ROI.
Nyamekye,
GL-ITC
Posts: 61
Joined: Tue Feb 25, 2020 7:09 am

Re: Radiometric vs Non-Radiometric Leptons

Postby GL-ITC » Mon Jul 13, 2020 1:18 pm

Yes I saw it. The spot meter requires me to give it the coordinates of an area I want to check. In terms of a server rack I cant say the hotspot will always be at a specifc point, hence why I wanted the raw data as well as an 8 bit greyscale image. We would then implement our own hotspot detection code to find the the hotspots and then use the raw temperature data to get temperatures for each of the hotspots which would be passed to a centralised cooling system.

The radiometric roi is not dynamic enough is what I'm trying to say
User avatar
kwagyeman
Posts: 4430
Joined: Sun May 24, 2015 2:10 pm

Re: Radiometric vs Non-Radiometric Leptons

Postby kwagyeman » Mon Jul 13, 2020 2:06 pm

Yes, but, you can move the ROI to a new location based on 8-bit hotspot tracking.

...

But, really, here's the deal. Why do you need 14-bits? 8-bits gives you enough resolution if you set the min and max temperature. You seem to be asking for a feature that you don't need.

What's your min and max temperature range? If you are going from 20C to 60C then 8-bits gives you ~0.15C precision.
Nyamekye,
GL-ITC
Posts: 61
Joined: Tue Feb 25, 2020 7:09 am

Re: Radiometric vs Non-Radiometric Leptons

Postby GL-ITC » Mon Jul 13, 2020 2:58 pm

Yes maybe your right, I'm just trying to understand it which the source code definitely helped with. The critical temperature range is 20 to 40 degrees, you wouldn't really want a server reaching 40 degrees anyway.
GL-ITC
Posts: 61
Joined: Tue Feb 25, 2020 7:09 am

Re: Radiometric vs Non-Radiometric Leptons

Postby GL-ITC » Tue Jul 14, 2020 6:54 am

Hi,

So I had the temperature settings set at -10 and 140 which by my calculations would result in a possible loss of precision of approximately 0.58C in an 8 bit greyscale image, is that correct? I didn't intend on leaving the temperature window that wide going forward anyway but I didn't realise it could potentially introduce that level of inaccuracy?

By moving the ROI based on 8 bit hotspot tracking, do you mean use greyscale colour tracking to find the hotspot then send the coordinates of the hotspot to the builtin Flir ROI functions using the IOCTL commands to get the raw temperature data (min, max and avg)?
User avatar
kwagyeman
Posts: 4430
Joined: Sun May 24, 2015 2:10 pm

Re: Radiometric vs Non-Radiometric Leptons

Postby kwagyeman » Tue Jul 14, 2020 12:09 pm

Yes, that's correct.

And for your second question yes too. You'd tell the camera which ROI you want dynamically and then query the precise temp from that.
Nyamekye,
GL-ITC
Posts: 61
Joined: Tue Feb 25, 2020 7:09 am

Re: Radiometric vs Non-Radiometric Leptons

Postby GL-ITC » Wed Jul 15, 2020 8:34 am

Hi,

I would like to summarise my understanding of this code and see If you can tell me whether I am on the right lines or not:

Code: Select all

int value = __REV16(row_ptr[fast_floorf(x * scale_inv)]);

                        if (measurement_mode) {
                            // Need to convert 14/16-bits to 8-bits ourselves...
                            if (!radiometry) value = (value - 8192) + kelvin;
                            float celsius = (value * 0.01f) - 273.15f;
                            celsius = IM_MAX(IM_MIN(celsius, max_temp), min_temp);
                            value = IM_MAX(IM_MIN(IM_DIV(((celsius - min_temp) * 255), (max_temp - min_temp)), 255), 0);
                        }

                       .....

                        switch (sensor->pixformat) {
                            case PIXFORMAT_GRAYSCALE: {
                                IMAGE_PUT_GRAYSCALE_PIXEL(image, t_x, t_y, value & 0xFF);
                                break;
                            }
If radiometry is enabled then do value = (value - 8192) + kelvin otherwise convert to celsuis. This line seems to be taking the FFC into account.
Check if the celsius value is less than max_temp and greater than min temp.
Im not 100% sure what the IM_DIV function is checking but I can see you are converting the values to between 0 and 255 - if you could explain that in more detail I would appreciate. And then provide the image with the coordinates of the pixel and the 14 bit to 8 bit converted greyscale value.


Getting back to my previous issues with accessing the raw data. I am not trying to display 14 bit data I simply want an array of celsius values or better yet an array of values that have no conversion applied to them or no clamping. The reason I need this is the temperature clamping you do means I lose all temperature data outside of the temperature window which I cannot lose. Or at least thats how I have interpreted all of this.
User avatar
kwagyeman
Posts: 4430
Joined: Sun May 24, 2015 2:10 pm

Re: Radiometric vs Non-Radiometric Leptons

Postby kwagyeman » Wed Jul 15, 2020 1:07 pm

Hi, help support is for the features we have exposed. If you are going to ask questions about the code you should make an error to install a good editor and then lookup what all parts do. IM_DIV() for example is the divide operation with protection for divide by 0.

If you want to understand that code read the datasheets for the FLIR Lepton 1/2/3. I don't remember exactly why it's done like that in our code. But, it's correct.

...

Regarding the 14-bit data any. I has a todo list task to support the FLIR Lepton in the FIR module. When this is done you will be able to get the 14-bit image. However, don't expect this for a few months.
Nyamekye,
GL-ITC
Posts: 61
Joined: Tue Feb 25, 2020 7:09 am

Re: Radiometric vs Non-Radiometric Leptons

Postby GL-ITC » Wed Jul 22, 2020 9:17 am

Hi,

I completely misread one of your earlier responses in this thread which is why I got so confused. You said
in non measurement mode the sensor runs an algorithm called AGC which adaptively scales the 14-bits to 8-bits using a method that changes per area of the image
. I have since looked at the flir data sheet in more detail and see that the 16 bit raw value was what i needed because I want tlinear output enabled:
With AGC disabled:
▪ With 14-bit raw data the first two bits of each pixel's two-byte word are set to 0.
▪ With TLinear output all 16 bits are used.
I had misread and thought I needed the 14 bit value but I don't as you correctly said. I have since modified the firmware so I am getting the 16 bit scaled output. Thanks.
User avatar
kwagyeman
Posts: 4430
Joined: Sun May 24, 2015 2:10 pm

Re: Radiometric vs Non-Radiometric Leptons

Postby kwagyeman » Wed Jul 22, 2020 11:18 am

Cool
Nyamekye,
GL-ITC
Posts: 61
Joined: Tue Feb 25, 2020 7:09 am

Re: Radiometric vs Non-Radiometric Leptons

Postby GL-ITC » Fri Jul 31, 2020 6:40 am

Hi,

Possibly a foolish question but do you think there is any reason why the openmv board couldn't be used to calibrate the lepton 3.5 as per the Flir guide https://www.flir.com/globalassets/impor ... -guide.pdf. I know it mentions communicating with the camera over i2c and using a VoSPI stream to take frames (which is basically what you do anyway when you take a snapshot.) Given that the ioctl functions can call the sdk commands I'd image it would work, just wondering if you think there is any reason it won't.
User avatar
kwagyeman
Posts: 4430
Joined: Sun May 24, 2015 2:10 pm

Re: Radiometric vs Non-Radiometric Leptons

Postby kwagyeman » Fri Jul 31, 2020 10:55 am

Yes, you can do that with the OpenMV Cam. And if you are feeling very generous you can post the python code here that performs the calibration (it would be a huge benefit to others).
Nyamekye,
GL-ITC
Posts: 61
Joined: Tue Feb 25, 2020 7:09 am

Re: Radiometric vs Non-Radiometric Leptons

Postby GL-ITC » Fri Jul 31, 2020 2:12 pm

Yeah no problem. I won't get to it immediately as I have a few other features to develop first but when I eventually get it done I'll post the code. Thanks
Stub
Posts: 7
Joined: Mon Dec 23, 2019 8:30 am

Re: Radiometric vs Non-Radiometric Leptons

Postby Stub » Fri Aug 07, 2020 11:57 am

Hi,

I have designed and built a low cost black body - if you need one to help with this process please let me know.
User avatar
RECOVA
Posts: 2
Joined: Tue Jul 21, 2020 8:25 pm
Location: Quito
Contact:

Re: Radiometric vs Non-Radiometric Leptons

Postby RECOVA » Sat Aug 08, 2020 10:44 pm

Hello,
I am trying to obtain the temperatures of each pixel of the lepton 2.5, could you tell me which would be the SDK for the configuration, I have tried with the example codes but I get sensor control failed. your help please
User avatar
kwagyeman
Posts: 4430
Joined: Sun May 24, 2015 2:10 pm

Re: Radiometric vs Non-Radiometric Leptons

Postby kwagyeman » Sun Aug 09, 2020 12:42 am

Hi, please read the forums posts by GL-ITC on this. He's already covered this topic is a lot of detail.

search.php?author_id=1812&sr=posts

Summary answer if you don't feel like reading all the threads from him:

-> The examples provide a valid temperature that's accurate and if you are trying to do something with richer data you aren't gaining much. You should follow them.
Nyamekye,

Return to “Technical Discussion”

Who is online

Users browsing this forum: acutetech and 4 guests