Radiometric vs Non-Radiometric Leptons

Hi,

Its probably not the right place to be asking the question but I’m struggling to find a specific answer, how does a radiometric lepton (3.5) differ from a non-radiometric. More specifically when reading about radiometric measurement it says each pixel can be interpreted as a temperature value. What is a pixel from a non-radiometric lepton considered then, surely it could also be mapped back to a temperature as well? In which case how are they different.

I have been trying to understand where the object detection examples you provide actually make use of the radiometric aspect of the Lepton. My understanding is that a radiometric camera will give each pixel a temperature value and this data is stored in the image metadata. So why are the object detection examples converting a color to a temperature using the map_g_to_temp() function and not using the raw temperature values that a radiometric camera is supposed to provide.

My other question is how do I maintain the image as a radiometric image when sending it between devices because it currently seems like I am sending a standard .jpg file instead of a radiometric .jpg file which would have the temperature data in the images metadata. My current compressed .jpg images don’t seem to contain this information which is critical if your doing post processing.

Thanks,
Gar

The Lepton outputs 14-bit data. We are an 8-bit device. So, given the min and max temp we scale that 14-bit data to 8-bits.

If you want to edit the c code on the device to process 14-bit data you can.

Radiometric means that the Lepton will try to output valid calibrated temperature readings. Without this feature the thermal image just looks pretty but isn’t accurate in any way shape or form. For like a hot spot detector this is fine. The thermal image doesn’t need to be calibrated.

The Lepton has a “hot spot” API call you can do to get the temperature of what’s in the center of the image. We don’t implement this feature in our firmware currently… But, it would allow you to get a 14-bit temperature reading given a region of interest. This hotspot reading is valid for any Lepton.

Hi,

By converting from 14 bit to 8 bit is the radiometric aspect lost?

So really in the current implementation of the Lepton code you provide, it doesn’t make use of the radiometric aspect of the camera? I’m not sure I fully understand your explanation. Does that mean then the map_g_to_temp() function is not providing me with a valid calibrated temperature reading in which case what does that temperature actually represent?

This page Radiometric JPEG images references radiometric data which I’m trying to understand if I can access on the H7/ ideally pass the radiometric jpeg to another device for post processing. It seems like the example code you have for the lepton is treating it as a “normal” jpeg which is different.

This line from your docs would suggest it should be giving an array of temperature values:

sensor.IOCTL_LEPTON_SET_MEASUREMENT_MODE - Pass this followed by True or False to turn off automatic gain control on the FLIR Lepton and force it to output an image where each pixel value represents an exact temperature value in celsius.

You are confusing yourself by thinking this is more complex than it is. All we are doing is scaling the output to 8-bits. The 8-bit value just looses precision - not accuracy. Meaning that instead of knowing the temperature to two decimal places you only know it to one.

Non-radiometic units might have precision but not accuracy. I.e. it will tell you temperature readings but they are wrong.

In non measurement mode the sensor runs an algorithm called AGC which adaptively scales the 14-bits to 8-bits using a method that changes per area of the image. This gives visually pleasing results but you will be unable to extract the temperature information from the image.

With jpeg compression on a thermal image you will effectively further comprise the precision depending on the jpeg quality. That may or may not be okay for you. Jpeg compression will probably result in 1/2-bits of precision loss effectively.

You can save images as BMP files to avoid this

Hi,

Could you provide me with a link to the file that scales the data from 14 bit to 8 bit please? I would like to look at the source code as it seems I will need to edit it to get the radiometric data that I require.

https://github.com/openmv/openmv/blob/master/src/omv/lepton.c#L602

Hi,

Is the logical AND of value and 0xff in the switch statement where the 14 bit to 8 bit conversion is actually happening. I had a quick look at the code and specifically looked up the 0xff operator which gives you the last 8 bits of the value. The code I’m looking at is here:

IMAGE_PUT_GRAYSCALE_PIXEL(image, t_x, t_y, value & 0xFF);

If that is the case, would a quick approach to get the raw 14 bit data be to remove this AND operation as well as the temperature conversion code:

float celsius = (value * 0.01f) - 273.15f;
celsius = IM_MAX(IM_MIN(celsius, max_temp), min_temp);
value = IM_MAX(IM_MIN(IM_DIV(((celsius - min_temp) * 255), (max_temp - min_temp)), 255), 0);

Hi, the camera has no way to display 14-bit data. None of our code works on that pixel depth.

If you want to shove a 14-bit grayscale output into a 16-bit RGB565 image you can do that. However, the only thing you can do then is just write the image to disk as a binary file. None of our CV methods can be used on it. But, you’d be able to save the raw 14-bit images.

Hi

I would like to access the raw temperature data the camera produces I understand that I would still need to convert it to 8 bit or whatever for display purposes. So in short I would like to have an array of raw temperature values and the 8 bit greyscale image. I understand what you mean about converting to 16 bit and its limitations.

Maybe the way you have it implemented is enough for what i need without having to change the firmware. We are trying to use the lepton to find hot spots on server racks and then direct the cool air towards that hot spot. We wanted to have the raw temperature data the camera produces as well as the image for any post processing we do.

There’s another thread about using the RAD spot-o-mter. This is what you want to use. It lets you get the temperature of an ROI.

There’s another thread about using the RAD spot-o-mter. This is what you want to use. It lets you get the temperature of an ROI.

Yes I saw it. The spot meter requires me to give it the coordinates of an area I want to check. In terms of a server rack I cant say the hotspot will always be at a specifc point, hence why I wanted the raw data as well as an 8 bit greyscale image. We would then implement our own hotspot detection code to find the the hotspots and then use the raw temperature data to get temperatures for each of the hotspots which would be passed to a centralised cooling system.

The radiometric roi is not dynamic enough is what I’m trying to say

Yes, but, you can move the ROI to a new location based on 8-bit hotspot tracking.

But, really, here’s the deal. Why do you need 14-bits? 8-bits gives you enough resolution if you set the min and max temperature. You seem to be asking for a feature that you don’t need.

What’s your min and max temperature range? If you are going from 20C to 60C then 8-bits gives you ~0.15C precision.

Yes maybe your right, I’m just trying to understand it which the source code definitely helped with. The critical temperature range is 20 to 40 degrees, you wouldn’t really want a server reaching 40 degrees anyway.

Hi,

So I had the temperature settings set at -10 and 140 which by my calculations would result in a possible loss of precision of approximately 0.58C in an 8 bit greyscale image, is that correct? I didn’t intend on leaving the temperature window that wide going forward anyway but I didn’t realise it could potentially introduce that level of inaccuracy?

By moving the ROI based on 8 bit hotspot tracking, do you mean use greyscale colour tracking to find the hotspot then send the coordinates of the hotspot to the builtin Flir ROI functions using the IOCTL commands to get the raw temperature data (min, max and avg)?

Yes, that’s correct.

And for your second question yes too. You’d tell the camera which ROI you want dynamically and then query the precise temp from that.

Hi,

I would like to summarise my understanding of this code and see If you can tell me whether I am on the right lines or not:

int value = __REV16(row_ptr[fast_floorf(x * scale_inv)]);

                        if (measurement_mode) {
                            // Need to convert 14/16-bits to 8-bits ourselves...
                            if (!radiometry) value = (value - 8192) + kelvin;
                            float celsius = (value * 0.01f) - 273.15f;
                            celsius = IM_MAX(IM_MIN(celsius, max_temp), min_temp);
                            value = IM_MAX(IM_MIN(IM_DIV(((celsius - min_temp) * 255), (max_temp - min_temp)), 255), 0);
                        }

                       .....

                        switch (sensor->pixformat) {
                            case PIXFORMAT_GRAYSCALE: {
                                IMAGE_PUT_GRAYSCALE_PIXEL(image, t_x, t_y, value & 0xFF);
                                break;
                            }

If radiometry is enabled then do value = (value - 8192) + kelvin otherwise convert to celsuis. This line seems to be taking the FFC into account.
Check if the celsius value is less than max_temp and greater than min temp.
Im not 100% sure what the IM_DIV function is checking but I can see you are converting the values to between 0 and 255 - if you could explain that in more detail I would appreciate. And then provide the image with the coordinates of the pixel and the 14 bit to 8 bit converted greyscale value.


Getting back to my previous issues with accessing the raw data. I am not trying to display 14 bit data I simply want an array of celsius values or better yet an array of values that have no conversion applied to them or no clamping. The reason I need this is the temperature clamping you do means I lose all temperature data outside of the temperature window which I cannot lose. Or at least thats how I have interpreted all of this.

Hi, help support is for the features we have exposed. If you are going to ask questions about the code you should make an error to install a good editor and then lookup what all parts do. IM_DIV() for example is the divide operation with protection for divide by 0.

If you want to understand that code read the datasheets for the FLIR Lepton 1/2/3. I don’t remember exactly why it’s done like that in our code. But, it’s correct.

Regarding the 14-bit data any. I has a todo list task to support the FLIR Lepton in the FIR module. When this is done you will be able to get the 14-bit image. However, don’t expect this for a few months.