FLIR Thermal Camera Person Detection

Ibrahim made this example script showing off person detection using the FLIR Thermal camera. Works with any FLIR Lepton.

import sensor, image, time, os, nn

sensor.reset()                         # Reset and initialize the sensor.
sensor.set_pixformat(sensor.GRAYSCALE)    # Set pixel format to RGB565
sensor.set_framesize(sensor.QQVGA)      # Set frame size to QVGA (320x240)
sensor.set_windowing((32, 64))       # Set 128x128 window.
sensor.skip_frames(time=1000)

net = nn.load('/person.network')

clock = time.clock()                # Create a clock object to track the FPS.
while(True):
    clock.tick()                    # Update the FPS clock.
    img = sensor.snapshot()         # Take a picture and return the image.
    out = net.forward(img, softmax=True)
    if out.index(max(out)) == 0:
        print("person detected")

Save the person.network to your SD card.
person.zip (64.1 KB)

HELLO
I loaded the code to my openmv cam h7 with a lepton 2.5 sensor and at the time of execution I got the error: OsERROR: this function is unavaible on your open MV cam could you help me please

We removed the NN module.

Ibrahim, do you have the dataset this model was trained on? It could be quickly moved to TensorFlow with the dataset.

Yes, I think it was this dataset:

http://portal.uc3m.es/portal/page/portal/dpto_ing_sistemas_automatica/investigacion/IntelligentSystemsLab/research/InfraredDataset

https://e-archivo.uc3m.es/handle/10016/17370

Note, you need to downsample the 14 bit images to 8 bit before training and testing, and use FLIR module in GS mode.

Will this work with the Lepton PureThermal2 USB camera plugged into a Rpi? If so, this would be a godsend. I’m trying to create a FLIR sensor that detects bears and blows an airhorn for our yurt off grid in Alaska. Brown bear tore our place to pieces while we were gone and we’re hoping to keep that from happening again. I’m going to get to researching this in the morning but many thanks for any help!

Just saw this video and I can’t tell you how excited I am if this will prove a solution. I’ll need to train it over time to recognize bears by capturing my own video I imagine (I doubt there is a bear FLIR training set out there, but who knows) and I suppose I can do that by just having it blow the horn anytime it detects something from 80-105F just to be safe. Maybe it will be a moose, but I’ll collect the images and train it over time.

Super excited about this!

I have the PureThermal2 and I think I can pop the lepton camera off of it, could I then pop that into this adapter?

Yep, that’s fine.