Use FLIR Lepton and Camera toghether on one module

Hey guys,
Can we use both FLIR Lepton and a Comera on a OpenMV Module together ? How can I do that ?

Hi, for various reasons it’s actually extremely hard to make the FLIR Lepton 3 work with any MCU. The device has silicon bugs in it that cause it to crash if not clocked by SPI constantly. Then given it’s weird vospi protocol we had to write a very creative driver solution to pull data from it without issues. The device does not lend itself to being something controlled in python via polling.

As for the Lepton 1 - that one can be polled. If you want to use it with and OpenMV Cam you can write a driver for it in python and control it via the I2C bus and read data from it’s VOSPI bus.

There are numerous example drivers to write a code bunder from that you could look at. The VOSPI protocol is not complex for the Lepton 1 and you only have to execute a few I2C commands to get it to start operating.

We don’t have the time to support you right now however for anything other than our planned usage which is a main camera replacement.

So… The easiest pah with the least amount of suprises is to use our I2C and SPI code and make a python driver for the LEPTON 1.

Alternatively, if you know C and are comfortable with the STM32 line of chips chainging the I/O pins for our C driver isn’t that hard.

Hi Kwabena,

Having fun with the H7, I’m trying out a FLIR Lepton 1 but ran into trouble and wonder if I’m doing something wrong. The Hello World script failed with a “OSError: Reset Failed”. I then put a “print (sensor.get_id())” and it returned a 84 before failing on the Reset. The normal cam returns a 119, in contrast to the documentation which indicates it might have a more descriptive ID. ??

Hi, the FLIR Lepton definitely works. I tested all 20 of the pre production units with a FLIR Lepton 3. Um, is the sensor seated right and does it work with another board? 84 is correct.

I think I’ve got the little sensor snapped properly into the adapter - two bumps on one side and one bump on the opposite. I tried popping it back out and back in but same Reset Failed message. Hmm. My thought is that if it says it an 84, then at least it’s not dead or fried. I don’t have another a board to mount it on other than the H7 and adapter. Other thoughts?

Hmm, getting 84 just means that we got an I2C ack from the chip. To get this the reset and pwdn pins are correct.

Um, but, a fail on init means the device didn’t boot given an I2C wakeup command.

Um, the error is somewhere here:

https://github.com/openmv/openmv/blob/master/src/omv/lepton.c#L223

This code works with all the lepton units I tested and I ran the FLIR 1/3 through the module I gave you when testing the board quality.

Um, does the module work with anything else?

I can give you a firmware image that prints an error message at the particular line if need be.Most likely the boot status bit isn’t going high on the lepton within 1 second. Extending the timeout might help but I’m not sure… I’ve been testing on like about 7 lepton modules.

Hi Kwabena,

Perhaps if I could know the exact place the Reset is failing, it would help me make a case for getting a replacement Lepton unit. The H7 is the only way I have to run it. So if you would be so kind as to make up a diagnostic firmware image I could move in that direction. Perhaps with longer timeout, just to rule out that possibility. Thanks again for your quick and generous support.

Here’s the firmware with test outputs.
firmware.zip (926 KB)

Thanks Kwabena. The message is “FFC failure”. Not sure what that means but I’ll contact the vendor about a replacement. I assume there’s no harm having this firmware in place for other applications on the H7.

I wrote to the vendor saying:
"Hi there. I’m trying to use the FLIR Lepton 1 in an OpenMV mounting and running a simple program that first tries to initialize the Lepton. This program fails with a message complaining of an “FFC failure”. Indications are that I2C communication is working, but it is not “booting” as required. I would like you to replace this Lepton with another. Please advise.

I heard from the vendor and he says:
"Another module will do the same thing due to your setup. These are brand new and tested. Please see the following discussion here with some solutions. : Redirecting to Google Groups

Info on the referenced Lepton forum discussion is a bit out of my depth. So I’m not sure what more to do here.

Hmm, maybe you have a FLIR one older than mine. I can skip the FFC step. It’s not really required in the FLIR 1 per say (it is kinda required - good practice).

Um, the error they mention there is because of their software package. We had to do a lot of work for the FLIR3 to get around the SPI bus issue. Basically, the FLIR dies unless you clock it’s SPI bus fast enough. I haven’t seen any RPi code that uses real time threads for the Lepton 3 so I would expect that when the RPi does random Linux stuff and missing timing deadlines this causes the module to crash.

Hi Kwabena,

Again, thanks for your response. I’m so out of my depth here that I am willing to try anything you think of. It sounds like shooting in the dark, but let’s try a version that skips the FFC step. If it then errors on another step, we can write it off.

Try this build.
firmware.zip (929 KB)

Lepton1.mp4 (30.1 KB)
Whoah! It works. Whatever you did, you did it. Thanks very much Kwabena. I love it when these toys work.

Okay, I’ll make that call optional. It’s weird your Lepton fails on it.

Sorry if it’s the wrong thread, but now I’d love to get ahold of a Singtown Video shield. I’m envisioning my drone with FPV thermal imaging. Any chance of getting one soon?

I have an extra one at my place. I just got the units from them. I asked for a small hardware change and then I think they will build more.

I need to test it out however. Will get to it this weekend.

Will these firmware builds work on a FLIR Lepton 3?

I’m trying to get OpenMV H7 Plus, Adapter and FLIR Lepton 3 working. OpenMV works great with supplied camera. Just received my second FLIR Lepton 3 from Sparkfun, same as first presumably defective FLIR, OpenMV flashes indicating camera not initializing.

Should I use any special hardware settings, firmware, python or c code?

Thank you!
Kevin

Hi, it sound work without any issue… can you take a picture of what your setup is and make sure the Lepton module is fully seated?