Use FLIR Lepton and Camera toghether on one module

General discussion about topics related to OpenMV.
Amirhossein
Posts: 1
Joined: Tue Jan 08, 2019 2:11 am

Use FLIR Lepton and Camera toghether on one module

Postby Amirhossein » Tue Jan 08, 2019 2:24 am

Hey guys,
Can we use both FLIR Lepton and a Comera on a OpenMV Module together ? How can I do that ?
User avatar
kwagyeman
Posts: 3132
Joined: Sun May 24, 2015 2:10 pm

Re: Use FLIR Lepton and Camera toghether on one module

Postby kwagyeman » Tue Jan 08, 2019 2:57 am

Hi, for various reasons it's actually extremely hard to make the FLIR Lepton 3 work with any MCU. The device has silicon bugs in it that cause it to crash if not clocked by SPI constantly. Then given it's weird vospi protocol we had to write a very *creative* driver solution to pull data from it without issues. The device does not lend itself to being something controlled in python via polling.

As for the Lepton 1 - that one can be polled. If you want to use it with and OpenMV Cam you can write a driver for it in python and control it via the I2C bus and read data from it's VOSPI bus.

There are numerous example drivers to write a code bunder from that you could look at. The VOSPI protocol is not complex for the Lepton 1 and you only have to execute a few I2C commands to get it to start operating.

We don't have the time to support you right now however for anything other than our planned usage which is a main camera replacement.

So... The easiest pah with the least amount of suprises is to use our I2C and SPI code and make a python driver for the LEPTON 1.

Alternatively, if you know C and are comfortable with the STM32 line of chips chainging the I/O pins for our C driver isn't that hard.
Nyamekye,
User avatar
dcolemans
Posts: 31
Joined: Fri Aug 25, 2017 9:43 am

Re: Use FLIR Lepton and Camera toghether on one module

Postby dcolemans » Fri Jan 25, 2019 3:41 pm

Hi Kwabena,

Having fun with the H7, I'm trying out a FLIR Lepton 1 but ran into trouble and wonder if I'm doing something wrong. The Hello World script failed with a "OSError: Reset Failed". I then put a "print (sensor.get_id())" and it returned a 84 before failing on the Reset. The normal cam returns a 119, in contrast to the documentation which indicates it might have a more descriptive ID. ??
Dave
User avatar
kwagyeman
Posts: 3132
Joined: Sun May 24, 2015 2:10 pm

Re: Use FLIR Lepton and Camera toghether on one module

Postby kwagyeman » Fri Jan 25, 2019 11:28 pm

Hi, the FLIR Lepton definitely works. I tested all 20 of the pre production units with a FLIR Lepton 3. Um, is the sensor seated right and does it work with another board? 84 is correct.
Nyamekye,
User avatar
dcolemans
Posts: 31
Joined: Fri Aug 25, 2017 9:43 am

Re: Use FLIR Lepton and Camera toghether on one module

Postby dcolemans » Sat Jan 26, 2019 1:46 am

Image I think I've got the little sensor snapped properly into the adapter - two bumps on one side and one bump on the opposite. I tried popping it back out and back in but same Reset Failed message. Hmm. My thought is that if it says it an 84, then at least it's not dead or fried. I don't have another a board to mount it on other than the H7 and adapter. Other thoughts?
Dave
User avatar
kwagyeman
Posts: 3132
Joined: Sun May 24, 2015 2:10 pm

Re: Use FLIR Lepton and Camera toghether on one module

Postby kwagyeman » Sat Jan 26, 2019 2:40 am

Hmm, getting 84 just means that we got an I2C ack from the chip. To get this the reset and pwdn pins are correct.

Um, but, a fail on init means the device didn't boot given an I2C wakeup command.
Nyamekye,
User avatar
kwagyeman
Posts: 3132
Joined: Sun May 24, 2015 2:10 pm

Re: Use FLIR Lepton and Camera toghether on one module

Postby kwagyeman » Sat Jan 26, 2019 2:44 am

Um, the error is somewhere here:

https://github.com/openmv/openmv/blob/m ... ton.c#L223

This code works with all the lepton units I tested and I ran the FLIR 1/3 through the module I gave you when testing the board quality.

Um, does the module work with anything else?

I can give you a firmware image that prints an error message at the particular line if need be.Most likely the boot status bit isn't going high on the lepton within 1 second. Extending the timeout might help but I'm not sure... I've been testing on like about 7 lepton modules.
Nyamekye,
User avatar
dcolemans
Posts: 31
Joined: Fri Aug 25, 2017 9:43 am

Re: Use FLIR Lepton and Camera toghether on one module

Postby dcolemans » Sat Jan 26, 2019 10:02 am

Hi Kwabena,

Perhaps if I could know the exact place the Reset is failing, it would help me make a case for getting a replacement Lepton unit. The H7 is the only way I have to run it. So if you would be so kind as to make up a diagnostic firmware image I could move in that direction. Perhaps with longer timeout, just to rule out that possibility. Thanks again for your quick and generous support.
Dave
User avatar
kwagyeman
Posts: 3132
Joined: Sun May 24, 2015 2:10 pm

Re: Use FLIR Lepton and Camera toghether on one module

Postby kwagyeman » Sat Jan 26, 2019 3:50 pm

Here's the firmware with test outputs.
Attachments
firmware.zip
(925.72 KiB) Downloaded 89 times
Nyamekye,
User avatar
dcolemans
Posts: 31
Joined: Fri Aug 25, 2017 9:43 am

Re: Use FLIR Lepton and Camera toghether on one module

Postby dcolemans » Sun Jan 27, 2019 2:41 pm

Thanks Kwabena. The message is "FFC failure". Not sure what that means but I'll contact the vendor about a replacement. I assume there's no harm having this firmware in place for other applications on the H7.
Dave
User avatar
dcolemans
Posts: 31
Joined: Fri Aug 25, 2017 9:43 am

Re: Use FLIR Lepton and Camera toghether on one module

Postby dcolemans » Sun Jan 27, 2019 3:17 pm

I wrote to the vendor saying:
"Hi there. I'm trying to use the FLIR Lepton 1 in an OpenMV mounting and running a simple program that first tries to initialize the Lepton. This program fails with a message complaining of an "FFC failure". Indications are that I2C communication is working, but it is not "booting" as required. I would like you to replace this Lepton with another. Please advise.

I heard from the vendor and he says:
"Another module will do the same thing due to your setup. These are brand new and tested. Please see the following discussion here with some solutions. : https://groups.google.com/forum/m/#!top ... Sp9mA_BI28

Info on the referenced Lepton forum discussion is a bit out of my depth. So I'm not sure what more to do here.
Dave
User avatar
kwagyeman
Posts: 3132
Joined: Sun May 24, 2015 2:10 pm

Re: Use FLIR Lepton and Camera toghether on one module

Postby kwagyeman » Sun Jan 27, 2019 4:37 pm

Hmm, maybe you have a FLIR one older than mine. I can skip the FFC step. It's not really required in the FLIR 1 per say (it is kinda required - good practice).

Um, the error they mention there is because of their software package. We had to do a lot of work for the FLIR3 to get around the SPI bus issue. Basically, the FLIR dies unless you clock it's SPI bus fast enough. I haven't seen any RPi code that uses real time threads for the Lepton 3 so I would expect that when the RPi does random Linux stuff and missing timing deadlines this causes the module to crash.
Nyamekye,
User avatar
dcolemans
Posts: 31
Joined: Fri Aug 25, 2017 9:43 am

Re: Use FLIR Lepton and Camera toghether on one module

Postby dcolemans » Mon Jan 28, 2019 11:32 am

Hi Kwabena,

Again, thanks for your response. I'm so out of my depth here that I am willing to try anything you think of. It sounds like shooting in the dark, but let's try a version that skips the FFC step. If it then errors on another step, we can write it off.
Dave
User avatar
kwagyeman
Posts: 3132
Joined: Sun May 24, 2015 2:10 pm

Re: Use FLIR Lepton and Camera toghether on one module

Postby kwagyeman » Mon Feb 04, 2019 12:01 pm

Try this build.
Attachments
firmware.zip
(929.14 KiB) Downloaded 85 times
Nyamekye,
User avatar
dcolemans
Posts: 31
Joined: Fri Aug 25, 2017 9:43 am

Re: Use FLIR Lepton and Camera toghether on one module

Postby dcolemans » Tue Feb 05, 2019 1:16 pm

Lepton1.mp4
(30.14 KiB) Downloaded 113 times
Whoah! It works. Whatever you did, you did it. Thanks very much Kwabena. I love it when these toys work.
Last edited by dcolemans on Tue Feb 05, 2019 1:28 pm, edited 1 time in total.
Dave
User avatar
kwagyeman
Posts: 3132
Joined: Sun May 24, 2015 2:10 pm

Re: Use FLIR Lepton and Camera toghether on one module

Postby kwagyeman » Tue Feb 05, 2019 1:27 pm

Okay, I'll make that call optional. It's weird your Lepton fails on it.
Nyamekye,
User avatar
dcolemans
Posts: 31
Joined: Fri Aug 25, 2017 9:43 am

Re: Use FLIR Lepton and Camera toghether on one module

Postby dcolemans » Tue Feb 05, 2019 1:52 pm

Sorry if it's the wrong thread, but now I'd love to get ahold of a Singtown Video shield. I'm envisioning my drone with FPV thermal imaging. Any chance of getting one soon?
Dave
User avatar
kwagyeman
Posts: 3132
Joined: Sun May 24, 2015 2:10 pm

Re: Use FLIR Lepton and Camera toghether on one module

Postby kwagyeman » Tue Feb 05, 2019 6:47 pm

I have an extra one at my place. I just got the units from them. I asked for a small hardware change and then I think they will build more.

I need to test it out however. Will get to it this weekend.
Nyamekye,

Return to “General Topics”

Who is online

Users browsing this forum: No registered users and 8 guests