OpenMV

I need to install a stand alone system that detects bears using a Lepton camera (which will activate an electric airhorn) and will be powered by solar. I live deep in the bush off grid in Alaska and I have good solar and internet through a cell tower. A brown bear with six cubs just destroyed our yurt a couple months ago and I need something that will keep them away when we're off the property.

So I'm thinking the OpenMV Cam H7 Plus + FLIR® Lepton® Adapter Module along with either:

a. The Lepton camera from my PureThermal2 that I have (I think I can pop the camera off and put it on the Adapter Module)
b. Buying a new Lepton 3 camera if (a) won't work

I am assuming the H7 Plus when powered will run and do everything without needing to be hooked up to anything else. So when I deploy it, it would just be this board plugged into a power source and then I would USB plug it into my computer when I want to alter the code or view images and video collected?

Thanks very much for any help with my noob questions.

Yes, the H7 Plus can track thermal blobs from the Lepton video. It's pretty easy to make a reliable solution for what you want with the lepton. As for an internet connection, you can use the WiFi shield and have the camera send data to a wifi access point.

Thank you sir for the quick response. Would I need the WiFi sensor to send the data, or would the H7 run and save the data on the microSD that I could collect later manually?

I would send the data immediately. It's easier to deal with. Logging data at the edge is a lot more work.

That makes sense. The image detection happens on the H7 itself and should therefore recognize bears and blow the horn and take the video. We would hear the airhorn while there, but while away from the property we wouldn't be able to be notified unless we had it connected to the internet. I think I will get a WiFi shield and add that to the project.

Thanks again for the quick response. If this first one works well, I'll be buying many more of these to place around the property.

Yeah, you can just do a horn too only. That's like 1-bit per second wifi.

Okay, I have a couple H7 Pluses and a WiFi shield now (takes a bit to get stuff in Alaska).

My goal is to now get this thing to recognize a bear in a video feed. I think I want image detection rather than object detection (at least for now). My question is, should I:

1. Try to train a dataset on the regular H7 Plus camera using images of bears I've downloaded from online (and feed those into Edge Impulse) OR...

2. Should I install the Lepton adapter and Lepton camera (I have both) and instead try for blob size and temperature of the blob?

The reason I ask is because I am not sure if the H7 Plus can actually recognize bears generally, or if it will only recognize a certain bear or bears that I take pictures of with the H7 Plus. Can I train a model with enough pictures of bears from images on the internet to have it reliably see a bear in real life, or should I concentrate instead on the Lepton and then code it to recognize the size of a blob and a temperature instead (which might bite off on moose and people)?

Either approach works for me and ultimately I'd hope to get the Pure Thermal board when that comes out and use the logic for both regular and FLIR cameras to make my detection more accurate. But I do need to get something up with the H7 Plus before the bears wake up so I need to start with the H7 Plus although I can select either the regular or FLIR camera. If I can make it so it detects bears as opposed to people that would be great since this thing will have logic to use bear spray (not that any humans will be on our property other than us, we're way out here).

You should capture some thermal images of bears and then train a CNN on that. This will give you the best result.

I'd recommend to have the camera capturing snapshots. Every 10 seconds or so into multiple folders. Create a new folder each day and move new snapshots into those folders. Once you've captured enough images of bears then you can upload those pics to edge impulse to be able to find bears.

Note that the CNN will likely recognize anything that looks like a non-background image as a bear unless you add snaspshots of people and other things too.

Alternatively, don't train a CNN at all. Just use blob detection on the thermal image. This should work very well without much work.

As for image capture, I'd recommend sending the images wirelessly via the WiFi shield to the cloud or some server. SD cards for datalogging aren't the most reliable thing. Especially with the FAT File System.

kwagyeman wrote: ↑

Wed Jan 13, 2021 2:06 pm

You should capture some thermal images of bears and then train a CNN on that. This will give you the best result.

I'd recommend to have the camera capturing snapshots. Every 10 seconds or so into multiple folders. Create a new folder each day and move new snapshots into those folders. Once you've captured enough images of bears then you can upload those pics to edge impulse to be able to find bears.

Note that the CNN will likely recognize anything that looks like a non-background image as a bear unless you add snaspshots of people and other things too.

Alternatively, don't train a CNN at all. Just use blob detection on the thermal image. This should work very well without much work.

As for image capture, I'd recommend sending the images wirelessly via the WiFi shield to the cloud or some server. SD cards for datalogging aren't the most reliable thing. Especially with the FAT File System.

Thanks Kwagyeman.

If I understand you correctly, if I capture enough thermal images of bears and train a CNN on that, it should be able to detect other new bears that come later.

My thought on saving the files is similar to yours. When it detects a bear I'll have it save a file with "bear" plus date and time as the filename and put it into a folder created for that day. Ideally if I can get this thing to recognize moose and other wildlife in addition, I'd have it save the file with that object in the title.

For the training I'll add other photos of not bears that the camera is likely to see (birds, ermines, ATVs, us). Thanks for that tip.

Once I figure out the WiFi shield I'll definitely have it save the video files elsewhere.

Since I need to get something up soon (before I can take any images of bears) before the bears wake up and start trying to destroy our place again, I'll probably plan on doing the following using the Lepton:

1. Train a FLIR dataset of my wife and myself (I saw your person detector video but I'm assuming that was trained on just you)

And then have my script:

A. Detect a thermal blob
B. Ensure it's not me or the wife using my trained model
C. Save the images (for later use with the CNN) both on SD card and through WiFi
D. Verify the thermal temperature range is in the living mammal range
E. Activate the air horn if it meets those criteria
F. Somehow, if the same non-human blob persists and gets closer during X minutes of an air horn event, activate some bear spray (not sure how to measure range, perhaps blob size, perhaps I'll get a PIR sensor to add to the mix)

That should hopefully keep the bears away while providing imagery to train the dataset (although hopefully far images since they'll run away from the air horn) including bears and also other heat blobs that we regularly see that are not bears.