Weed Seeker Sprayer Project Proposal

Hi everyone,

Background (can skip)
So I’m a farmer over here in Australia and we grow a heap of crops. We grow them through the winter, then through the summer, we have fallow (bare) paddocks with only left over wheat or canola stubble in them. We spray these paddocks for weeds in an effort to conserve moisture for the following crops as we are in a low rainfall area, and every little bit of moisture helps. Currently we blanket spray with herbicides, spraying every inch of the paddock trying to kill the growing weeds that suck moisture and nutrients from the soil. But there is a better way.

On to the proposed project
What I want to do it find the green weeds in a paddock, and spray them individually using an OpenMV camera, coupled with an optocoupler mosfet or relay and a pressurised solenoid with spray nozzle.

Here are some details of things:

  1. Travel around 14-20km/h or 4-6m/s with the sprayer
  2. Monitor about a 1m wide patch of ground
  3. Have that 1m patch split into 2 sectors (left and right) that then triggers a left or right sprayer
  4. Detect green or greenish weeds on a fairly uniform straw or dirt background
  5. The weeds don’t need to be tracked, but I think just detecting of green is present in the frame (and sector) will be enough
  6. There may be a few ranges, or broad range of greens to find
  7. Trigger the 12v solenoid via a relay (PWM, MOSFET or regular 3.3 triggered)
  8. Chemical then flows through the solenoid, and spray nozzle, spraying the weed
  9. The plan is to have one every meter over a 36 meter boom so 36 camera and solenoid modules
  10. Each module will have its own enclosure
  11. Ultimately (if possible) I would like to also have each one linked somehow to a monitor in the cab of the tractor/sprayer displaying the status of each one
  12. Everything will be open source, including plans for a modular spray trailer, wiring and fluid diagrams and anything else I come across
  13. Ultimately the ultimate will be to have this system on a fully autonomous vehicle using a Pixhawk autopilot and ROS operating system

Questions

  1. Do you think the OpenMV is up to the task if the right artificial lighting is given?
  2. Will it be fast enough to detect and trigger the solenoid at a reasonable speed?
  3. Can Near InfraRed (NIR) filters be added to the camera if needed?
  4. Can anyone see any major problems or hurdles stopping this from working?
  5. Am I better off with openCV, and something like a RPi, ODroid or TX2?
  6. Can you think of any ways to optimise the detection system for weeds

I did have a working program that used openCV and python on my laptop that could detect green within a given range and threshold. I was going to port it onto a Raspberry Pi or something similar. But I thought the OpenMV seems like a better and more elegant solution to the problem. I have a camera turning up on Monday, and the rest of the parts are on the way so I can get started pretty soon.

There are other similar commercial systems out there, such as the WeediT system which I believe used a chlorophyll sensor and works quite well. For a 36m wide system we are looking at around $360,000(AUD), so they are a bit cost prohibitive for your average farmer. They do however claim to use only 10% of the chemical that you would otherwise use if you blanket spray an area. When we spend well over $100,000 on herbicides each year, the savings can be significant.

I plan on making everything open source, and freely available. I believe this would benefit many farmers and the environment, saving thousands of litres of chemical and therefore dollars. Where there is a risk of chemical run off, it would reduce the chemical in the system too as well as spray drift.

If you made it through all that I hope it makes sense, if anyone has any questions or needs me to clarify things, shoot away.

Thanks
Adam

Hey, thanks for the nice write up. I’d love if everyone wrote as much as you so its easy to understand what you need help with:

  1. Yes, the OpenMV Cam can do this. Other systems like the Pi can do this also. In general, you’re looking to do color tracking which is easy. That said, the OpenMV Cam color tracking code is more or less better than openCV’s support for this. You’ll find color tracking a lot easier to get working on the OpenMV Cam.
  2. Yes, you can turn I/O pins off an on in your code on the OpenMV Cam . This can trigger the solenoid. This happens at the FPS, so, if you’re getting 30 FPS color tracking with QVGA (320x240) then that’s at 30 times a second.
  3. You can switch the camera lens and add NIR LEDs so it only responds to seeing planets. That’s smart.
  4. Color tracking is rather finicky, you’ll need to learn multiple green color thresholds and track for all of them. The OpenMV Cam can do that easily. That said, being able to execute any project in general depends on your skill.
  5. If you need raw power for a higher res. You won’t beat the OpenMV Cam’s FPS though.
  6. Use the NIR system to make only green objects stand out. This gets way easier then.

Thanks Nyamekye,
Sounds like its doable, I guess it all comes down to me, and/or anyone else that helps out.

Do you think that LAB is the best colorspace for this? I was using HSV in openCV, but that was mainly because I modified and butchered a few other color tracking examples that used it. I guess under IR or NIR filters and light the green weeds might pop enough to use grayscale if I need to up the frame rate too.

Do you have any insight into monitoring multiple cameras on a remote in cab monitor? It doesn’t need to be wireless.

It says the i/o pins are 5v tolerant but am I better of using 3.3v or lower to trigger the relay? I thought this might be a bit easier on the little fella.

I have been collecting photos of weeds in the paddocks, so I might build a bit of a database with typical values to use in threshold values.

Thanks again,
Adam

LAB and HSV are the same for this type of thing. You won’t notice much difference.

As for multiple cameras, all OpenMV Cams have standard I/O peripherals. I would have them communicate via serial.

As for the I/O pins, we output 3.3v but are 5V tolerant meaning we can handle 5v being applied to the pins.

As for thresholds, yes, the best method is to collect multiple readings over the day and then try to search for them all. Our find_blobs method can handle 16 color thresholds at once.

So I’ve finally got all the bits and pieces I needed to get prototyping. I’ve played around with the camera a bit, but I have hit a bit of a hurdle. Now I dont claim to be a programmer of any sort, and im not too handy with electronics either, so this may all prove to be something very basic or easy to fix, but here’s the problem:

I can’t get the relay to stay closed (i.e. send 3.3v, or drop to 0v constantly on a pin) while there are blobs detected by the camera. What I am trying to do is keep the relay closed while there are blobs of my particular threshold in view of the camera. So I need to supply the 3.3v (high) when blobs = True, and go low (0v) when False. At the moment, when blobs are detected, I get constant switching on/off. Here’s one piece of code that I have tried, this one is the most basic but the outcomes were all the same:

I initialise the pin via:

solenoid = Pin('P1', Pin.OUT_PP, Pin.PULL_DOWN)

Then I run through the frames like this:

while(True):
    clock.tick()
    img = sensor.snapshot()
    solenoid.value(True) # This single relay module is off when High
    for blob in img.find_blobs([thresholds[threshold_index]], pixels_threshold=200, area_threshold=200, merge=True):
        img.draw_rectangle(blob.rect())
        img.draw_cross(blob.cx(), blob.cy())
        solenoid.value(False)

I have added a small video to show the outcome.

Here is what I have tried, but it all ended up with the same result:

  • if else loops along the lines of: if blobs, pin.value(True), else pin.value(False)
  • I also moved the solenoid.value(True) around to where I thought it should work, but it wouldnt
  • while loop: while blobs, pin.value(True)
  • I thought it might have had to do with pin setup, so I tried pull up down and none
  • I just did a basic program to set pin.value(True), and it stays on constantly. I also used the GPIO example on github, with the switch and it stays latched on.

I think the problem is in the code somewhere, but I cant figure out what I’m doing wrong.

These are my questions:

  • Is the problem in my code? That’s my guess but I cant figure it out. I’m sure its something basic
  • Is the problem to do with the pin setup

On a better note, I have some LAB thresholds that seem to work nicely, picking just the weeds out from a heap of sample images I took with my phone. This should give me a rough starting point, but the camera may look a bit different. I will just have to ground truth it with the OpenMV camera when i get it out into the paddock.
openmvrelay.mp4 (887 KB)

Hi, the problem is that you are clearing the state of the pin each loop and then setting it again. Do this:

solenoid.value(True) # This single relay module is off when High
while(True):
    clock.tick()
    img = sensor.snapshot()
    blobs = img.find_blobs([thresholds[threshold_index]], pixels_threshold=200, area_threshold=200, merge=True)
    for blob in blobs:
        img.draw_rectangle(blob.rect())
        img.draw_cross(blob.cx(), blob.cy())
    Solenoid.value(1 if len(blobs) else 0)

Notice how I set the state of the pin once at the top and then only change the state once per loop.

Ah ha!! Thanks kwagyeman, that’s a big help. I will give it a go tomorrow, its getting a bit late over here in Australia.

Thanks again
Adam

Hi

Just wondering how you got on with your weed seeker project?
I am interested in doing something simliar.

1 Like