counting flowers

OpenMV related project discussion.
LeondeMooij
Posts: 18
Joined: Tue Aug 04, 2020 1:19 pm

counting flowers

Postby LeondeMooij » Tue Aug 04, 2020 1:32 pm

Hello,

I'm new with OpenMV and want to start a project to count flowers in plants.
We do this now with Python and OpenCV.
And we think it can be much better and faster with OpenMV.
Only thing is how can we start on the right point ?

Image

Maby Tensorflow is an option ?

With OpenCV i get this result.

Image
User avatar
kwagyeman
Posts: 4458
Joined: Sun May 24, 2015 2:10 pm

Re: counting flowers

Postby kwagyeman » Tue Aug 04, 2020 10:30 pm

How are you counting them with OpenCV currently? Find_Blobs() will do what you need for that image. A simple color threshold will find those blobs.
Nyamekye,
LeondeMooij
Posts: 18
Joined: Tue Aug 04, 2020 1:19 pm

Re: counting flowers

Postby LeondeMooij » Wed Aug 05, 2020 2:57 am

Is it possible to filter all the green out an then find colored blobs ?
Because a flower always change color only it is never green.
So if we take a green background than it is very stable ?

Any suggestions about this ?

How can i filter all the different tones of green out ?
LeondeMooij
Posts: 18
Joined: Tue Aug 04, 2020 1:19 pm

Re: counting flowers

Postby LeondeMooij » Wed Aug 05, 2020 3:38 am

This is a simple example in Open CV to find the flowers.
Its with sliders to get the right adjustments.

Code: Select all

import cv2
import json

def trackCheck(x):
    pass

img = cv2.imread("Test_15.jpg")

r, c, z = img.shape
img = cv2.resize(img, (int(c*0.5), int(r*0.5)))

cv2.namedWindow("Image")
cv2.createTrackbar("High-H", "Image", 0, 255, trackCheck)
cv2.createTrackbar("High-S", "Image", 0, 255, trackCheck)
cv2.createTrackbar("High-V", "Image", 0, 255, trackCheck)
cv2.createTrackbar("Low-H", "Image", 0, 255, trackCheck)
cv2.createTrackbar("Low-S", "Image", 0, 255, trackCheck)
cv2.createTrackbar("Low-V", "Image", 0, 255, trackCheck)
cv2.createTrackbar("Area", "Image", 10, 1000, trackCheck)
while True:
    hsvImg = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
    orig = img.copy()
    highH = cv2.getTrackbarPos("High-H", "Image")
    highS = cv2.getTrackbarPos("High-S", "Image")
    highV = cv2.getTrackbarPos("High-V", "Image")

    lowH = cv2.getTrackbarPos("Low-H", "Image")
    lowS = cv2.getTrackbarPos("Low-S", "Image")
    lowV = cv2.getTrackbarPos("Low-V", "Image")

    targetArea = cv2.getTrackbarPos("Area", "Image")

    low = (lowH, lowS, lowV)
    high = (highH, highS, highV)

    mask = cv2.inRange(hsvImg, low, high)
    mask = cv2.dilate(mask, None, iterations=1)
    mask = cv2.erode(mask, None, iterations=1)

    result = cv2.bitwise_and(img, img, mask=mask)
    cnts, hierarchy = cv2.findContours(mask, cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE)
    for cnt in cnts:
        area = cv2.contourArea(cnt)
        if area>targetArea:
            #cv2.drawContours(newEdged, cnt, -1, (0,255,0), 3, 8)
            x,y,w,h = cv2.boundingRect(cnt)
            newEdged = cv2.rectangle(orig,(x,y),(x+w,y+h),(0,0,255),1)
    cv2.imshow("Image1", result)
    cv2.imshow("Im", orig)
    k = cv2.waitKey(50)
    if k == 27:
        break
LeondeMooij
Posts: 18
Joined: Tue Aug 04, 2020 1:19 pm

Re: counting flowers

Postby LeondeMooij » Wed Aug 05, 2020 6:17 am

How can i count the blobs when i use keypoints?
Now i can detect the flowers true keypoints.
Only thing is how can i count the blobs ?

Image
Image

You can see the camera detects the flowers.
LeondeMooij
Posts: 18
Joined: Tue Aug 04, 2020 1:19 pm

Re: counting flowers

Postby LeondeMooij » Wed Aug 05, 2020 6:19 am

Here is the code.

Code: Select all

sensor.skip_frames(time = 2000)
sensor.set_auto_gain(False, value=100)

def draw_keypoints(img, kpts):
    if kpts:
        img.draw_keypoints(kpts)
        img = sensor.snapshot()
        time.sleep(10000)

kpts1 = None


clock = time.clock()
while (True):
    clock.tick()
    img = sensor.snapshot()
    kpts1 = img.find_keypoints(max_keypoints=500, threshold=10, scale_factor=1.2)
    draw_keypoints(img, kpts1)
    
User avatar
kwagyeman
Posts: 4458
Joined: Sun May 24, 2015 2:10 pm

Re: counting flowers

Postby kwagyeman » Wed Aug 05, 2020 11:13 am

Hi, keypoint objects are documented in the API as having a x and y position. So, you need a clustering algorithm to count the keypoint blobs.

E.g. use something like this to cluster the blobs and.get the count:

https://github.com/CihanBosnali/Machine ... -Libraries

...

Regarding the OpenCV code. We literally have the exact same tooling but better with find_blobs() with the OpenMV Cam.

Just open one of the color tracking scripts and use the Threshold Editor under Machine Vision -> Tools to edit the color thresholds for the image. We support erode and dilate operations too. Additionally, find_blobs() supports blob merging so it will give you the output you want more or less by clustering for you.
Nyamekye,
LeondeMooij
Posts: 18
Joined: Tue Aug 04, 2020 1:19 pm

Re: counting flowers

Postby LeondeMooij » Wed Aug 05, 2020 1:49 pm

Ok so finding blobs is the way to go !

Only question is what can we do when the color of the flowers changes ?
Can we make the treshold so it looks between 2 trehholds ?
Because we need every color of blobs exept the green.

And maby an example how to count the blobs, is there a standard instruction for ?
LeondeMooij
Posts: 18
Joined: Tue Aug 04, 2020 1:19 pm

Re: counting flowers

Postby LeondeMooij » Wed Aug 05, 2020 2:30 pm

Found this post from you:
Postby kwagyeman » Sat Sep 29, 2018 11:08 pm

Okay, start with the color tracking examples built into OpenMV IDE under File->Examples. You should also loo into the sensor control examples which let you control the sensor exposure and whatnot. These help greatly. Anyway, to do color tracking well you can do do this...

Assuming you 100% control the lighting of your garden (if it's indoors). Then you can do everything via color tracking. First, you just need to determine the color of the soil. You can use the Threshold Editor in OpenMV IDE to do this (see tools -> machine vision). Once you have the color of the soil you can find_blobs() with the invert flag set to True to find all colors that aren't the color of the soil. This will give you blobs that represent each plant. You can then pass options to find_blobs() to merge blobs and whatnot. Anyway, after doing this you can then proceed to look at the color within each blob using get_stats(). This will then return the color distribution for each blob. Note that you want to use a loop over the objects returned by find_blobs() and call get_stats() with the roi argument on each blob returned by find_blobs().

Anyway, for color tracking getting the lighting stable will be your enemy. Focus on that carefully.
Nyamekye,
This what we are looking for , maby you have a example of this functions?
User avatar
kwagyeman
Posts: 4458
Joined: Sun May 24, 2015 2:10 pm

Re: counting flowers

Postby kwagyeman » Wed Aug 05, 2020 3:03 pm

See the Examples->Color Tracking->Multicolor-tracking examples in the IDE.

Please see what find blobs does:

https://docs.openmv.io/library/omv.imag ... find_blobs

it has all the features you need in one function. Please try out the example and then modify the arguments to find_blobs() to do what you need to do.

You can get the color thresholds using the IDE by using the Tools->Machine Vision->Threshold Editor function.
Nyamekye,
LeondeMooij
Posts: 18
Joined: Tue Aug 04, 2020 1:19 pm

Re: counting flowers

Postby LeondeMooij » Thu Aug 06, 2020 8:28 am

I try almost everything and it won't work.
Maby somebody has an idea to solve this ?

Image

camera must count 3
User avatar
kwagyeman
Posts: 4458
Joined: Sun May 24, 2015 2:10 pm

Re: counting flowers

Postby kwagyeman » Thu Aug 06, 2020 2:14 pm

Hi, it's that a picture captured from the camera? I can quickly give you a script that does the job...
Nyamekye,
LeondeMooij
Posts: 18
Joined: Tue Aug 04, 2020 1:19 pm

Re: counting flowers

Postby LeondeMooij » Thu Aug 06, 2020 5:19 pm

Yes this is a picture from the cam.

If you can give me a start it wil be great.
User avatar
kwagyeman
Posts: 4458
Joined: Sun May 24, 2015 2:10 pm

Re: counting flowers

Postby kwagyeman » Thu Aug 06, 2020 11:23 pm

Code: Select all

# Hello World Example
#
# Welcome to the OpenMV IDE! Click on the green run arrow button below to run the script!

import sensor, image, time

FROM_FILE = True

sensor.reset()                      # Reset and initialize the sensor.
sensor.set_pixformat(sensor.RGB565) # Set pixel format to RGB565 (or GRAYSCALE)
sensor.set_framesize(sensor.QVGA)   # Set frame size to QVGA (320x240)
#sensor.skip_frames(time = 2000)     # Wait for settings take effect.
clock = time.clock()                # Create a clock object to track the FPS.

while(True):

    img = None

    if FROM_FILE:
        img = image.Image("test.bmp", copy_to_fb=True)
    else:
        img = sensor.snapshot()

    img.binary([(0, 60, 0, 127, 0, 127)])
    img.dilate(7, threshold=7)
    img.erode(2, threshold=5)

    blobs = img.find_blobs([(90, 100)], merge=True, margin=10)
    
    for b in blobs:
        img.draw_rectangle(b.rect(), color=(255, 0, 0))
    
    print(len(blobs))
Please use the feature in the product versus giving up easily. Copy the file to the disk as a bmp format image and run the above script.
Attachments
test.jpg
test.jpg (6.16 KiB) Viewed 393 times
Nyamekye,
LeondeMooij
Posts: 18
Joined: Tue Aug 04, 2020 1:19 pm

Re: counting flowers

Postby LeondeMooij » Fri Aug 07, 2020 3:36 am

You are great !

It works , sorry for all the questions only it is a bit different when you are normaly using opencv.

Whats the best option to communicate to a PLC?
I need the number of blobs and want to trigger the cam.

We are using modbus ip only there is no ethernet port on the cam.
So maby modbus trough the uart port and convert it to rs485 with this : https://www.benselectronics.nl/uart-ttl ... erter.html
Or is the Can shield the way to go ?

Leon
User avatar
kwagyeman
Posts: 4458
Joined: Sun May 24, 2015 2:10 pm

Re: counting flowers

Postby kwagyeman » Fri Aug 07, 2020 1:31 pm

Yeah, that ttl adapter will work.

Then use the PYB module to control the uart. It's very simple to send things.
Nyamekye,
LeondeMooij
Posts: 18
Joined: Tue Aug 04, 2020 1:19 pm

Re: counting flowers

Postby LeondeMooij » Tue Aug 18, 2020 4:25 am

We are close to get a result.
Only thing is we want to trigger the cam and make just 1 picture.

Modbus works very well.
Only we get a runtime error.

Code: Select all

import sensor, image, time
from pyb import UART
from modbus import ModbusRTU
uart = UART(3,115200, parity=None, stop=2, timeout=1, timeout_char=4)
modbus = ModbusRTU(uart, register_num=9999, slave_id=0x03)


FROM_FILE = True

sensor.reset()                      # Reset and initialize the sensor.
sensor.set_pixformat(sensor.RGB565) # Set pixel format to RGB565 (or GRAYSCALE)
sensor.set_framesize(sensor.QVGA)   # Set frame size to QVGA (320x240)
sensor.skip_frames(time = 2000)     # Wait for settings take effect.
sensor.ioctl(sensor.IOCTL_SET_TRIGGERED_MODE, True)
#clock = time.clock()                # Create a clock object to track the FPS.





while(True):
    if (modbus.REGISTER [0] == 1):
    
       img = sensor.snapshot()

       img.binary([(0, 100, -2, 127, 6, 127)])
       img.dilate(7, threshold=10)
       img.erode(2, threshold=5)

       blobs = img.find_blobs([(90, 100)], merge=True, margin=-10)

       for b in blobs:
           img.draw_rectangle(b.rect(), color=(255, 0, 0))
           modbus.REGISTER[1] = (len(blobs))
           modbus.REGISTER[0] = 0


       if modbus.any():
           modbus.handle(debug=True)

       print(len(blobs))
       print (modbus.REGISTER[0])
       print (modbus.REGISTER[1])
       

What's wrong ?

Here is the error:

Traceback (most recent call last):
File "<stdin>", line 14, in <module>
RuntimeError: Capture Failed: -4
MicroPython v1.12-omv OpenMV v3.6.7 2020-07-20; OPENMV4P-STM32H743
Type "help()" for more information.
User avatar
kwagyeman
Posts: 4458
Joined: Sun May 24, 2015 2:10 pm

Re: counting flowers

Postby kwagyeman » Tue Aug 18, 2020 12:00 pm

-4 is a sensor timeout. Please make sure your camera module is seated correctly.
Nyamekye,
LeondeMooij
Posts: 18
Joined: Tue Aug 04, 2020 1:19 pm

Re: counting flowers

Postby LeondeMooij » Tue Aug 18, 2020 3:53 pm

This line making trouble:

sensor.ioctl(sensor.IOCTL_SET_TRIGGERED_MODE, True)

How can i make the trigger mode work correct ?
User avatar
kwagyeman
Posts: 4458
Joined: Sun May 24, 2015 2:10 pm

Re: counting flowers

Postby kwagyeman » Tue Aug 18, 2020 4:44 pm

The OV7725 doesn't support triggered mode. So, remove that line.
Nyamekye,
LeondeMooij
Posts: 18
Joined: Tue Aug 04, 2020 1:19 pm

Re: counting flowers

Postby LeondeMooij » Wed Aug 19, 2020 2:38 pm

Ok, so just make a small movie and then return the amount of flowers.

There are 3 parameters in this line :

blobs = img.find_blobs([(90, 100)], merge=True, margin=-10)

Can you explain the 90 and 100 ?
Because how i read it it is area and pixels.
So when i want to detect only blobs that are bigger than 300 pixels i do this ?

blobs = img.find_blobs([(90, 300)], merge=True, margin=-10)

Only it doesn't work.
User avatar
kwagyeman
Posts: 4458
Joined: Sun May 24, 2015 2:10 pm

Re: counting flowers

Postby kwagyeman » Wed Aug 19, 2020 3:01 pm

90 and100 are the grayscale color thresholds applied to an RGb565 image. So, they are LAB thresholds on the L channel of the image which ranges from 0 to 100. Since the image is binarized only pixels of 0 and 100 in the LAB color space are in it.

You need to add a new parameter area= and pixels= to find_blobs().
Nyamekye,
LeondeMooij
Posts: 18
Joined: Tue Aug 04, 2020 1:19 pm

Re: counting flowers

Postby LeondeMooij » Wed Aug 19, 2020 3:44 pm

Ok that's make sense.

Only what do we here ?

img.binary([(0, 100, -2, 127, 6, 127)])

I'm was thinking these are the coler tresholds to detect the blobs ??

So now we are detecting blobs in greyscale, so is the global shutter cam better for this aplication ?
User avatar
kwagyeman
Posts: 4458
Joined: Sun May 24, 2015 2:10 pm

Re: counting flowers

Postby kwagyeman » Wed Aug 19, 2020 3:59 pm

I'm binarizing the image first, then apply the erode and dilate filters, and then finding the blobs.

This is two threshold steps. The second step isn't really needed... but, find_blobs() always thresholds things so I'm just passing it a list of thresholds that select an already thresholded image.
Nyamekye,
LeondeMooij
Posts: 18
Joined: Tue Aug 04, 2020 1:19 pm

Re: counting flowers

Postby LeondeMooij » Wed Aug 19, 2020 4:33 pm

Ok, thank you for this short course openmv.

Just one simple question.
We checking plants on a conveyor belt so we trigger the cam with a plc trough modbus.
This works ok when we are connected with the pc software.
Only when we disconnected the pc software the cam reaction is different.
How is this possible ?
Is this a speed change ?
Can we give the cam a fix FPS ?
User avatar
kwagyeman
Posts: 4458
Joined: Sun May 24, 2015 2:10 pm

Re: counting flowers

Postby kwagyeman » Wed Aug 19, 2020 4:51 pm

When connected to the PC the camera has the load of streaming images to the PC. When not connected it's not loaded anymore.

You can give a fixed FPS by rate limiting the script using the pyb module:

Code: Select all

import pyb

start_ms = pyb.millis()

while True:
    if (pyb.mills() - start) < 50: pyb.delay(1)
    start += 50
This above will limit you to 1000/50 = 20 Hz
Nyamekye,
LeondeMooij
Posts: 18
Joined: Tue Aug 04, 2020 1:19 pm

Re: counting flowers

Postby LeondeMooij » Thu Aug 20, 2020 2:39 pm

I think there are some mistakes in this lines ?

import pyb

start_ms = pyb.millis()

while True:
if (pyb.mills() - start) < 50: pyb.delay(1)
start += 50

This give errors.
When i do this:

import pyb

start = pyb.millis()

while True:
if (pyb.millis() - start) < 50: pyb.delay(1)
start += 50

There are no errors only there is nothing happened.
User avatar
kwagyeman
Posts: 4458
Joined: Sun May 24, 2015 2:10 pm

Re: counting flowers

Postby kwagyeman » Thu Aug 20, 2020 3:07 pm

Yeah, I miss named the variable. But, do you see the idea of what the code is doing?
Nyamekye,
LeondeMooij
Posts: 18
Joined: Tue Aug 04, 2020 1:19 pm

Re: counting flowers

Postby LeondeMooij » Thu Aug 20, 2020 3:34 pm

Yess i see where you want to go , only it doesn't do anything.

Code: Select all

import sensor, image, time
from pyb import UART
import pyb
from modbus import ModbusRTU
uart = UART(3,115200, parity=None, stop=2, timeout=1, timeout_char=4)
modbus = ModbusRTU(uart, register_num=9999, slave_id=0x03)





sensor.reset()                      # Reset and initialize the sensor.
sensor.set_pixformat(sensor.RGB565) # Set pixel format to RGB565 (or GRAYSCALE)
sensor.set_framesize(sensor.QVGA)   # Set frame size to QVGA (320x240)
sensor.skip_frames(time = 2000)     # Wait for settings take effect.
clock = time.clock()                # Create a clock object to track the FPS.


start = pyb.millis()




while(True):

    clock.tick()
    if (pyb.millis() - start) < 50: pyb.delay(1)
    start += 50
    
    if modbus.any():
        modbus.handle()#debug=True)
    if (modbus.REGISTER [0] == 1):


       img = sensor.snapshot()
       modbus.REGISTER[0] = 0
       img.binary([(50, 100, -10, 127, 20, 127)])
       img.dilate(7, threshold=19)
       img.erode(2, threshold=19)

       blobs = img.find_blobs([(90, 100)],pixels_threshold=180 , merge=False)

       for b in blobs:
           img.draw_rectangle(b.rect(), color=(255, 0, 0))
           modbus.REGISTER[1] = (len(blobs))
           print(len(blobs))
           #print (modbus.REGISTER[0])
           #print (modbus.REGISTER[1])
           print(clock.fps())
User avatar
kwagyeman
Posts: 4458
Joined: Sun May 24, 2015 2:10 pm

Re: counting flowers

Postby kwagyeman » Thu Aug 20, 2020 4:48 pm

Code: Select all

import pyb, time

start_ms = pyb.millis()

clock = time.clock()
while True:
    clock.tick()
    while pyb.elapsed_millis(start_ms) < 50: pyb.delay(1)
    start_ms += 50
    print(clock.fps())
if should have been while.
Nyamekye,
LeondeMooij
Posts: 18
Joined: Tue Aug 04, 2020 1:19 pm

Re: counting flowers

Postby LeondeMooij » Thu Aug 20, 2020 5:13 pm

That's looking more understanding. :D

Thank you alot.

Return to “Project Discussion”

Who is online

Users browsing this forum: No registered users and 3 guests