Checking LCD segments with OpenMV H7

Project Discussion
1

Hello!
We use an OpenMV H7 camera to track LCD defects during production:

At the moment I am using Color tracking + blob to count all segments in the display. Unfortunately, I ran into a problem - different cameras have different images. Because There are many cameras, very long and difficult to calibrate and configure them. The segments on the displays are checked at predetermined positions, the camera is fixed rigidly:

    if test_number == '01':
        test = CheckResult(tests(1, [35 + cal_factor_x, 33 - cal_factor_y, 580, 180], long_delay, thresholds, 'merge_on'))
        if test == False:
            return

    elif test_number == '02' or test_number == '03' or test_number == '04':
    #2-4

        test = CheckResult(tests(1, [(21 + cal_factor_x), (119 - cal_factor_y), 25, 53], long_delay, thresholds_for_T, 'merge_off'))
        if test == False:
            return
        test = CheckResult(tests(2, [(50 + cal_factor_x), (121 - cal_factor_y), 38, 93], long_delay, thresholds, 'merge_off'))
        if test == False:
            return
        test = CheckResult(tests(3, [(101 + cal_factor_x), (121 - cal_factor_y), 42, 99], long_delay, thresholds, 'merge_off'))
        if test == False:
            return

Аnd if the next LCD is checked, the image may shift a little, so I entered the calibration coefficients for X and Y:

def calibrate(thres,roi):
    global cal_factor_x, cal_factor_y
    clock.tick()
    img = sensor.snapshot()
    blobs = img.find_blobs(thres, roi=roi, x_stride=5, y_stride=5, pixels_threshold=10, area_threshold=10, merge=False)
    img.draw_rectangle(roi)
    for blob in blobs:
        img.draw_rectangle(blob.rect(), thickness=3, color=(255,0,0))
        cal_factor_x = blob.x() - x_reference
        cal_factor_y = y_reference - blob.y() 
        print('Coordinates:', str(blob.x()), str(blob.y()))
        print('Factor X = ' + str(cal_factor_x))
        print('Factor Y = ' + str(cal_factor_y))
    if len(blobs) == 1:
         outpackage(send_pass)
    else:
         outpackage(send_fail)

But this is not enough if the camera is higher or lower, or another lens and its curvature.
Please tell me how can I get rid of the camera position dependence? How to automatically scale the image and not depend on the curvature of the lens / LCD range? Maybe сolor tracking is not suitable for this task?

And the black frame around the display falls into the thresholds, so I had to use specific areas, I can’t ignore it, maybe you can tell me how.
Thank you advance!

2

Hi, please use lens_corr() followed by roation_corr(). You can map the LCD screen to the frame buffer using these two methods. See the Examples->Filters->lens correction and rotation correction and perspective correction.

This limits calibration to just the lens correction strength which should not change much between cameras and then the rotation correction values which will have to be calibrated per camera to deal with small pixel shifts.

After that I would use binary to get a black and white high contrast image.

3

As for the binary image - I’ll try it, thanks.
The problem is I can’t use lens_corr (), there is not enough controller power (while doing the tasks I need). + I do it in VGA, and as far as I know, it is impossible to use lens_corr / rotation at this resolution. If I set the resolution lower, then I will not be able to track small objects on the screen.

4

Oh, that makes life hard. Um, the H7 Plus has enough RAM to do lens correction. Can you get your hands on one?

5

Only with your help. These cameras are available for pre-order only. But we would love to buy five for tests.

6

Please buy from SparkFun: OpenMV Cam H7 Plus - SEN-16989 - SparkFun Electronics