Frame differencing provides false changes when using on disk reference image

Hey.

I’m facing a difficult problem.
I use basic frame differencing to detect foreign objects.
Lights are controlled and powerful enough.
Camera auto controls are turned off.

When I run in memory frame differencing, everything works perfectly.
But as soon i use on disk frame differencing, problems will arise: after taking reference image and starting frame differencing, system will work as excpected but when time goes, false detections will occur ant differencing number goes up.

I’m using OpenMV H7 Plus camera, resolution is VGA (640x480px) monochrome.

My first guess is that image object is compressed in memory, and uncompressing it will produce different image. Also, when taking a reference image, storing it on disk, and testing it later, is always results differences between reference and actual images even when there is no differences.

It only happens using on disk frame differencing, but i can’t use in memory mode because i need to turn off system daily and always taking a new reference image is no go.

Please store images as BMP files which are lossless and not JPEG files.

Sorry, i forgot to tell that im using bitmap file.
I made some tests and found that error is smaller when using QVGA (320x240px) frame size.
But like I already mentioned, there will be increasing frame difference number.
I modified the code to perform it using extra framebuffer and then result is better.

import sys, sensor, image, time, os, pyb, time, micropython
from pyb import Pin, SPI


sensor.reset() # Initialize the camera sensor.
sensor.set_pixformat(sensor.GRAYSCALE) # or sensor.GRAYSCALE
sensor.set_framesize(sensor.QVGA) # or sensor.QQVGA (or others)
sensor.set_auto_whitebal(False) # Turn off white balance.
sensor.set_auto_exposure(False)
sensor.set_auto_gain(False)
sensor.skip_frames(time = 3000) # Let new settings take affect.
clock = time.clock() # Tracks FPS.

if not "frame" in os.listdir(): os.mkdir("frame") # Make a frame directory

#sensor.skip_frames(time = 3000) # Give the user time to get ready.

if not "reference.bmp" in os.listdir("/frame"):
    img = sensor.snapshot()
    img.laplacian(1, sharpen=True)
    img.gaussian(2)
    img.save("frame/reference.bmp")
    sensor.snapshot().save
    print("reference image saved")
    
# load reference image
refimage = image.Image("frame/reference.bmp")

extra_fb = sensor.alloc_extra_fb(sensor.width(), sensor.height(), sensor.GRAYSCALE)
extra_fb.replace(refimage)

while(True):


    clock.tick() # Track elapsed milliseconds between snapshots().
    i = 0
    diff = 0
    diffarray = bytearray(10)
    while i < 10:
        sensor.skip_frames(time = 100)
        img = sensor.snapshot() # Take a picture and return the image.
        img.laplacian(1, sharpen=True)
        img.gaussian(2)

        # Replace the image with the "abs(NEW-OLD)" frame difference.
        img.difference(extra_fb)

        hist = img.get_histogram()
    # This code below works by comparing the 99th percentile value (e.g. the
    # non-outlier max value against the 90th percentile value (e.g. a non-max
    # value. The difference between the two values will grow as the difference
    # image seems more pixels change.

        diffarray[i] = hist.get_percentile(0.99).l_value() - hist.get_percentile(0.90).l_value()
        i +=1

    print(diffarray)

Result:

bytearray(b’\x01\x02\x02\x01\x02\x02\x01\x01\x02\x01’)
bytearray(b’\x01\x02\x02\x01\x02\x02\x01\x01\x02\x01’)
bytearray(b’\x01\x02\x02\x01\x02\x02\x01\x01\x02\x02’)
bytearray(b’\x01\x02\x02\x01\x01\x02\x01\x01\x02\x02’)
bytearray(b’\x01\x02\x02\x01\x01\x02\x01\x01\x02\x02’)
bytearray(b’\x01\x02\x02\x01\x01\x02\x01\x01\x02\x02’)
bytearray(b’\x01\x02\x02\x01\x01\x02\x01\x01\x02\x02’)
bytearray(b’\x01\x02\x02\x01\x02\x02\x01\x01\x02\x01’)
bytearray(b’\x01\x02\x02\x01\x02\x02\x01\x01\x02\x01’)
bytearray(b’\x01\x02\x02\x01\x02\x02\x01\x01\x02\x01’)
bytearray(b’\x01\x02\x02\x01\x02\x02\x01\x01\x02\x01’)
bytearray(b’\x01\x02\x02\x01\x02\x02\x01\x01\x02\x01’)

As you see, there is some differences, probably caused by sensor noise.

But when using this code, error will be increasing constantly: like +1 in 30 seconds

import sys, sensor, image, time, os, pyb, time, micropython
from pyb import Pin, SPI


sensor.reset() # Initialize the camera sensor.
sensor.set_pixformat(sensor.GRAYSCALE) # or sensor.GRAYSCALE
sensor.set_framesize(sensor.VGA) # or sensor.QQVGA (or others)
sensor.set_auto_whitebal(False) # Turn off white balance.
sensor.set_auto_exposure(False)
sensor.set_auto_gain(False)
sensor.skip_frames(time = 3000) # Let new settings take affect.
clock = time.clock() # Tracks FPS.

if not "frame" in os.listdir(): os.mkdir("frame") # Make a frame directory

#sensor.skip_frames(time = 3000) # Give the user time to get ready.

if not "reference.bmp" in os.listdir("/frame"):
    img = sensor.snapshot()
    img.laplacian(1, sharpen=True)
    img.gaussian(2)
    img.save("frame/reference.bmp")
    sensor.snapshot().save
    print("reference image saved")
    
# load reference image
#refimage = image.Image("frame/reference.bmp")

#extra_fb = sensor.alloc_extra_fb(sensor.width(), sensor.height(), sensor.GRAYSCALE)
#extra_fb.replace(refimage)

while(True):


    clock.tick() # Track elapsed milliseconds between snapshots().
    i = 0
    diff = 0
    diffarray = bytearray(10)
    while i < 10:
        sensor.skip_frames(time = 100)
        img = sensor.snapshot() # Take a picture and return the image.
        img.laplacian(1, sharpen=True)
        img.gaussian(2)

        # Replace the image with the "abs(NEW-OLD)" frame difference.
        img.difference("frame/reference.bmp") #extra_fb

        hist = img.get_histogram()
    # This code below works by comparing the 99th percentile value (e.g. the
    # non-outlier max value against the 90th percentile value (e.g. a non-max
    # value. The difference between the two values will grow as the difference
    # image seems more pixels change.

        diffarray[i] = hist.get_percentile(0.99).l_value() - hist.get_percentile(0.90).l_value()
        i +=1

    print(diffarray)

I have lot of OpenMV modules, at the moment i’ve tested two of them, and both will produce similar output so its not a single problem caused by faulty hardware.

Hmm… it’s probably related to disk i/o traffic causing increased power usage and heat.

Do this. Use the Image() constructor to alloc a new fb on the frame buffer stack with the image from disk. This way you read in the image once versus every call.

This should fix your issue.

I had few days busy with other things, started today and encountered a strang memory error:
MemoryError: memory allocation failed, allocating 307200 bytes

It happen when code loads reference image, i stripped code down to few lines and even now it throw exception

import sys, sensor, image, time, os, pyb, time, micropython


micropython.alloc_emergency_exception_buf(100)



refimg = image.Image("frame/reference.bmp")


del refimg
gc.collect()

It’s because you aren’t using the frame buffer but the heap.

add “load_to_fb=True” to your Image call.