Calibrating camera for color tracking

I’m struggling to get consistant color detection results in different lighting conditions (a known problem in CV, of course). I see that the auto_gain and auto_whitebalance functions should not be used while color tracking, but is it useful to use them for a few frames before tracking, like this?


sensor.set_auto_gain(True)
sensor.set_auto_whitebal(True)

For i in range(0,3):
        img = sensor.snapshot() # Take a few pictures to calibrate the camera

sensor.set_auto_gain(False)   # now turn off autocalibration before we start color tracking
sensor.set_auto_whitebal(False)

Yes it will let the sensor’s algorithms run for a while, setting the gain, whitebal, exposure values, before you disable them. The values will remain in the sensor’s registers, but you need to skip more frames like 30 or 60 frames. Note there’s a function for that:

sensor.skip_frames(30)

If the lighting conditions change too much after disabling the functions the color detection results will change, not sure if there’s a fix for that, if you know the values you should use you can set them manually:

sensor.set_auto_gain(False, value=gain)
sensor.set_auto_whitebalance(False, value=(r_gain, g_gain, b_gain))
sensor.set_auto_exposure(False, value=exposure)

Also, make please try to relax and color tracking bounds. Use the color threshold editor to pick really relaxed bounds. In general, avoid touching the L bound if possible. Just pull in the A and B bounds (from the lo or high side). There’s somewhat a bit of trial and error to this.

Thanks both of you! Will try that tomorrow.

Hi,

I’ve been looking into this, and I found out there’s an advanced AWB mode. The simple AWB (default) adjusts the R,G,B gains until RGB averages are equal, so if there’s a big red object everything looks blueish:

The advanced AWB adjusts the R,G,B gains based on the light color temperature, however it needs specific optics settings set in undocument registers, but using the defaults it still looks way better:

To switch to advanced AWB use this:

sensor.__write_reg(0x6B, 0x22)

You have to leave AWB enabled of course, and try with auto gain enabled too. Give it a try and if it works better I’ll make a function for it or make the default.

Hi,
Could you please provide registers where: value=(r_gain, g_gain, b_gain) is written to?

sensor.set_auto_whitebalance(False, value=(r_gain, g_gain, b_gain))

I would like to enable auto whitebalance, then read gain values from register use them manually with disabled AWB, so I don’t have to calibrate camera on every power cycle.

Hi, if you look in the datasheet for the camera module they are written to registers 0, 1, and 2. They are 8 bit values.

Thanks for your quick answer, it helped.
I found out that:
0x00 - Auto gain
0x01 - Blue WB
0x02 - Red WB
0x03 - Green WB

sensor.set_auto_gain(False, value=99) # must be turned off for color tracking
sensor.set_auto_whitebal(False, value=(10,20,30)) # must be turned off for color tracking
sensor.skip_frames(time = 2000) # Let new settings take affect.
auto_gain_value = sensor.__read_reg(0x00)
auto_whitebal_value = (sensor.__read_reg(0x02), sensor.__read_reg(0x03), sensor.__read_reg(0x01))
print(auto_gain_value, auto_whitebal_value)

Perhaps: Once you are successfully tracking the color you want - you can detect “drift” in the color by evaluating the median value of blobs (averaged over a short time) - so that you may be able to maintain calibration over slow-ish lighting changes.