Pulsing Beacon Tracking on N6

OpenMV Boards
1

Hello!

Looking for some advice here. I’m trying to use the N6 board for a pan/tilt application to line up a gimbal to an infrared flashing beacon. I’ve set up the camera with a lens with an infrared bandpass filter. This flashing infrared beacon is not something that I control, so I have to work around whatever frequency it’s flashing at. By exclusively looking for light sources flashing at this frequency, I can ignore any other potential infrared sources and make sure that it is tracking the beacon I want it to track. The light flashes in a square wave pattern with a pattern at 16Hz with a low duty cycle, it is on for 5ms and off for 57.5ms. Because of the low duty cycle, I need to measure at least ~150fps.

I was wondering if you had any examples or advice for this kind of pulsing beacon tracking. I realize that the event camera is probably the best tool for this, but I would prefer to avoid it because of the cost and the M6 lens.

Thanks!

2

Hi Roberto!

I asked AI and it came up with something like this.

The idea: run the PAG7936 at QVGA/470 fps with short fixed exposure, find bright blobs every frame, associate them to persistent tracks, and lock onto the track whose rising edges land on the 62.5 ms grid. Constant IR sources never produce edges, and flashers at other frequencies miss the grid, so only the 16 Hz beacon survives.

# Pulsing IR beacon tracker for the OpenMV N6 (PAG7936).
#
# Locks onto a beacon flashing at 16 Hz (5 ms on / 57.5 ms off) seen
# through an IR bandpass filter, and rejects every other IR source:
# constant sources never blink, and sources blinking at other rates
# don't land on the 62.5 ms period grid.

import csi
import time

from ulab import numpy as np

BEACON_HZ = 16
PERIOD_US = 1000000 // BEACON_HZ  # 62500 us between rising edges
TOL_US = 4000        # how close an interval must be to the grid (~ +/-1 Hz)
MIN_OFF_US = 15000   # dark gap that separates two pulses (off time is 57.5 ms)
TRACK_TIMEOUT_US = 5 * PERIOD_US
MAX_EDGES = 8        # rising edges kept per track (~0.5 s of history)
MIN_EDGES = 4        # edges needed before a track can qualify
LOCK_SCORE = 0.75    # fraction of intervals that must land on the grid
GATE_PX = 40         # association radius; raise if the gimbal slews fast

EXPOSURE_US = 800    # must stay under the 2.1 ms frame time at 470 FPS
GAIN_DB = 24         # tune for beacon brightness at your range
THRESHOLD = (200, 255)

DRAW = True          # IDE overlay; set False for maximum frame rate

csi0 = csi.CSI()
csi0.reset()
csi0.pixformat(csi.GRAYSCALE)
csi0.framesize(csi.QVGA)
csi0.framerate(470)
csi0.snapshot(time=2000)  # Wait for settings to take effect.

# Auto gain/exposure hunt and change the blob brightness between frames,
# which breaks thresholding - fix both.
csi0.auto_gain(False, gain_db=GAIN_DB)
csi0.auto_exposure(False, exposure_us=EXPOSURE_US)
csi0.snapshot(time=500)


class Track:
    def __init__(self, x, y, t):
        self.x = x
        self.y = y
        self.last_seen = t
        self.edges = [t]
        self.score = 0.0

    def update(self, x, y, t):
        self.x = x
        self.y = y
        if time.ticks_diff(t, self.last_seen) > MIN_OFF_US:
            self.edges.append(t)  # was dark long enough -> new pulse
            if len(self.edges) > MAX_EDGES:
                self.edges.pop(0)
            self.score = self._score()
        self.last_seen = t

    def _score(self):
        if len(self.edges) < MIN_EDGES:
            return 0.0
        ints = np.array([time.ticks_diff(b, a)
                         for a, b in zip(self.edges, self.edges[1:])])
        # Snap each interval to the nearest multiple of the period. Using
        # the grid instead of requiring exactly one period keeps the lock
        # when a pulse is occasionally missed (dropped frame, occlusion).
        n = np.floor(ints / PERIOD_US + 0.5)
        r = ints - n * PERIOD_US
        on_grid = np.sum(r * r < TOL_US * TOL_US) / len(ints)
        # But a subharmonic (e.g. an 8 Hz flasher) also lands on the grid,
        # so most intervals must be exactly one period.
        r1 = ints - PERIOD_US
        if np.sum(r1 * r1 < TOL_US * TOL_US) < len(ints) / 2:
            return 0.0
        return on_grid


tracks = []
clock = time.clock()
last_print = time.ticks_ms()

while True:
    clock.tick()
    img = csi0.snapshot()
    t = time.ticks_us()

    blobs = img.find_blobs([THRESHOLD], pixels_threshold=2,
                           area_threshold=4, merge=True)

    for blob in blobs:
        bx, by = blob.cx, blob.cy
        best, best_d = None, GATE_PX * GATE_PX
        for tr in tracks:
            d = (tr.x - bx) ** 2 + (tr.y - by) ** 2
            if d < best_d:
                best, best_d = tr, d
        if best:
            best.update(bx, by, t)
        else:
            tracks.append(Track(bx, by, t))

    tracks = [tr for tr in tracks
              if time.ticks_diff(t, tr.last_seen) < TRACK_TIMEOUT_US]

    beacon = None
    for tr in tracks:
        if tr.score >= LOCK_SCORE and (beacon is None or tr.score > beacon.score):
            beacon = tr

    if beacon:
        # Normalized -1..1 error from the image center - feed these to
        # your pan/tilt PID loops (see scripts/libraries/pid.py).
        x_err = (beacon.x - img.width / 2) / (img.width / 2)
        y_err = (beacon.y - img.height / 2) / (img.height / 2)
        if DRAW:
            img.draw_cross(int(beacon.x), int(beacon.y), size=10, color=255)

    if time.ticks_diff(time.ticks_ms(), last_print) > 250:
        last_print = time.ticks_ms()
        if beacon:
            print("beacon (%d, %d) score %.2f fps %.1f"
                  % (beacon.x, beacon.y, beacon.score, clock.fps()))
        else:
            print("searching... %d tracks, fps %.1f" % (len(tracks), clock.fps()))
  • Exposure is the critical setting. At 470 fps the frame time is 2.1 ms, so exposure must be well under that — the script fixes it at 800 µs and makes up brightness with gain. With the bandpass filter the scene should be nearly black except for IR sources, which is exactly what you want for a cheap bright threshold.
  • Loop rate matters more than sensor rate. snapshot() returns the newest frame, so if the Python loop runs slower than the sensor, frames are silently dropped — and a 5 ms pulse only spans 2–3 frames. The loop must stay under ~5 ms per iteration to guarantee catching every pulse. That’s why the blob thresholds are tight, drawing is optional, and printing is rate-limited. Watch clock.fps(): streaming to the IDE roughly halves it, so judge the real rate disconnected.
  • Missed pulses don’t break the lock. The frequency test snaps each rising-edge interval to the nearest multiple of 62.5 ms rather than requiring exactly one period, so an occasionally dropped pulse (interval of 125 ms) still counts — while the one-period majority check stops an 8 Hz subharmonic from sneaking through.
  • Even at ~150–200 fps effective loop rate the scheme still works because the sensor and beacon are asynchronous — the sampling phase drifts across the pulse rather than parking in a blind spot — it just takes a few more periods to accumulate MIN_EDGES edges.
  • TOL_US, GATE_PX, and THRESHOLD are the knobs to tune in the field: tolerance for beacon frequency error, association radius for gimbal slew speed, and threshold for range/brightness.
3

I had to make a couple of edits to get it to run, but it works decently, albeit in need of some additional tuning. It can track the target at low camera rotation speeds, but will lose it at higher speeds. Pretty impressive for AI.

For future reference of anyone looking at this thread, this is the edited code that runs:

# Pulsing IR beacon tracker for the OpenMV N6 (PAG7936).
#
# Locks onto a beacon flashing at 16 Hz (5 ms on / 57.5 ms off) seen
# through an IR bandpass filter, and rejects every other IR source:
# constant sources never blink, and sources blinking at other rates
# don't land on the 62.5 ms period grid.

import csi
import time

from ulab import numpy as np

BEACON_HZ = 16
PERIOD_US = 1000000 // BEACON_HZ  # 62500 us between rising edges
TOL_US = 4000        # how close an interval must be to the grid (~ +/-1 Hz)
MIN_OFF_US = 15000   # dark gap that separates two pulses (off time is 57.5 ms)
TRACK_TIMEOUT_US = 5 * PERIOD_US
MAX_EDGES = 8        # rising edges kept per track (~0.5 s of history)
MIN_EDGES = 4        # edges needed before a track can qualify
LOCK_SCORE = 0.75    # fraction of intervals that must land on the grid
GATE_PX = 40         # association radius; raise if the gimbal slews fast

EXPOSURE_US = 800    # must stay under the 2.1 ms frame time at 470 FPS
GAIN_DB = 24         # tune for beacon brightness at your range
THRESHOLD = (200, 255)

DRAW = True          # IDE overlay; set False for maximum frame rate

csi0 = csi.CSI()
csi0.reset()
csi0.pixformat(csi.GRAYSCALE)
csi0.framesize(csi.QVGA)
csi0.framerate(470)
csi0.snapshot(time=2000)  # Wait for settings to take effect.

# Auto gain/exposure hunt and change the blob brightness between frames,
# which breaks thresholding - fix both.
csi0.auto_gain(False, gain_db=GAIN_DB)
csi0.auto_exposure(False, exposure_us=EXPOSURE_US)
csi0.snapshot(time=500)


class Track:
    def __init__(self, x, y, t):
        self.x = x
        self.y = y
        self.last_seen = t
        self.edges = [t]
        self.score = 0.0

    def update(self, x, y, t):
        self.x = x
        self.y = y
        if time.ticks_diff(t, self.last_seen) > MIN_OFF_US:
            self.edges.append(t)  # was dark long enough -> new pulse
            if len(self.edges) > MAX_EDGES:
                self.edges.pop(0)
            self.score = self._score()
        self.last_seen = t

    def _score(self):
        if len(self.edges) < MIN_EDGES:
            return 0.0
        ints = np.array([time.ticks_diff(b, a)
                         for a, b in zip(self.edges, self.edges[1:])])
        # Snap each interval to the nearest multiple of the period. Using
        # the grid instead of requiring exactly one period keeps the lock
        # when a pulse is occasionally missed (dropped frame, occlusion).
        n = np.floor(ints / PERIOD_US + 0.5)
        r = ints - n * PERIOD_US
        on_grid = np.sum(r * r < TOL_US * TOL_US) / len(ints)
        # But a subharmonic (e.g. an 8 Hz flasher) also lands on the grid,
        # so most intervals must be exactly one period.
        r1 = ints - PERIOD_US
        if np.sum(r1 * r1 < TOL_US * TOL_US) < len(ints) / 2:
            return 0.0
        return on_grid


tracks = []
clock = time.clock()
last_print = time.ticks_ms()

while True:
    clock.tick()
    img = csi0.snapshot()
    t = time.ticks_us()

    blobs = img.find_blobs([THRESHOLD], pixels_threshold=2,
                           area_threshold=4, merge=True)

    for blob in blobs:
        bx, by = blob.cx, blob.cy
        best, best_d = None, GATE_PX * GATE_PX
        for tr in tracks:
            d = (tr.x - bx) ** 2 + (tr.y - by) ** 2
            if d < best_d:
                best, best_d = tr, d
        if best:
            best.update(bx, by, t)
        else:
            tracks.append(Track(bx, by, t))

    tracks = [tr for tr in tracks
              if time.ticks_diff(t, tr.last_seen) < TRACK_TIMEOUT_US]

    beacon = None
    for tr in tracks:
        if tr.score >= LOCK_SCORE and (beacon is None or tr.score > beacon.score):
            beacon = tr

    if beacon:
        # Normalized -1..1 error from the image center - feed these to
        # your pan/tilt PID loops (see scripts/libraries/pid.py).
        w = img.width()
        h = img.height()
        x_err = (beacon.x - w / 2) / (w / 2)
        y_err = (beacon.y - h / 2) / (h / 2)

        if DRAW:
            img.draw_cross((int(beacon.x), int(beacon.y)), size=10, color=255)

    if time.ticks_diff(time.ticks_ms(), last_print) > 250:
        last_print = time.ticks_ms()
        if beacon:
            print("beacon (%d, %d) score %.2f fps %.1f"
                  % (beacon.x, beacon.y, beacon.score, clock.fps()))
        else:
            print("searching... %d tracks, fps %.1f" % (len(tracks), clock.fps()))

I get a framerate of 400fps in QVGA and 100FPS in VGA.

4

Nice! You’ll want to mess with find_blobs() settings and arguments to get more speed. It has some arguments that make it run faster.