Hi Suhail!
Thanks for the detailed post! We’ve been working hard on our documentation for the OpenMV Cam. Thanks to the increased quality, you can now ask Claude, for example, to write most of the application for you. Here’s a response from it for your project:
This is a response from Claude (Anthropic’s AI assistant), put together from the OpenMV docs.
This is a great first project, and the good news is that OpenMV is well-suited to it. The whole pipeline (color tracking → trajectory math → I/O output) runs on-device in MicroPython, so you don’t need a laptop or Pi in the loop. Here’s how I’d approach it.
The big architectural reality check first
Your hard constraint is reaction time: a ball traveling ~1–1.5 m, and you want a decision in ~0.1 s. That means your camera loop needs to comfortably hit a high frame rate, because “3–4 frames” of reaction budget only works if those 3–4 frames happen in well under 100 ms. The single biggest lever you have here is resolution and pixel format. Track the ball at QVGA (320×240) or even QQVGA in RGB565, disable auto-gain and auto-white-balance after warmup, and you can realistically get well over 100 fps for simple blob tracking. Don’t track at VGA — you don’t need the pixels, and they’ll cost you the reaction time you can’t afford.
Detecting the orange ball
For a single brightly colored ball, image.find_blobs() with a tuned LAB color threshold is the right tool — not the neural network stuff. It’s fast and deterministic. The single most important step for reliability is tuning your threshold under your actual table lighting using the Threshold Editor in OpenMV IDE (Tools menu). Don’t guess at the LAB values — point the camera at your table, grab a frame, and let the IDE generate them. And do lock auto_gain(False) / auto_whitebal(False), otherwise the camera will keep “correcting” the orange and your threshold will drift out from under you.
Trajectory detection (not just color)
You’re right that single-frame color detection isn’t enough — you need a heading. The standard approach: keep a short ring buffer of recent ball centroids, only treat the ball as “incoming” once it’s moving toward the goal, fit a straight line through the recent points, and extrapolate to your goal-line y to get the predicted crossing x. Then map that x to left/center/right by comparing against two boundary values you calibrate once. Linear extrapolation is fine here — over a flat 1.5 m foosball shot, curvature is negligible, so you don’t need any physics/gravity model.
Talking to the ESP32 — your “three I/O ports”
This maps perfectly onto machine.Pin. Configure three pins as outputs and drive the one matching the predicted zone HIGH (others LOW); the ESP32 just reads three GPIOs. Your spec (“the left port must trigger and must not trigger the others”) is just exclusivity — only one pin high at a time. A few notes:
-
The exact pin names depend on which OpenMV board you buy — check the Boards / pinout page for your model before wiring.
-
Watch your logic levels. Most OpenMV boards and the ESP32 are both 3.3 V, so you can usually wire directly — but confirm both sides are 3.3 V and share a common ground.
-
Add a confirmation count (require the predicted zone to agree across 2 frames) so a single noisy frame doesn’t twitch the goalie. With your frame budget you can afford a 2-frame confirm and still respond in time.
-
Alternatively, machine.UART lets you send richer data (predicted x, velocity) over two wires. Three GPIOs is the simplest thing that meets your requirement, so I’d start there and move to UART only if you want more than “which zone.”
Which board to get
Since reaction speed is the binding constraint, a faster board gives headroom; you’ll want the AE3, N6, or H7 R1 as these cameras can run at 120+ FPS out of the box.
Complete starter script
Here’s a full, runnable script that does all of the above — blob detection, the least-squares trajectory fit, zone mapping, the confirmation filter, and the three output pins. I’ve tested the trajectory math separately; you’ll need to plug in your own LAB threshold from the IDE and calibrate the four geometry constants at the top (goal-line y and the two zone boundaries) to your camera mounting:
# Automated Foosball Goalkeeper - OpenMV
#
# Tracks an orange ball, predicts which goal zone (left/center/right) it is
# heading toward, and drives one of three GPIO output pins HIGH to signal an
# ESP32. Designed for high frame rate so the decision lands within a few frames.
#
# WORKFLOW BEFORE RUNNING:
# 1. Use Tools -> Machine Vision -> Threshold Editor in OpenMV IDE to get the
# LAB threshold for YOUR orange ball under YOUR table lighting. Replace
# ORANGE_THRESHOLD below with the values it gives you.
# 2. Confirm the pin names ("P0"/"P1"/"P2") exist on your specific board
# (check the Boards / pinout page for your model).
# 3. Confirm both OpenMV and ESP32 are 3.3V I/O and share a common ground.
# 4. Calibrate GOAL_LINE_Y and the two zone boundaries (see CALIBRATION).
import csi
import time
from machine import Pin
# ---------------------------------------------------------------------------
# CONFIGURATION
# ---------------------------------------------------------------------------
# LAB threshold: (L_min, L_max, A_min, A_max, B_min, B_max)
# Orange sits in positive A and strongly positive B. TUNE THIS in the IDE.
ORANGE_THRESHOLD = [(20, 90, 10, 60, 25, 90)]
# CALIBRATION (all in image pixel coordinates at QVGA = 320x240) -------------
# Camera is mounted directly above the pitch. Assume the goal is along the
# bottom edge of the frame and the ball travels toward increasing y.
GOAL_LINE_Y = 230 # y of the goal line (where the goalie rod sits)
ZONE_BOUNDARY_L = 107 # x < this -> LEFT zone
ZONE_BOUNDARY_R = 213 # x > this -> RIGHT zone; between the two -> CENTER
# TUNING ---------------------------------------------------------------------
HISTORY_LEN = 5 # how many recent centroids to keep for the line fit
MIN_POINTS_FIT = 3 # need at least this many points before predicting
MIN_FORWARD_DY = 1.0 # px/frame toward goal before we treat it as a shot
CONFIRM_FRAMES = 2 # predicted zone must agree this many frames in a row
PIXELS_THRESHOLD = 20 # ignore blobs smaller than this (noise)
ZONE_LEFT, ZONE_CENTER, ZONE_RIGHT = 0, 1, 2
# ---------------------------------------------------------------------------
# HARDWARE: three output pins to the ESP32
# ---------------------------------------------------------------------------
left_pin = Pin("P0", Pin.OUT_PP)
center_pin = Pin("P1", Pin.OUT_PP)
right_pin = Pin("P2", Pin.OUT_PP)
def signal_zone(zone):
"""Drive exactly one pin HIGH (or all LOW if zone is None).
Exclusivity is enforced here: triggering one zone guarantees the other
two stay LOW, which is exactly the behavior the project requires.
"""
left_pin.value(zone == ZONE_LEFT)
center_pin.value(zone == ZONE_CENTER)
right_pin.value(zone == ZONE_RIGHT)
def zone_for_x(x):
"""Map a predicted goal-line x-coordinate to a defensive zone."""
if x < ZONE_BOUNDARY_L:
return ZONE_LEFT
elif x > ZONE_BOUNDARY_R:
return ZONE_RIGHT
return ZONE_CENTER
def predict_crossing_x(history):
"""Least-squares fit x as a function of y over recent points, then
extrapolate to GOAL_LINE_Y. Over ~1.5 m of flat travel a straight line
is plenty -- no physics/gravity model needed.
history is a list of (cx, cy) tuples. Returns predicted x, or None if the
points are degenerate (e.g. ball not really moving in y).
"""
n = len(history)
sum_y = sum_x = sum_yy = sum_xy = 0.0
for (x, y) in history:
sum_x += x
sum_y += y
sum_yy += y * y
sum_xy += x * y
denom = (n * sum_yy) - (sum_y * sum_y)
if abs(denom) < 1e-6:
return None # all points at ~same y; can't extrapolate a heading
# x = slope * y + intercept
slope = ((n * sum_xy) - (sum_x * sum_y)) / denom
intercept = (sum_x - (slope * sum_y)) / n
return (slope * GOAL_LINE_Y) + intercept
# ---------------------------------------------------------------------------
# CAMERA SETUP
# ---------------------------------------------------------------------------
csi0 = csi.CSI()
csi0.reset()
csi0.pixformat(csi.RGB565)
csi0.framesize(csi.QVGA) # 320x240 -- small = fast. Don't use VGA here.
csi0.snapshot(time=2000) # let AWB/AGC stabilize on startup
csi0.auto_gain(False) # CRITICAL: lock exposure...
csi0.auto_whitebal(False) # ...and white balance so the threshold holds
clock = time.clock()
# ---------------------------------------------------------------------------
# MAIN LOOP
# ---------------------------------------------------------------------------
history = [] # recent (cx, cy) centroids
candidate_zone = None # zone we are about to confirm
candidate_count = 0 # how many frames it has agreed
signal_zone(None) # start in the default (idle) stance
while True:
clock.tick()
img = csi0.snapshot()
blobs = img.find_blobs(ORANGE_THRESHOLD,
pixels_threshold=PIXELS_THRESHOLD,
area_threshold=PIXELS_THRESHOLD,
merge=True)
if blobs:
# The ball is the largest matching blob.
ball = max(blobs, key=lambda b: b.pixels())
cx, cy = ball.cx(), ball.cy()
img.draw_rectangle(ball.rect(), color=(255, 0, 0))
img.draw_cross(cx, cy, color=(0, 255, 0))
history.append((cx, cy))
if len(history) > HISTORY_LEN:
history.pop(0)
# Is the ball actually heading toward the goal?
moving_forward = False
if len(history) >= 2:
dy = history[-1][1] - history[-2][1]
moving_forward = dy >= MIN_FORWARD_DY
predicted_zone = None
if len(history) >= MIN_POINTS_FIT and moving_forward:
px = predict_crossing_x(history)
if px is not None:
predicted_zone = zone_for_x(px)
img.draw_line(history[-1][0], history[-1][1],
int(px), GOAL_LINE_Y, color=(0, 0, 255))
# Confirmation filter: require the same zone for CONFIRM_FRAMES frames
# so a single noisy frame can't twitch the goalie.
if predicted_zone is not None and predicted_zone == candidate_zone:
candidate_count += 1
else:
candidate_zone = predicted_zone
candidate_count = 1 if predicted_zone is not None else 0
if candidate_zone is not None and candidate_count >= CONFIRM_FRAMES:
signal_zone(candidate_zone)
else:
# Ball not visible -- clear history so a new shot starts fresh.
history.clear()
candidate_zone = None
candidate_count = 0
signal_zone(None) # return to idle/default stance
print(clock.fps(), "fps")
Suggested build order
-
Get find_blobs reliably locating the ball and printing fps. Confirm you’re well above 100 fps at QVGA.
-
Add the centroid history and print the predicted goal-line crossing x. Verify it tracks reality by rolling the ball slowly.
-
Calibrate GOAL_LINE_Y and the two zone boundaries.
-
Wire up the three pins and test the full loop.
-
Tune CONFIRM_FRAMES and MIN_FORWARD_DY for jitter vs. latency.
A couple of the “related topics” the forum surfaced under your post — the ball trajectory detection with 100ms reaction time thread and the golf ball detection, speed and direction thread — are tackling almost exactly your problem and worth reading; they’ll have real tuning numbers from people who hit the same frame-rate wall.
Happy to go deeper on any single piece if you tell me which board you end up getting.