How to Use OpenMV in FRC for The RoboRio

thebluedirt · March 8, 2021, 6:26pm

Hi, everyone:
I’m hoping someone can help us.
We have an OpenMV H7, and we’re using it to do vision processing for the FIRST robotics league. We have it connected to a USB port, and sending tracking coordinates over serial link to the to RoboRio ( the processing cpu for FIRST robotics).

Everything is working well, but we want to be able to see the framebuffer of the camera as well as get serial data. We can’t seem to figure out how to do that.

We know that the OpenMV can be put in UVC mode, and then on linux /dev/video0 is available like all webcams.
In our case, though, we’re using a custom detection script, running as main.py on the OpenMV. When we plug the OpenMV into our USB port, we see a virtual com port in linux, but we do not see a video device.

FRC libraries have a camera server that expects a video source, so all we need is /dev/video0.

Is there a way to get a video feed of the framebuffer, available on /dev/video0 or similar, while also still running a custom script that’s sending data over the serial link?

kwagyeman · March 8, 2021, 6:26pm

Hi, the UVC mode feature isn’t really something that we put a lot of effort into supporting. We just did it. If you want video data you should use these scrips:

github.com

openmv/openmv/blob/master/tools/pyopenmv.py

#!/usr/bin/env python2
# This file is part of the OpenMV project.
#
# Copyright (c) 2013-2021 Ibrahim Abdelkader <iabdalkader@openmv.io>
# Copyright (c) 2013-2021 Kwabena W. Agyeman <kwagyeman@openmv.io>
#
# This work is licensed under the MIT license, see the file LICENSE for details.
#
# Openmv module.

import struct
import sys,time
import serial
import platform
import numpy as np
from PIL import Image

__serial = None
__FB_HDR_SIZE   =12

This file has been truncated. show original

github.com

openmv/openmv/blob/master/tools/pyopenmv_fb.py

#!/usr/bin/env python2
# This file is part of the OpenMV project.
#
# Copyright (c) 2013-2021 Ibrahim Abdelkader <iabdalkader@openmv.io>
# Copyright (c) 2013-2021 Kwabena W. Agyeman <kwagyeman@openmv.io>
#
# This work is licensed under the MIT license, see the file LICENSE for details.
#
# An example script using pyopenmv to grab the framebuffer.

import sys
import numpy as np
import pygame
import pyopenmv
from time import sleep

script = """
# Hello World Example
#
# Welcome to the OpenMV IDE! Click on the green run arrow button below to run the script!

This file has been truncated. show original

thebluedirt · March 8, 2021, 6:26pm

Thank you for the amazingly fast reply!

Apologies for my ignorance!
Is this code that we’d run on the host computer, to get data from the openMV? Or is this code we’d run ON the openMV, so that it presents itself as a /dev/videoX device?

kwagyeman · March 8, 2021, 6:26pm

The camera will never appear as a devx device in linux. This code is just the python code you need to execute on a linux computer with python to speak the OpenMV Cam’s debug protocol to pull data from it.