Thanks for the nice products. What I am using is the Genx320 event camera with RT1062 board. From the IDE, I can set the frequency to 200FPS, and I saw the streaming frequency for USB is about 18FPS from the downright corner of IDE. When I run pyopenmv_fb.py in my linux PC, the receiving frequency if also 18FPS which should be correct.
However, this is contrary to our purpose of using the event camera: we would rather use the high refresh rate feature of the event frame. I know this is limited by the USB tranfer rate (12Mb for full speed USB?), but I noticed the USB for RT1062 also support high speed (480Mb). Do you have any idea how to realize the high frequency tranfer? Thanks.
There’s a benchmark mode in pyopenmv_fb.py that could help with estimating the maximum USB throughput. Try something like:
python pyopenmv_fb.py --bench --poll 10
Keep in mind that none of these cameras are designed to just stream video over USB as fast as possible, it’s only there to help with development.
1 Like
Another issue is the JPEG compression. The RT1062 can’t compress images fast enough to send to the PC.
Can you do the processing on the RT1062? It’s not meant to be a frame grabber. If this is all you want the platform for, there’s a way to stream RAW uncompressed video over USB back to the PC. However, this will still be slower than the frame rate if you process the frames onboard the system.
1 Like
Can you please talk more about the way to stream RAW uncompressed video over USB to the PC? We need to do more complex operations for event frame, I guess the RT1062 is not enough for processing.
Hi, attached is a firmware for the RT1062 with RAW mode output enabled.
firmware.zip (1.4 MB)
I’ve enabled RAW mode output and removed the frame rate limit of 20 Hz.
I’ve also attached an updated version of the pyopenmv_fb script with an updated benchmark script to work for the GENX320.
pyopenmv.zip (4.1 KB)
You should get 90 FPS back to the PC. There are things that can be done to go faster… but they get into the single-purpose system optimization realm.
The diff of changes are below if you are comfortable editing the firmware and want to play with it yourself if you need to push it beyond 90 FPS. The USB interface could do up to 30 MB/s, but, it’s not easy to hit this.
diff --git a/src/omv/boards/OPENMV_RT1060/omv_boardconfig.h b/src/omv/boards/OPENMV_RT1060/omv_boardconfig.h
index ceade308..bf2a5a34 100644
--- a/src/omv/boards/OPENMV_RT1060/omv_boardconfig.h
+++ b/src/omv/boards/OPENMV_RT1060/omv_boardconfig.h
@@ -73,6 +73,11 @@
#define OMV_USB_PHY_TXCAL45DP (0x06U)
#define OMV_USB_PHY_TXCAL45DM (0x06U)
+// Enable RAW preview.
+#define OMV_RAW_PREVIEW_ENABLE (1)
+#define OMV_RAW_PREVIEW_WIDTH (512)
+#define OMV_RAW_PREVIEW_HEIGHT (320)
+
// Linker script constants (see the linker script template mimxrt10xx.ld.S).
// Note: fb_alloc is a stack-based, dynamically allocated memory on FB.
// The maximum available fb_alloc memory = FB_ALLOC_SIZE + FB_SIZE - (w*h*bpp).
diff --git a/src/omv/common/usbdbg.c b/src/omv/common/usbdbg.c
index bd8763c4..38f0fe92 100644
--- a/src/omv/common/usbdbg.c
+++ b/src/omv/common/usbdbg.c
@@ -233,7 +233,6 @@ void usbdbg_data_in(uint32_t size, usbdbg_write_callback_t write_callback) {
uint8_t byte_buffer[size];
memset(byte_buffer, 0, size);
uint32_t *buffer = (uint32_t *) byte_buffer;
- static uint32_t last_update_ms = 0;
// Set script running flag
if (script_running) {
@@ -247,8 +246,7 @@ void usbdbg_data_in(uint32_t size, usbdbg_write_callback_t write_callback) {
}
// Limit the frames sent over USB to 20Hz.
- if (ticks_diff_ms(last_update_ms) > 50 &&
- mutex_try_lock_alternate(&JPEG_FB()->lock, MUTEX_TID_IDE)) {
+ if (mutex_try_lock_alternate(&JPEG_FB()->lock, MUTEX_TID_IDE)) {
// If header size == 0 frame is not ready
if (JPEG_FB()->size == 0) {
// unlock FB
@@ -261,7 +259,6 @@ void usbdbg_data_in(uint32_t size, usbdbg_write_callback_t write_callback) {
buffer[1] = JPEG_FB()->w;
buffer[2] = JPEG_FB()->h;
buffer[3] = JPEG_FB()->size;
- last_update_ms = mp_hal_ticks_ms();
}
}
For even faster speeds you can increase the USB buffers from 512 bytes to 4096 bytes to max out the USB transfers.
firmware.zip (1.4 MB)
The updated firmware is attached. This changed required reducing the heap available on the board.
-#define OMV_GC_BLOCK1_SIZE (288K)
+#define OMV_GC_BLOCK1_SIZE (270K)
And in lib/micropython
diff --git a/shared/tinyusb/tusb_config.h b/shared/tinyusb/tusb_config.h
index 0cc5ef039..b51712722 100644
--- a/shared/tinyusb/tusb_config.h
+++ b/shared/tinyusb/tusb_config.h
@@ -75,6 +75,11 @@
#define CFG_TUD_MSC (0)
#endif
+ // CDC Endpoint transfer buffer size, more is faster
+ #define CFG_TUD_CDC_EP_BUFSIZE (4096)
+ #define CFG_TUD_CDC_RX_BUFSIZE (4096)
+ #define CFG_TUD_CDC_TX_BUFSIZE (4096)
+
// CDC Configuration
#if CFG_TUD_CDC
#ifndef CFG_TUD_CDC_RX_BUFSIZE
Any further speed increases would require substantial modifications to how the triple buffered frame buffer works and the IDE image buffer. Right now the system copies each frame to be sent to a buffer the IDE reads async which allows the python script running to continue operation without waiting on the IDE to read a frame.
1 Like
Note to the above. That was me running the benchmark script. Performance is different on the PC when viewing:
Note the --bench being removed.
The RT1062 is still running at 150 FPS onboard versus the PC viz which seems to be the limiter.
Thanks for your reply! This is amazing, now I can get 100FPS from USB, that’s enough for my project!
1 Like