Hi all,
@yokonav Pushed us to get the hardware jpeg decoder working which happened this week. Now that this is done you can play AVI MJPEG files on your Arduino Giga and Pure Thermal OpenMV.
Checkout the video here: https://youtu.be/eosdnea7uCY
Anyway, how can you do this yourself? First, you need @yokonav’s AVIParse script here: avi_giga_display_openmv/AVIParse.py at main · metanav/avi_giga_display_openmv (github.com). Save this script to your OpenMV Cam’s flash drive.
Once that’s done you can run the following script to play MJPEG AVI files on your Arduino Giga:
import image
import time
import display
from AVIParse import *
filename = 'video.avi'
avi = AVIParse(filename)
ret = avi.parser_init()
lcd = display.DSIDisplay(
framesize=display.FWVGA, portrait=True, refresh=60, controller=display.ST7701()
)
clock = time.clock()
t = time.ticks_us()
while True:
while avi.avi_info['cur_img'] < avi.avi_info['total_frame']:
clock.tick()
frame_type = avi.get_frame()
if frame_type == avi.AVI_VIDEO_FRAME:
avi.avi_info['cur_img'] += 1
img = image.Image(avi.avi_info['width'], avi.avi_info['height'],
image.JPEG, buffer=avi.buf, copy_to_fb=True)
lcd.write(img, hint=image.CENTER|image.ROTATE_90)
while time.ticks_diff(time.ticks_us(), t) < avi.avi_info['sec_per_frame']:
pass
t = time.ticks_add(t, avi.avi_info['sec_per_frame'])
print(clock.fps())
avi.avi_info['cur_img'] = 0
And you can use this script for your Pure Thermal OpenMV:
import image
import time
import display
import tfp410
from AVIParse import *
filename = 'video.avi'
avi = AVIParse(filename)
ret = avi.parser_init()
lcd = display.RGBDisplay(framesize=display.FWVGA, refresh=60)
lcd.backlight(True)
hdmi = tfp410.TFP410()
clock = time.clock()
t = time.ticks_us()
while True:
while avi.avi_info['cur_img'] < avi.avi_info['total_frame']:
clock.tick()
frame_type = avi.get_frame()
if frame_type == avi.AVI_VIDEO_FRAME:
avi.avi_info['cur_img'] += 1
img = image.Image(avi.avi_info['width'], avi.avi_info['height'],
image.JPEG, buffer=avi.buf, copy_to_fb=True)
lcd.write(img, hint=image.CENTER)
while time.ticks_diff(time.ticks_us(), t) < avi.avi_info['sec_per_frame']:
pass
t = time.ticks_add(t, avi.avi_info['sec_per_frame'])
print(clock.fps())
avi.avi_info['cur_img'] = 0
You can run either of these scripts from the IDE or save them as main.py.
Finally, you need to create a video file to playback. On the Pure Thermal OpenMV you can store the video file on an SD card which allows you to playback files that are hundreds of megabytes in size. On the Arduino Giga you only have the external flash which limits playback to about ten megabytes.
Anyway, to create a video file to use you can convert any mp4 or etc. format to an MJPEG AVI by using FFMPEG which is installed with OpenMV IDE. However, note that our convert video feature will not work just yet for this. FFMPEG doesn’t automatically turn video files into MJPEG AVIs unless you pass a few arguments which I will have to build into the IDE. For now, if you invoke the command like below you can convert a file using FFMPEG:
./ffmpeg -i input.mp4 -vcodec mjpeg -vf "scale=800:-1" -q:v 4 -an video.avi
You can find FFMPEG under <openmvide_installdir>/share/qtcreator/ffmpeg. In the command above, note that I am scaling the video down to 800 pixels in width while maintaining the aspect ratio. This saves on having to rescale the video later which will cause your framerate to drop. Additionally, the -q:v 4
part is important as this lowers the video quality per JPEG frame. Given the way the AVIParse script works you’ll run out of memory when reading an image from disk if you don’t have this set at 4 or higher (which results in a lower quality MJPEG video). Lower numbers mean higher quality, but I was unable to reliably get values 1, 2, and 3 working on a large multi-hundred-megabyte test video.
Note that if we ever code the AVIParse file directly into the MJPEG class in C we would have more resources to use which would remove this limitation on quality.
Anyway, have fun playing around with this. After you convert the file on your desktop just save it to your OpenMV Cam’s flash/sdcard before running the script above. If you have issues AVIParse complaining about that it ran out of memory then increase the -q:v 4
value to like -q:v 5
and up.
Second note, the hardware JPEG compressor has not yet been released (4.5.3). You need to install the development firmware. The current (4.5.2) firmware doesn’t have it yet. Use Tools → Install Latest Devlopement Release to get it.