Trouble converting ImageWriter to video (mp4) on H7 Plus

OpenMV Products
#1

I have been having a couple of issues trying to convert video (and sometimes play it) captured from the image_writer_1.py example script (and also the mjpeg_1.py script). I’m using MacOS and the H7 Plus with an SD card in the H7 Plus. There are two issues and they alternate.

  1. Sometimes I get the “failed to launch ffmpeg” error message when I click on the “Convert Video” tool and try to convert to MP4. Sometimes this happens even when I’m able to view the video using the Play Video tool. Sometimes, however, I get an error trying to play the video (terminal shows " Invalid data found when processing input" and doesn’t play the video)

  2. Sometimes the “Convert Video” tool appears to work, and places an .mp4 video file on my Mac desktop as I would expect. But eight out of ten times now, the file does not show a thumbnail image and when I click on it to play it on the Mac it says the document could not be open and is not compatible with QuickTime player. Twice, however, it worked (although the videos had this weird psychedelic look to it that was unlike the IDE preview window).

After I take a video and the script finishes I eject the H7 Plus, unplug it and plug it back in to see the created file. Whether its the /stream.bin being converted or the example.mjpeg files I’m having this issue.

The Micro SD card I’m using is a 128 GIG EVO card that came formatted in EX-FAT.

Anybody running into something like this?

#2

I used the mjpeg_1.py example script and saved a video to a new SD card (swapped out the SamSung one for a SanDisk in case that was the issue). Ejected and replugged in the H7 and then tried to play it with the play video tool. Terminal showed the following message:

[avi @ 0x7fb1b5052800] Could not find codec parameters for stream 0 (Video: mjpeg (MJPG / 0x47504A4D), none(bt470bg/unknown/unknown), 320x240): unspecified pixel format
Consider increasing the value for the ‘analyzeduration’ and ‘probesize’ options
Input #0, avi, from ‘/Volumes/CAMSD/example.mjpeg’:
Duration: 00:13:46.90, start: 0.000000, bitrate: 11 kb/s
Stream #0:0: Video: mjpeg (MJPG / 0x47504A4D), none(bt470bg/unknown/unknown), 320x240, 15.55 fps, 15.55 tbr, 15.55 tbn, 15.55 tbc
nan M-V: nan fd= 0 aq= 0KB vq= 0KB sq= 0B f=0/0 0

#3

Question, did you copy the file from the SD card to the PC? Do not try to launch the viewer program from the file on the as card directly.

Will check when I get home.

#4

I did not copy the file from the SD card to the PC. I was trying to convert the file from the SD card and save the converted file to my PC.

After reading your post, I copied the stream.bin file to my PC and the example.mjpeg and tried to convert those files from the PC to the PC. The stream.bin file threw the error about failing to launch ffmpeg just as it did when I tried it before from the SD card. Trying to convert example.mjpeg from the PC also did the same as it did when trying it from the SD card…it created a .mp4 file on my PC that Mac says can’t be played. But if I use the play video tool, the video does in fact play and the terminal shows:

[mjpeg @ 0x7faa33015a00] Format mjpeg detected only with low score of 12, misdetection possible!
[mjpeg @ 0x7faa330b0e00] Found EOI before any SOF, ignoring
Input #0, mjpeg, from ‘/Users/mycomputer/Desktop/example.mjpeg’:
Duration: N/A, bitrate: N/A
Stream #0:0: Video: mjpeg, yuvj444p(pc, bt470bg/unknown/unknown), 320x240, 25 tbr, 1200k tbn, 25 tbc
[mjpeg @ 0x7faa33068800] Found EOI before any SOF, ignoring
[swscaler @ 0x7faa18090000] deprecated pixel format used, make sure you did set range correctly
[mjpeg @ 0x7faa33068800] overread 80KB vq= 145KB sq= 0B f=0/0
[mjpeg @ 0x7faa33068800] overread 80KB vq= 275KB sq= 0B f=0/0
[mjpeg @ 0x7faa33068800] overread 80KB vq= 151KB sq= 0B f=0/0
[mjpeg @ 0x7faa33068800] huffman table decode errorq= 0B f=0/0
[mjpeg @ 0x7faa33068800] overread 80KB vq= 130KB sq= 0B f=0/0
[mjpeg @ 0x7faa33068800] overread 80KB vq= 34KB sq= 0B f=0/0
25.78 M-V: 0.000 fd= 0 aq= 0KB vq= 0KB sq= 0B f=0/0

I see no difference between those two scenarios. Is it possible that me not formatting the SD cards out of the box is an issue? I checked that they were already formatted in the right file structure, but I don’t know if that matters. I tried one from Samsung and one from Sandisk to see if it might be a card issue but that doesn’t seem to be the issue.

#5

I’m thinking maybe the file wasn’t closed by the camera before you removed the SD card leaving the file corrupt. I’ll try this right now and let you know if it works for me.

#6

Okay, I’ll try it again and I will wait until after the script has completed for a minute before disconnecting and replugging in the H7 and then I’ll try it again and see what happens.

Edit: This time I recorded a mjpeg using the example script and after the script completed and said “done” and told me to reset the camera I waited a minute. Then I disonnected from the H7 in the IDE. Then I ejected it from the OS. Then I plugged it back in. I opened it up on my desktop, and noticed it’s showing 0 bytes for its size. I dragged it to the desktop.

I then launched the Convert Video tool from the IDE and selected the example.mjpeg file on my desktop, had it save to my desktop, chose not to rescale it and it gave me the “failed to launch ffmpeg” error.

In case it’s helpful I sometimes get a “OSError: Assertion failed” error when running the script. But then I run it again and it works.

So I swapped out the SD card again to the original. Did it all again. This time the example.mjpeg file showed 1MB in size. But trying to convert it resulted that time in an mp4 that can’t be read.

#7

Files being zero bytes is a known issue with how MP pydisks work. It doesn’t mean the file is zero bytes in size.

Actually I winder about the zero bytes thing. It’s never been a problem. But, I’ve noticed that.

Okay, I’ll have some test results in about an hour.

#8

Thank you sir, much appreciated!

I also have another H7 Plus in the packaging that I could try as well if it’s helpful for troubleshooting.

#9

Hey, I’ll get to this in the morning.

#10

Awesome, thanks!

#11

This morning I formatted the Samsung SD card in Ex-FAT and tried again. Now the H7 doesn’t seem to save the files to the SD card (I eject and re-plugin before checking). Perhaps because I renamed the SD card from “untitled” to something else?

I renamed it back to “untitled” and now it saves the example.mjpeg file to the card. I copied it to the desktop and converted it and it resulted in an .mp4 that Quick Time says won’t play still.

#12

Hi, I was able to record GIF/MJPEG files without issue.

I ran the examples with an SD card attached to the camera. Then once the examples were finished… I went to Tools->Reset Camera and then after the SD card popped up I dragged the file over to the PC and then use the Tools->Play Video feature to play the video file.

Now, for the ImageWriter, Ibrahim changed the API in the latest firmware so I had to update the example:

# Image Writer Example
#
# USE THIS EXAMPLE WITH A USD CARD! Reset the camera after recording to see the file.
#
# This example shows how to use the Image Writer object to record snapshots of what your
# OpenMV Cam sees for later analysis using the Image Reader object. Images written to disk
# by the Image Writer object are stored in a simple file format readable by your OpenMV Cam.

import sensor, image, pyb, time

record_time = 10000 # 10 seconds in milliseconds

sensor.reset()
sensor.set_pixformat(sensor.RGB565)
sensor.set_framesize(sensor.QQVGA)
sensor.skip_frames(time = 2000)
clock = time.clock()

img_writer = image.ImageIO("/stream.bin", 'w')

# Red LED on means we are capturing frames.
red_led = pyb.LED(1)
red_led.on()

start = pyb.millis()
while pyb.elapsed_millis(start) < record_time:
    clock.tick()
    img = sensor.snapshot()
    # Modify the image if you feel like here...

    img_writer.write(img)
    print(clock.fps())

img_writer.close()

# Blue LED on means we are done.
red_led.off()
blue_led = pyb.LED(3)
blue_led.on()

print("Done")
while(True):
    pyb.wfi()

Anyway, after running that, resetting the camera, and then copying the file and playing it I see the issue. The image has crazy colors. This is because I have not removed the RGB565 byte inversion in the IDE but it was removed in the OpenMV Cam firmware. So, the issue is with the IDE code. I will have to release a new version of the IDE with this changed.

As for ffmpeg having issues on Mac. I will have to test this to make sure the binary is getting launched correctly. On windows I don’t have any issues.

#13

I downloaded VLC and it opens the files AND they look normal without that weird psychedelic look to them. This is using the mjpeg script. I suspect using the image_writer script would still produce the odd colors. I tried it again to test but got the “ffjpeg failed to launch” again.

But for mjpeg, Quick Time player doesn’t play it, but VLC does. Sweet!

#14

To clarify on the above. For the longest, our firmware stored RGB565 pixels in a byte inverted manner because the camera generated images in big endian mode. On the original M4 OpenMV Cam we just didn’t have time to fix this when launching the product.

This wasn’t an issue until we started trying to use the hardware acceleration of the STM32 which only works with non-inverted RGB565 data. So, I fixed this… but, I guess not in the IDE yet.

#15

Thanks for the update.

For my purposes I’ll just continue with having it save in mjpeg format I suppose for my purposes and go from there.