Do you have any examples of using the image snapshot interrupt registration and handler please? I was looking for new help under ‘docs’ or new examples and I couldn’t see any clues as to how to setup the interrupt handler you mentioned in the blog of the 16th May. I was hoping for an example with the fifo, so that other blocking code (for example a network connect) would not stop the capture to disk, by capturing to the fifo in the callback and writing to disk when possible in the foreground…
Any clues or pointers would be appreciated and would be enough to get me going (even if that is a link to something I may have missed!)
Thanks again for the help and the continuous improvements.
Ah ha cool - thanks again. I obviously checked again just before you released the docs. Interestingly I have not had any problems when writing jpegs to disk since that feature was fixed, though I have not yet tried the new class.
Just a note, don’t do any work beside calling micropython.schedule() in the interrupt handler. It’s just a hook for this purpose and maybe setting a variable or toggling an I/O pin. You should not call snapshot in it.
Hi, Thanks for the tip, I was wondering how much I would get away with. Not much at all. Never mind.
Any thoughts on how I might continue to capture whilst connecting to a WIFI router/network that will be off most of the time until an event happens? (powered by a battery/solar panel), It is not urgent, so please don’t spend any time on this, just any initial starters that spring to mind as to how you would do this would be gratefully received. Thanks again!
Um, the new frame grabber driver is always capturing images.
Are you saying you want a video with with FPS with no frame drops? Then turn the video fifo feature on by doing set_framebuffers(100) or more. However, it’s pretty hard for you to write out that much data if you fall behind on reading frames out of the fifo.
Anyway, not sure what you want. Maybe, you want the camera to run two threads at once?
If so, MicroPython schedule() is what you want. It executes between MicroPython byte codes of another thread so you can basically do snapshot(), image processing, and more that will interrupt a regular MicroPython loop.
Schedule is not done during interrupt context so it’s safe to use for anything. It’s just the MicroPython VM context switching in the middle of executing MicroPython code.
I thought I read somewhere that schedule() would not help overcome a blocked thread like the wifi connect (which only takes a couple of seconds, but it needs to connect and start streaming just as we want to capture video as a visible event is happening.) I can’t see that in the documentation any more, so maybe I imagined it!
Not after especially high frame rates though they are HD or larger sized. Can we control the frame rate into the FIFO? Say 1-10 FPS at 1080P? Or can I just run at full speed and just discard say 9/10s of the frames coming out of the FIFO and only process and write to disk/stream every tenth one for example? It - the fifo - only needs to be say 3-5 seconds long…
Yeah, it won’t help bypass C code running. Only Python code.
Yeah… so, mmm, okay, you’re going to basically want two things.
I have to add a policy setting to sensor that disables the fifo being flushed when a frame drop occurs. Right now once it’s started it will auto delete all frames captured if there’s not space for a frame. This is done to ensure you don’t have stale frames in the fifo preventing new ones from being added.
If you set the camera into JPEG mode with the OV5640 and 1080p image size then you’re going to be able to handle like 10 frames in the frame buffer FIFO. So, you just need to slow down the FPS to fit all those images. Right now there’s not a great option for this. You’d want something that drops frames from the camera to reduce the camera FPS.
Anyway, so, you’d want two new settings for sensor. Being able to disable the auto flush policy which can be done with a boolean flag and you’d want a frame skip counter that would let you mod the frame rate by N.
With these two features when you call snapshot() it would then start to accumulate pics in the background. You’d probably also want some sort of kick function to start the frame capture without having to wait for frames.
…
All this stuff is pretty easy to add. However, can you describe your application in a little more detail and how this might be helpful to others than just you? While I think these features are nice… I’m not 100% sure that I should add them.
The event may be ongoing. It is for an ultra-low power CCTV camera that can run in the winter off a solar panel/battery in the northern UK (Scotland!) Notoriously difficult to do. So I have a PCB layout ready to be made up with three OpenMV camera holders so we can capture 180° at 5MP or have two PCBs and do the 360° at 15MP and do some digital PTZ using a 720P or 1080P window. The PCB also has a control over power to a 4G router and a long range lorawan like radio input from remote PIR sensors. So the master camera is capturing 24x7 at say a frame every other second. On event from a PIR we wake up the other cameras, the wifi and the router. 30Seconds later or so the router has connected to the 4G network and we can connect the wifi card to it. All the time we are writing to disk with time and date overlay. Also when the event occurs I up the frame rate to 5FPS or so (old firmware) Obviously I am 95% of the way there, it would just be nice if we could capture that moment when we are waiting for the WIFI to connect as the operator will notice that when he goes to playback the event (the client software will plays live and pre-event from the disk). This is all written and working, just about to send off the gerber files… Also this design would allow us to put together a thermal wide angle camera. This will be commercial and I have a client, but I would be prepared to publish the python and maybe the CAD (layout and 3D printed housing) elements to help others. Please let me know if you need more details…
Obviously we can in the long run do some edge processing of the CCTV, people counting and the like but that is for the future. Solar power is needed for sites like new bars, fly tipping in the countryside, large impromptu beach parties etc. We have also invented a simple way of installing up a pole/lamppost without a cherry picker…
Okay, sounds like you might want triggering support too.
Um, so, do the above features give you what you need? Being able to turn off the video fifo flush feature and dropping camera frames?
Note that this doesn’t save power like actually lowering the camera output rate. However, that’s quite hard to do with OmniVision cameras. They don’t really go slow. But, from an application perspective it gives you control over the frame rate roughly and gives you a rather large memory fifo.
(When auto flushing is turned off the camera capture literally will be running indefinitely and a new frame will be added to the fifo whenever you make space. However, there can be a major time lag if you don’t keep up. As the camera itself is constantly generating new frames. This is why being able to drop frames from it’s important).
However, you’re going to need to be able to have disk write performance keep up with the image generation speed.
Yes those features sound right as long as I can vary the input frame dropping ratio on the fly from the python code. I suspect I will be dropping a lot, but that is better than no frames.
Hi, attached is OpenMV Cam H7 Plus firmware with support for the features you wanted:
You can now specify frames to drop with sensor.set_framedrop(x) where x is the number of frames to drop before capturing a frame. E.g. if you set that to 1 it cuts the FPS in half.
You can now disabling flushing of the image fifo handled by sensor by doing: sensor.disable_full_flush(True) which will turn off automatic flushing on overflow. When you disable this is basically means that there’s no age limit on frames anymore in the fifo. So… you can now have images from an unlimited time ago in the fifo.
Run the script below to see the effect. Note that while the image fifo will fill up at the max frame rate of the camera and then continue to be feed at whatever your read frame of frames out of the fifo is. This behavior is expected and there’s not really a way to fix it. So, you should play with the depth of the fifo and the number of frames you want to drop to get the performance you want.
# Hello World Example
#
# Welcome to the OpenMV IDE! Click on the green run arrow button below to run the script!
import sensor, image, time, pyb
sensor.reset() # Reset and initialize the sensor.
sensor.set_pixformat(sensor.GRAYSCALE) # Set pixel format to RGB565 (or GRAYSCALE)
sensor.set_framesize(sensor.QVGA) # Set frame size to QVGA (320x240)
#sensor.skip_frames(time = 2000) # Wait for settings take effect.
clock = time.clock() # Create a clock object to track the FPS.
sensor.set_framedrop(0)
sensor.disable_full_flush(True)
sensor.set_framebuffers(100)
while(True):
clock.tick() # Update the FPS clock.
img = sensor.snapshot() # Take a picture and return the image.
pyb.delay(200)
print(clock.fps()) # Note: OpenMV Cam runs about half as fast when connected
# to the IDE. The FPS should increase once disconnected.
Anyway, the main thing is that the camera will capture frames in the background now and not drop any frames while you are doing something else like connecting to the internet and etc.
For the best experience you want to find a frame drop and fifo depth such that the fifo never is overflowing. In general this is not behavior you ever want. So, I would recommend you try to find a depth where disable_full_flush() is not needed. However, I added it so you don’t have to worry too much about getting that depth exactly right.
That looks awesome, thanks very much - most appreciated. I will give it a go with the connect running in the foreground… I will let you know how I get on.
I have just ordered the first ten PCBs from HK, so making progress. I am having to swap tasks for a bit now, but still hope to test the FIFO alterations this week or next. Will let you know…
I have had a good play with this simply using a six second delay and some non-compressed HD image captures. Works really well… Obviously uses lots of memory - but that is what the H7+ is for!
