Program stops after writing 1025 images

I wrote a program to create a time lapse camera and am storing each image in a directory on an 8 GB SD card. The card was formatted as FAT32 using Ubuntu 16.04. Every time I run the script the program stops after 1025 images have been recorded. There is still ~7.8 GB of space on the card so space doesn’t appear to be the limitation. I have the same problem with the M4 and M7. Does anyone have ideas what might be causing the problem? Here is the code I’m using:

import pyb, machine, sensor, image, pyb, os

# Create and init RTC object.
rtc = pyb.RTC()

if(os.listdir('/images') == []): # If there are not images in directory then set the RTC
   #datetime format: year, month, day, weekday (Monday=1, Sunday=7), hours (24 hour clock), minutes, seconds, subseconds (counds down from 255 to 0)
   rtc.datetime((2018, 3, 9, 5, 13, 0, 0, 0)) 

# Enable RTC interrupts every 10 seconds, camera will RESET after wakeup from deepsleep Mode.
rtc.wakeup(10000)

BLUE_LED_PIN = 3
newName=''

sensor.reset() # Initialize the camera sensor.
sensor.set_pixformat(sensor.RGB565) 
sensor.set_framesize(sensor.VGA) # or sensor.QQVGA (or others)
sensor.skip_frames(time = 1000) # Let new settings take affect.
pyb.LED(BLUE_LED_PIN).on()
if(os.listdir('/images') == []): # If images directory is empty then first image  is labeled image_0001.bmp
   newName='images/image_0001.bmp'
   with open("time.txt", "a") as timeFile:  # Write text file to keep track of date, time and image number
      timeFile.write("Date and time format: year, month, day, weekday, hours, minutes, seconds, subseconds" + '\n')
      timeFile.write("date and time for image 1: " + ' '.join(str(s) for s in rtc.datetime()) + '\n')
else :
   lastName=os.listdir('/images')[-1] # Get the last image file name then add 1 to file name
   newNumber = int(lastName.split('.')[-2].split('_')[-1])+1
   newName='images/image_' + '%04d' % (newNumber) + '.bmp'
   with open("time.txt", "a") as timeFile:  # Append to date, time and image number to text file
      timeFile.write("date and time for image " + str(newNumber) + ": " + ' '.join(str(s) for s in rtc.datetime()) + '\n')
# Take photo and save to SD card
sensor.snapshot().save(newName)
pyb.LED(BLUE_LED_PIN).off()

# Enter Deepsleep Mode.
machine.deepsleep()

This is probably a limitation of OS list dir(). That returns a string of all files on the disk. You’re likely running out of RAM when it returns a massive string list.

Mmm, need another way to solve this…

Mmm, okay, so, the best way to deal with this is to make a log file format. I.e. open the file, seek to the end, write the size of the image that you’re about to save and then write the image. The close the file.

Does MicroPython offer a directory iterator? That’s what you need right now.

Um, to write an image out in byte form just do write() on the image object. This is an uncompressed data stream. Do compressed() first to get jpegs instead of raw bytes. Grayscale images are just a 0-255 byte per pixel. RGB565 images are in RGB565 format with their bytes reversed.

Thank you kwagyeman - I need to keep in mind that RAM is quite limited. I changed my logic to see if the log file exists using try and generate the image name based on the RTC so I don’t need to list the directory. I changed the output to JPEG format since I wasn’t able to save BMP files at VGA resolution. I’m going to let it run overnight to see how well it works. Here is the latest code in case anyone has suggestions - especially on how to reduce power consumption.

import pyb, machine, sensor, image, pyb, os

# Create and init RTC object.
rtc = pyb.RTC()
newFile  =False

try:
   os.stat('time.txt') 
except OSError: # If the log file doesn't exist then set the RTC and set newFile to True
   #datetime format: year, month, day, weekday (Monday=1, Sunday=7), hours (24 hour clock), minutes, seconds, subseconds (counds down from 255 to 0)
   rtc.datetime((2018, 3, 9, 5, 13, 0, 0, 0)) 
   newFile = True
dateTime=rtc.datetime()
year = str(dateTime[0])
month = '%02d' % dateTime[1]
day = '%02d' % dateTime[2]
hour = '%02d' % dateTime[4]
minute = '%02d' % dateTime[5]
second = '%02d' % dateTime[6]
subSecond = str(dateTime[7])
 
newName='I'+year+month+day+hour+minute+second # Image file name based on RTC 

# Enable RTC interrupts every 10 seconds, camera will RESET after wakeup from deepsleep Mode.
rtc.wakeup(10000)

BLUE_LED_PIN = 3

sensor.reset() # Initialize the camera sensor.
sensor.set_pixformat(sensor.RGB565) 
sensor.set_framesize(sensor.VGA) 
sensor.skip_frames(time = 1000) # Let new settings take affect.
pyb.LED(BLUE_LED_PIN).on()
if(newFile): # If log file does not exist then create it   
   with open('time.txt', 'a') as timeFile:  # Write text file to keep track of date, time and image number
      timeFile.write('Date and time format: year, month, day, hours, minutes, seconds, subseconds' + '\n')
      timeFile.write(newName + ',' + year + ',' + month +  ',' + day +  ',' + hour +  ',' + minute +  ',' + second +  ',' + subSecond + '\n')
else:
   with open('time.txt', 'a') as timeFile:  # Append to date, time and image number to text file
      timeFile.write(newName + ',' + year + ',' + month +  ',' + day +  ',' + hour +  ',' + minute +  ',' + second +  ',' + subSecond + '\n')
# Take photo and save to SD card
img=sensor.snapshot()
img.save('images/' + newName, quality=90)
pyb.LED(BLUE_LED_PIN).off()

# Enter Deepsleep Mode
machine.deepsleep()

This is what I’m using to create timestamp based filenames:

rtc = RTC()

def create_time_based_filename():
    datetime = rtc.datetime()
    print(datetime)
    return "{0}{1:02d}{2:02d}{3:02d}{4:02d}{5:02d}".format(datetime[0], datetime[1], datetime[2], \
            datetime[4], datetime[5], datetime[6])

You would have to add the subsecond field.