What is the best way to check memory usage?

I have a project that has about 500 lines of code and one class in an external .py file. I am getting memory errors when I try to start the project.

print('Used: ' + str(gc.mem_alloc()) + ' Free: ' + str(gc.mem_free()))

Claims that I have about 23K free yet the errors I get complain about memory allocation failed for hundreds of bytes.
The code that pushes the project over the edge doesn’t involve any large memory allocations. I can light switch the problem by adding and removing a dozen lines of basic code.
What is the best way to troubleshoot problems like this?

Regards,

Greg

Can you post the code that’s the issue? Python hides a lot of memory alloc sometimes. Usually, heap fragmentation is a problem if you allocate large things and remove them repeatedly.

Hi Kwagyeman,

I can’t post all of it as it’s for a client. I’m using the OpenMV as rapid proto tool…

I have the project working in pieces and didn’t hit the problem till I tried to combine the parts.
The interesting thing is that I’m using some dotstar LEDs from adafruit for the illumination.
I was able to lightswitch the problem by just adding or removing a block like:

dotStarWhite = [0xFF,0x0F,0x0F,0x0A,\
                0xFF,0x0F,0x0F,0x0A,\
                0xFF,0x0F,0x0F,0x0A,\
                0xFF,0x0F,0x0F,0x0A,\
                0xFF,0x0F,0x0F,0x0A,\
                0xFF,0x0F,0x0F,0x0A,\
                0xFF,0x0F,0x0F,0x0A,\
                0xFF,0x0F,0x0F,0x0A,\
                0xFF,0x0F,0x0F,0x0A]

illuminatorWhite = dotStarPrefix
illuminatorWhite += bytes(dotStarWhite)
illuminatorWhite += dotStarSuffix

Regards,

Greg

From my knowledge of MicroPython that array is actually 4X larger than you think since each number is stored in 4 bytes. Then, MP will allocate a new array on the += and another new array on the += again. Your script takes space too in RAM when it’s parsed into a byte array.

This seems to work on my M7:

import gc
print('Used: ' + str(gc.mem_alloc()) + ' Free: ' + str(gc.mem_free()))
dotStarPrefix = [0xFF,0x0F,0x0F,0x0A,\
                0xFF,0x0F,0x0F,0x0A,\
                0xFF,0x0F,0x0F,0x0A,\
                0xFF,0x0F,0x0F,0x0A,\
                0xFF,0x0F,0x0F,0x0A,\
                0xFF,0x0F,0x0F,0x0A,\
                0xFF,0x0F,0x0F,0x0A,\
                0xFF,0x0F,0x0F,0x0A,\
                0xFF,0x0F,0x0F,0x0A]
print('Used: ' + str(gc.mem_alloc()) + ' Free: ' + str(gc.mem_free()))
dotStarWhite = [0xFF,0x0F,0x0F,0x0A,\
                0xFF,0x0F,0x0F,0x0A,\
                0xFF,0x0F,0x0F,0x0A,\
                0xFF,0x0F,0x0F,0x0A,\
                0xFF,0x0F,0x0F,0x0A,\
                0xFF,0x0F,0x0F,0x0A,\
                0xFF,0x0F,0x0F,0x0A,\
                0xFF,0x0F,0x0F,0x0A,\
                0xFF,0x0F,0x0F,0x0A]
print('Used: ' + str(gc.mem_alloc()) + ' Free: ' + str(gc.mem_free()))
illuminatorWhite = dotStarPrefix
print('Used: ' + str(gc.mem_alloc()) + ' Free: ' + str(gc.mem_free()))
illuminatorWhite += bytes(dotStarWhite)
print('Used: ' + str(gc.mem_alloc()) + ' Free: ' + str(gc.mem_free()))
illuminatorWhite += dotStarPrefix
print('Used: ' + str(gc.mem_alloc()) + ' Free: ' + str(gc.mem_free()))
print(illuminatorWhite[0])
print('Used: ' + str(gc.mem_alloc()) + ' Free: ' + str(gc.mem_free()))

It prints:

Used: 3072 Free: 52816
Used: 3440 Free: 52448
Used: 3792 Free: 52096
Used: 4000 Free: 51888
Used: 4432 Free: 51456
Used: 4928 Free: 50960
255
Used: 5120 Free: 50768

That is interesting and certainly good to know about the 4x.
The part I didn’t explain very well is that adding or removing that block of code can light switch the problem.
Based on your experiment we are talking about roughly 2k.

print('Used: ' + str(gc.mem_alloc()) + ' Free: ' + str(gc.mem_free()))

Claims that I still have around 20k available.
Any suggestions for further troubleshooting this?

Regards,

Greg

Mmm, since I can’t see the code it’s kinda hard. Can you make some code that shows the error that I can try? I can debug then.

One thing to also try is to preallocate. I.e. allocate a bunch of large things first, then, deallocate one, put a new thing in it’s place, and keep going. The heap may be fragmented where it can’t find a large chunk free. I guess we should add a call to MP that returns the max free block size available.

By allocating your large items first you free prevent space they use from being fragmented.

Hi, yes use this to dump GC alloc table:

import machine
machine.info(1)

Dots are free blocks, anything else is an allocated GC block. Note 1 GC block == 16 bytes.