Flashing the firmware using pyopenmv.py, functions not working as intended

OpenMV Boards
1

Good morning to all, I’m currently working on a method in python3 to flash the firmware to the camera without passing through the OpenMVIDE. I need this to work somehow reliably for a project I’m working on: something related to continuous learning, where the firmware (and the micropython code) need to be “updatable” autonomously.

What I want to obtain is a way to be able to call a python script that flashes the firmware into the camera. I’m currently working on some sort of reverse engineering of the OpenMVIDE code using the functions available at:

The problem I’m stumbling upon is the sequence of commands to send to be able to access the bootloader to start erasing the flash and flashing the firmware. I tried every combination of:

  • reset()
  • reset_to_bl()
  • bootloader_start()
  • bootloader_reset()

As of today, calling bootloader_start() (which should return something through the serial) does not return anything, no matter the timing and the amount of time.sleep() delays I put into the code between resets and bootloader_start() calls.

What I’m asking is: there’s a new method to be implemented in order to be able to access, somewhat reliably, the bootloader? Or, at least, there’s a specific sequence of command to follow (with a specific timing)?
Maybe a specific command not reported in the list of constants at the beginning of the file linked (such as:
__BOOTLDR_START = 0xABCD0001
__BOOTLDR_RESET = 0xABCD0002
...)?

A schematic, a flowchart, a piece of papere with something useful written on it would help me a lot.
Also some explanations on what the methods do would be welcome!

Thanks a lot,
keep up the good work!

2

Reset to BL for OpenMV cams just resets the cam because our bootloader always runs for about 200->500ms. You need to reconnect quickly to the cam and then send bootloader commands, see the bootloader source code to understand what it expects, specifically:

3

Thanks for the quick answer!
I didn’t understand: do I have to send the reset_to_bl() or the reset() command (and then start sending bootloader commands from there)?

I’ll see what I can find in the source code you attached, thanks again a lot!

4

Both do the same thing for OpenMV cams (they just reset the board), for other boards reset-to-bootloader is handled differently. Just send reset to bootloader.

5

Good morning, again!
Thanks to your help, I was able to connect to the bootloader successfully.
I needed to use the pyserial package directly, importing pyopenmv.py and using its methods didn’t seem to work.

I’m also able to delete the FLASH (hoping so, the camera only flashes green for a second after the erasing). What I’m stuck on is the writing of the binary: OpenMVIDE seems to write the .bin file using chunks of 56 Bytes with a time delay between writings of 1 ms. Also, if the last chunk has less than 56 bytes, it needs to be filled with b’255.

These settings are not working for me. I’ll try to attach what I’ve done so far, but it’s a bit long and not as clean as it should be (or as I would like it to be :sweat_smile:).

However, the main part I need help with is the following:

with open(__fw, 'rb') as f:
	__bin = f.read()
	# chunk = f.read(CHUNK_SIZE)
	# chunks.append(chunk)

chunks = []

for xfer_bytes in range(0, len(__bin), FLASH_WRITE_CHUNK_SIZE):
		
chunk = min(FLASH_WRITE_CHUNK_SIZE, len(__bin) - xfer_bytes)

# print(xfer_bytes)
		
data_mid = __bin[xfer_bytes: xfer_bytes + chunk]

# print(data_mid)

chunks.append(data_mid)

# If the last chunk has less than 56 bytes, (56-x) with a value of 255 have to be added.

if(len(chunks) and (len(chunks[-1]) % FLASH_WRITE_CHUNK_SIZE)):

size_b = FLASH_WRITE_CHUNK_SIZE - len(chunks[-1])

listOfStrings = bytearray(size_b * [255])

chunks[-1] += listOfStrings

After the “chunking” of the binary, I erase the memory and then the following code should flash the camera:

l = len(chunks)

pb.printProgressBar(0, l, length = 50)
for i, chunk in enumerate(chunks):

       __serial.write(struct.pack("<I", __BOOTLDR_WRITE) + chunk)
       sleep(FLASH_WRITE_DELAY)
       # Update Progress Bar
       pb.printProgressBar(i + 1, l, length = 50)

where pb is a custom progress bar, nothing fancy nor of interest.

I tried using the

__BOOTLDR_QSPIF_ERASE 		= 0xABCD1004
__BOOTLDR_QSPIF_WRITE 		= 0xABCD1008

constants, but it does not erase the flash; I assume it’s because I’m working with what the “normal” memory layout, namely:


FLASH_SECTOR_START              = 4
FLASH_SECTOR_END                = 11
FLASH_SECTOR_ALL_START          = 1
FLASH_SECTOR_ALL_END            = 11

Let me know what you think, if I’m on the right track or if I’m missing something.
Any comment would be of great help.
If you need the whole code, let me know. I’ll try to upload it somewhere, the upload function does not work since I’m a new user.

Thanks again!

6

That’s a lot to process, if you narrow it down to a specific question I can answer, note we don’t really support this, so you’re pretty much on your own. That said, note QSPI is not used for code, it is used for the filesystem (and on some boards we store WiFi firmware blob there too).

7

I think that the main thing I’d like to know is the parsing of the binary.
I don’t get why it’s 56 bytes to be sent. I get the overhead of 4 Bytes due to the command, but there are 4 more bytes (according to openmv/usbd_cdc_interface.c at master · openmv/openmv · GitHub):
#define FLASH_BUF_SIZE (64)

However, I may have found the problem, there’s something wrong with the opening of the firmware.bin file from my end. It encodes the newline characters (\n), which I don’t think they should be flashed.

I’ll see what I can do, it seems that the parsing of the .bin file might be the problem. I’d like to ask if you can show me where the parsing is done in the OpenMVIDE code.

Thanks again, you’ve been really helpful!

8

It’s much easier to figure out the protocol from the bootloader code than go through all the GUI code in the IDE, but you can open an issue in the openmv-ide repo and ask for that. BTW where did you see the 56 bytes ? It should be 4 bytes for command + 60 bytes payload.

There’s no complex parsing involved, you just need to open it as a binary file and send it in chunks of 64 bytes (4 for the command, 60 bytes data) and pad the last packet if needed. Again see the bootloader code, specifically the BOOTLDR_FLASH_WRITE command.

9

56 Bytes came from what is used in the IDE.

I tried with 64 bytes (4 bytes for the command BOOTLDR_FLASH_WRITE and a chunk of 60 Bytes) but it hangs (pinging one core of my pc CPU at 100%). I’m assuming that this means I can’t send 64 bytes.

I tried with 4 + 58 bytes and it does not hang, maybe there’s some overhead, I don’t really know. I’m trying every possible combination but I can’t figure out what the problem is.

10

Okay then try with 4 command + 56 payload ? I don’t know exactly why this packet size is used in the IDE, but I know for sure the max is 64 bytes (USB EP buffer size) so we should be able to support 4+60. Also you don’t need to pad the last packet because when you send the BOOTLDR_RESET command after uploading all of the firmware, the last packet is checked and it gets padded and flushed (if needed).

EDIT:

You know what I think it had something to do with 64 bytes USB packets needed another empty packet after to flush it, or something like that. So I’d use 4 + 56 as mentioned above. Keep in mind the IDE has a lot of extra things that may not make sense but they are there to support different versions of the bootlaoder and different OSes.

11

Don’t deviate from what the IDE does. We had to fight so many terrible issues with the USB stacks on different OSes. What the IDE does is what works on Windows/Mac/Linux with different versions of each.

12

I’m trying to follow as much as I can what the IDE does, but it seems that it doesn’t work and I don’t really know why. I tried the 4 + 56 bytes so many times with different padding, different parsing of the binary, different timings between writings…

Everything should work, according to what’s written in the IDE and in the usbd_cdc_interface.c file, but it doesn’t. I tried modifying the baud_rate, the connection type, …

It erases the memory (because it boots without making the self test, i.e. green light blinking for 2-3 seconds) but I can make the BOOTLDR_FLASH_WRITE command work.

Should I stick to the padding at the end of the binary as the IDE does?

13

Yes, you should. Also, the IDE spaces the writes out by 1 millisecond. This is on purpose.

14

The bootloader pads & flushes the last packet when reset is sent (after uploading the fw image). You do send the reset command after uploading all the chunks, right ? Also you should test this on Linux maybe it’s a serial port issue.

15

@kwagyeman I do space the writes by 1 ms.
@kwagyeman I do send the BOOTLDR_RESET after the upload.

I’ll try testing it on Linux. I’m actually on macOS, so they should be the same, but it might make a difference.
I’ll try to parse the binary in a different way since, as said, opening it with the usual:

with open('firmware.bin', br) as f

takes into account the \n and spaces too, which I think might be wrong, seeing how the commands are sent through the struct.pack() method.

If it’s not a problem, I’ll ask you more if everything I’ll try won’t work; if it will work, I’ll come back the same to thank you and maybe upload the code, since someone else was asking something similar.

Thanks a lot for the help!

16

That’s very likely the wrong sector, the only cam that uses sector 4 for the fw is OPENMV-3, all the other cams the fw starts at sector 2, and end at 15… look at the omv_boardconfig.h files for flash layout.