Better filesystem support for logging (e.g. append mode for files)

[eduvik]

My use case is to log various occurences to the filesystem, so they can be later retreived, regardless of power cycles. Files cannot be opened in append mode, so the current recommendation is to read the entire file, add lines to it, then write it back to the file system. Given the RAM constraints of the microbit, this is not feasible after a little while - there is ~30 KB of usable disk space, but 16KB RAM in total (presumably much less is available to the user program).

My current approach is to store my entire data structure in RAM, and periodically write it to disk, but this severely limits how much I can log as RAM is much smaller than disk space. This is also made harder as there is no way to tell how much disk space is free; the only way seems to be to try writing, and catch the OSError if it fails.

Being able to write to files in append mode would be the simplest solution here from my perspective, but if that's too difficult, are there other ideas?

[eduvik]

I'm looking at implementing this feature, but before I start, I have some questions

• Is there interest in this from anyone more heavily involved in microbit development?
• Is it better to develop it in this fork, or upstream in micropython (i.e. is there substantial differences in the relevant filesystem code)?
• Any pointers to development tools for debugging and testing (e.g. a microbit simulator or ICE/JTAG support)? Or do we build the hex, dump it to the microbit, and test manually?

[rhubarbdog]

have you thought about having a 1Kb data buffer in your program and writing upto 30 files called results-1.txt, results-2.txt results-3.txt. i think the reason that append and insert mode weren't added to the simplified file system is that they would require a block management layer. taking up a block or 2 and loads of extra code in the micropython engine and a specialised version of ufs put/get to handle files written like this. The microbit has limited resources and @dpgeorge had to take design decisions to make things work.
Like it would be nice to have bluetooth in microbit micropython but because bluetooth takes 12Kb of buffer space. But once the micropython engine was running and the 12kb of buffers were allocated there was little or no space for user code and it's execution.
Have you thought of trying a PyBoard https://store.micropython.org/product/PYBv1.1H these have larger resources the micropython is different but not a huge paradigm leap and have a full file system and can store a few KB on board but also have a slot to insert an SD card

[eduvik]

I can understand that insert mode makes things significantly more complex, but I don't understand how append mode is more complex than writing to a new file; once a new file has had some data written to it, isn't writing more data to it the same as appending data to an existing file? That said, my knowledge of filesystem internals is rudimentary, so apologies if I'm missing something.

I'm using micro:bits in an educational context (school) so other platforms are not an option in this case (I tinker with other devices in my spare time, but in this case we need micro:bit support)

[eduvik]

With the micro:bit v2 coming out, with more RAM and flash, this might be worth looking at again.

[deshipu]

This version of MicroPython will not work on the new micro:bit. There is a proper version of MicroPython for the nRF52833 already.