unpack/mkrbf produce incorrect rbf files for MakeCode uf2/elf files

[germanicianus]

I stumbled over that problem when trying to upload a uf2 file to the EV3 (via Bluetooth on Linux).

How to reproduce:

1. Open https://makecode.mindstorms.com/#editor in a browser and create a really simple program like the following.
   
2. Download the uf2 file of the just created program, e.g. lego-Untitled.uf2.
3. Unpack the uf2 file with the following command.
   ev3duder uf2 unpack lego-Untitled.uf2
4. Upload the unpacked files with the following commands to the EV3 (here via Bluetooth - rfcomm0).

ev3duder --serial=/dev/rfcomm0 up Untitled.elf ../prjs/BrkProg_SAVE/Untitled.elf
ev3duder --serial=/dev/rfcomm0 up Untitled.rbf ../prjs/BrkProg_SAVE/Untitled.rbf

5. Execute the program on the EV3 via it's menu.
   After starting the program, the EV3 will show a blank screen until it's return button is pressed.

I compared the correct rbf file, which is created by the EV3 when putting the uf2 file on it via USB, with the rbf file created by unpack. To achieve that, I downloaded the working rbf file via ev3duder Bluetooth from the brick. The only difference is the file length - the file created by the brick is 256 bytes long - which is 144 null bytes longer than the one created by unpack. Appending null bytes to the file created by unpack can be achieved with the following command.
truncate -s +144 Untitled.rbf

mkrbf creates a non-working rbf file which differs in content as well as in size from the working one.
ev3duder mkrbf ../prjs/BrkProg_SAVE/Untitled.elf Untitled.rbf

For OpenRoberta uf2 files, both means produce a working rbf file - unpack as well as mkrbf.

Some more remarks:

• The workaround regarding the EV3 file system problem doesn't work for me - see How to upload an uf2 file #31 (comment).
  Edit 2021-03-30: Regarding the file system issue see also Cannot reliably mount emulated storage under Linux microsoft/uf2-linux#6 and Cannot reliably mount emulated storage under Linux microsoft/pxt-ev3#980.
• It would be nice to know if it is possible to upload uf2 files directly to a directory of the EV3 where it finds them and automatically produces the correct output. When connecting the EV3 via USB, there is a directory where the uf2 files still reside, but I didn't find it (yet) on the EV3 itself.
  Edit 2021-03-30: Apparently latter uf2 files are emulated according to this comment.
• Edit 2021-03-30: The tool files2uf2 produces the correct files including directory structure from a MakeCode uf2:
  https://github.com/simonedegiacomi/files2uf2

[germanicianus]

@JakubVanek, you may be able to quickly fix this issue as you added the unpack functionality :) .

[a3f]

I've never used this uf2 feature, so I am afraid I can't help. If you got patches, I can merge them though.

[JakubVanek]

Hi,

I'm sorry for a late reply, I'll take a look at it today.

With regards

Jakub

[JakubVanek]

I think I've found the problem. Unfortunately, the files produced by MakeCode are somewhat buggy and they trigger a sanity check in the current ev3duder code.

The issue is the fileSize field in the UF2 header. For the RBF file, its value is 112. However, the file still contains 256 bytes worth of data as indicated by the payloadSize field. Ev3duder checked for this and it dropped the extra data at the end. However, uf2d (UF2 unpacking daemon running on the brick) and files2uf2 handle this differently. In uf2d, the size in fileSize field is used to resize the file before the next data block is written. The following write may then make the file larger and this is what we're seeing with the 256 vs 112 bytes.

PR #38 should fix this.