microbit-foundation/microbit-fs

A TypeScript library to manipulate the micro:bit MicroPython filesystem.

microbit-fs's Issues

Little question about this library

Sorry if my question looks idiot, but ... How exactly can I use this library ?
Thanks !

Support unicode characters longer than a byte

This bug has been "ported" from the PythonEditor: bbcmicrobit/PythonEditor#60

At the moment the String.charCodeAt() method is used to get the user script character codes, and it's return value is stored in a Uint8Array.
If the value returned is larger than a byte, as it would happen with some UTF-8 characters, then some of that information is lost and the wrong character is encoded into the hex file.

The code is in the src/common.ts file:
https://github.com/microbit-foundation/js-microbit-micropython-fs/blob/ee7081de3b29faf91407eb3535bf0b8cc29fb334/src/common.ts#L7-L18

https://github.com/microbit-foundation/js-microbit-micropython-fs/blob/ee7081de3b29faf91407eb3535bf0b8cc29fb334/src/common.ts#L22-L31

Stop checking for the append-script marker in filesystem operations

Right now, it is constantly checking the micropython hex string to see if it has the old marker to append a script to a known flash address: :::::::::::::::::::::::::::::::::::::::::::

It's not a lot of processing, but the hex strings can be large and we are trying to increase performance.

The hex string provided to the MicropythonFsHex constructor should be correctly formed, and that should be the responsibility of the caller (like the Python Editor).

Append script to a hex file with included filesystem

If we attempt to append a script to a hex file we should first check if the filesystem is already in use.
If so we should throw an error.

Technically we could move any possible magnetometer calibration data and persistent config page, but this use case is so small it is not worth the effort.

Identify MakeCode hex files via UICR

We can check if a hex file is MakeCode by looking at the magic number on the UICR. This can then be used by the Python Editor to inform the user.

Loading a hex file with "free" chunks are read as files

File:
upy-full-fs-3.hex.zip

file_1.txt should not be read as a valid file as the chunks have been freed.

Add append() method to MicropythonFsHex

microbit-fs/src/micropython-fs-hex.ts

Lines 81 to 90 in a418ade

	append(filename: string, content: string): void {
	if (!filename) {
	throw new Error('Invalid filename.');
	}
	if (!this.exists(filename)) {
	throw new Error(`File "${filename}" does not exist.`);
	}
	// TODO: Implement this.
	throw new Error('Append operation not yet implemented.');
	}

MicroPython fs-builder: Incorrect generation of a full chunk.

Creating a file that takes exactly 1 chunk (128 Bytes) doesn't generate the right hex content:

Code to create the file (Chunk has 1 Byte marker, 1 Byte ail, 1 Byte offset, 1 Byte file name length, 13 Bytes file name, 111 Bytes of data):

microbitFs.write(
    'a.py',
    'a = """abcdefghijklmnopqrstuvwxyz\n' +
        'abcdefghijklmnopqrstuvwxyz\n' +
        'abcdefghijklmnopqrstuvwxyz\n' +
        'abcdefghijklmnopqrstuvwxyz\n' +
        'a"""\n'
);

Generates 1 chunk:

:108D0000FE0004612E707961203D202222226162E2
:108D1000636465666768696A6B6C6D6E6F707172AB
:108D2000737475767778797A0A61626364656667C9
:108D300068696A6B6C6D6E6F70717273747576773B
:108D400078797A0A6162636465666768696A6B6CE0
:108D50006D6E6F707172737475767778797A0A6157
:108D600062636465666768696A6B6C6D6E6F70716B
:108D700072737475767778797A0A612222220AFFF3

MicroPython generates 2 chunks (second one is empty):

:10CA0000FE0004612E707961203D202222226162A5
:10CA1000636465666768696A6B6C6D6E6F7071726E
:10CA2000737475767778797A0A616263646566678C
:10CA300068696A6B6C6D6E6F7071727374757677FE
:10CA400078797A0A6162636465666768696A6B6CA3
:10CA50006D6E6F707172737475767778797A0A611A
:10CA600062636465666768696A6B6C6D6E6F70712E
:10CA700072737475767778797A0A612222220A7E37
:10CA80007DFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF38
:10CA9000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFA6
:10CAA000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF96
:10CAB000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF86
:10CAC000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF76
:10CAD000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF66
:10CAE000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF56
:10CAF000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF46

Need to add babel pollyfills for IE

We also need to make sure they are "tree shaken" correctly, as we need a relatively small portion of polyfills and a simple test increase the UMD module from 90 KBs to 270 KBs.

rollup-plugin-babel-minify is deprecated

Needs to be replaced by rollup-plugin-terser .

• https://github.com/Comandeer/rollup-plugin-babel-minify#readme
• https://www.npmjs.com/package/rollup-plugin-babel-minify
• https://www.npmjs.com/package/rollup-plugin-terser
• https://github.com/TrySound/rollup-plugin-terser#readme

After the filesystem runs out of space the filesystem doesn't appear to remove files correctly

http://python-editor-parallax-filesystem-review.microbit.org.s3-eu-west-1.amazonaws.com/editor.html#

1. Add a a large file to the filesystem
2. Click download

• This will append main.py to the filesystem

3. Oops out of memory will appear
4. Check microbitFs in console

Refresh, and with the same program

1. Click download

• This will append main.py to the filesystem

2. This will download correctly
3. Check microbitFs in console

For both the only file in files should be main.py

UICR: End address might be misscalculated if start page is not 0

microbit-fs/src/uicr.ts

Lines 200 to 208 in 9738a5b

	return {
	flashPageSize: pageSize,
	runtimeStartPage: startPage,
	runtimeStartAddress: startPage * pageSize,
	runtimeEndUsed: pagesUsed,
	runtimeEndAddress: pagesUsed * pageSize,
	versionAddress,
	version,
	};

Will need to check that runtimeEndUsed represents numbers of pages used from the start page: https://microbit-micropython.readthedocs.io/en/latest/devguide/hexformat.html#uicr-format

Then update the code and add more tests.