mateoconlechuga/convbin

Converts files to other formats

convbin's Issues

Feature request: per-input compression

According to the info that convbin outputs when you run it without arguments, it says

-i, --input <file> Input file. Can be specified multiple times, input files are appended in order.

I find the ability to append multiple input files very useful, but one problem I have run into is that --compress compresses the entire output.
This makes it impossible to, in one run of convbin, append together data that I want compressed and data that I don't want compressed. If there was another option (or if the current one was modified to behave this way, but that would probably be problematic and break things) that sets compression per-input (like how each --iformat only applies to the following --input).

TL;DR: A --compress-like option that applies to only one --input when using multiple inputs in one command.

Random issue with compression

There seems to be an edge case with compression where the result causes a "Need LibLoad" or something

[Feature] .8xg => Contained Variable Conversion

Implement conversion from .8xg (auto-extracting/or non-auto-extraction) to unpacked contained variables.
Reasoning: the software can already convert TO groups of either type, may be useful to be able to convert FROM them as well.
I have a few use cases for this in mind myself.

Suggestion: Add Huffman coding pass

Since the typical calculator assembly opcode distribution is far from uniform (of the 256 possible values a lot of them are push,jump,mov etc) it makes sense to apply an entropy encoding after running RLE to capture the hardcoded data. I had good results with this in Axe parser a few years ago.

It won't work on top of the current LZ77 implementation though because it misaligns bytes with stuff like "write_elias_gamma". I'll see if I can code up a byte-aligned RLE and Huffman coding to test on some 8xp files if you're interested.

Automatic selection of best compression mode

when using 8xp-compressed there should be an auto option for convbin to try both zx7 and zx0 and use the one that produces a smaller final binary.

Add support for 84+ non-CE programs

If it's not too hard, could you add support for regular 83/84+ programs? I tried with a few and it doesn't work.

Compressed programs seem to be leaking memory

Looks like the size bytes might be the cause.

Add way to specify contents of 8X comment section

String to place in 8X comment section would be nice to have

Exporting larger than 64kB

This would entail splitting a large input file into multiple appvars; which the main program would then combine into a singular entity.

Data Name Character Limit Applies in C Headers

Using the "-n" argument to input a name in convbin, the name is limited to 8 characters for the appvar in all output formats. This means that even if one passes "-oformat C" and creates a C header file that has no such character limitations, the array name is still subject to the character limit mean for appvars. So the input name "menu_room_tilemap" is converted to "menu_roo" and doesn't throw up any errors or warnings about the name size.

Binary structure of 8XG files - do you know what the bytes represent?

Hi there,

Thanks for sharing your application. This is not an issue, but more of a question. I'd like to understand the binary structure of 8XG files. I can parse 90% of it based on material I've found online, however I'm not sure about a few of the bytes.

I know that an 8XG group file contains:

• 55 byte header
• 19 byte meta data section
• variable length body
• 2 byte checksum

Then inside the body section there can be one or more variables or programs, where each contains:

• 1 byte representing the type of variable (5 for programs)
• 5 unknown bytes, where typically the first and last are 0x00 👈
• 1 byte length of name/label
• the name
• 2 byte length of data

Could you happen to shed any light on what those 5 unknown bytes are for? I can't seem to determine any clear meaning for them.

Here's a screenshot of my parser in Kaitai IDE, which shows a simple 8XG file containing three very small sample programs. Each only a few bytes long.

I tried to look through your C code, but I'm not quite clear on how it all works.

Cheers,

Simon

Missing C Marker in Compressed Programs

When an input program is being compressed, if it has an icon and/or description, then that header is copied to the output including the C marker, but if there is neither an icon nor a description, then the C marker does not get copied.

If you really need it, a repro is to modify examples/template/main.c like this

 int main(void)
 {
-    return 0;
+    static int x[100] = {1};
+    return x;
 }

and then run
$ make clean; make COMPRESSED=YES DESCRIPTION= ICON=
and see that it shows up as eZ80 instead of C in Cesium.