Each row value is followed by 4 empty bytes to signal the end of a column. Currently I am skipping these bytes rather than reading them. I considered actually reading the "skipped" bytes, and asserting if they are NON zero but they're always going to be null bytes. Variable length data is always padded due to alignment.
Some talented friends took a look at this project and offered some advice and suggestions.
The first part is that lines are int64
, not int
and Line ids should be sequential.
I am still uncertain on the terminology concerning "rows" and "lines", synonym or not.
For game_over.tbl
, the values of name, text, and type are byte positions in the string data of the size in the header. The strings data block might be a dictionary of key value pairs. This makes sense for localization. To calculate the key position in the string data block you can just subtract the string_data_size
from the file_size
to get the start position, and seek to it. It is unknown what the line numbers mean. I'll have to play around with it to see how they map to the strings.
Below is an image showing how key positions may be calculated.
![03__2019-01-03_16_44_26](https://user-images.githubusercontent.com/1912967/50671027-3ed5ec80-0f9d-11e9-88b7-a75957a02185.png)
Kaitai
I was also introduced to Kaitai which looks like a great solution for this project, and others. An existing KSY specification was provided by fireundubh which can be examined in the Kaitai IDE Online. See also the documentation.
The KSY files can be used as blueprints to describe the file format, and then in c#, these blueprints can used. There is also a tool for generating CLR classes from KSY files. This is similar to how ANTLR grammar files work.
Below are some screenshots of game_over.tbl
being examined with a tbl.ksy
grammar.
![01__2019-01-03_16_29_04](https://user-images.githubusercontent.com/1912967/50671021-38e00b80-0f9d-11e9-889b-6b1afca7f35f.png)
![02__2019-01-03_16_29_49](https://user-images.githubusercontent.com/1912967/50671024-3aa9cf00-0f9d-11e9-8f6b-b98cd6093e87.png)
Here is a simple example in python for parsing binaries with KSY.
https://gist.github.com/fireundubh/fb811ac0e41143945556592b267ba4aa
@fireundubh, pinging you since this is a heavily altered quotation. Thanks for the amazing assistance.