Document a high level explanation of how "decompilation" works.

[bruvzg]

Document a high level explanation of how "decompilation" works.

[nikitalita]

I do want to write an explanation of how the PCK and resources are created at some point. The script “compilation” is pretty simple; all the tokens (excepting comments, but including white space) are just converted into binary numbers and just concatenated together, which is why it’s fairly trivial to recover the original scripts. The hard part is just knowing which version of the byte code it’s using.

ther’s a description of the contents of the PCK here as well: https://github.com/Bioruebe/godotdec

[KoxaKoxama]

@nikitalita Looking foward to that explanation!
How exactly gdscript tokens converted into binary numbers? I've looked at /modules/gdscript/gdscript_tokenizer.cpp and saw array of token_names, did they just saved by their index number in array? And what about numbers and characters?

I also wonder why tscn scene files are packed as plain text in pck, I know there is an option to convert_text_resources_to_binary_on_export but it's off by default in 3.x, is there is any problem with that? Is tscn tokenized the same way as gdscripts? I know that Unity uses YAML for scene files and then somehow converts that to binary as well, but I'm wondering how Godot is doing that.

There is also no info about any internal Godot formats from .import folder such as stex, scn. And why all fbx/obj/glb mesh is converted to scn in .import? I thought scn was a binary scene format converted from tscn only.
In 4.x it's .ctex and .mesh, if there is any info about any of that please share!

[nikitalita]

How exactly gdscript tokens converted into binary numbers? I've looked at /modules/gdscript/gdscript_tokenizer.cpp and saw array of token_names, did they just saved by their index number in array? And what about numbers and characters?

Yup, they're stored as just index numbers. For values and strings, they're just stored inline; Variant type number, and then value/string. For details, check out https://github.com/bruvzg/gdsdecomp/blob/master/compat/variant_decoder_compat.cpp .

GDScript is entirely interpreted, there's no real "compiliation" going on, just a conversion to bytecode that is interpreted the same way as text.

I also wonder why tscn scene files are packed as plain text in pck, I know there is an option to convert_text_resources_to_binary_on_export but it's off by default in 3.x, is there is any problem with that? Is tscn tokenized the same way as gdscripts? I know that Unity uses YAML for scene files and then somehow converts that to binary as well, but I'm wondering how Godot is doing that.

Conversion of text scenes to binary was broken a while back and they couldn't be bothered to fix it. I submitted a PR to fix it a few months ago.

There is also no info about any internal Godot formats from .import folder such as stex, scn. And why all fbx/obj/glb mesh is converted to scn in .import? I thought scn was a binary scene format converted from tscn only.

All imported formats are converted to Godot resources upon import; FBX/OBJ/GLB formats describe 3d scenes, so they are converted into Godot scenes upon import. It doesn't really matter if they are stored in binary or text, but binary takes up far less space, and it's likely that imported models/scenes will be rather hefty, is my guess.

In 4.x it's .ctex and .mesh, if there is any info about any of that please share!

They just changed the way the format is stored. I didn't pay attention much to the changes for textures, I just backported the texture loading from v3, but you can compare the way that we load images for v3 textures vs. v4 here: https://github.com/bruvzg/gdsdecomp/blob/master/compat/texture_loader_compat.cpp#L83