jujuadams/snap

Data format converters for GameMaker LTS 2022

SNAP 6.1.1

Data format converters for GameMaker Studio 2022 LTS

 

• Download the .yymps

• Read the documentation

• Find further experimental data formats here

 

 

Whilst SNAP is focused on encoding/decoding structs and arrays, @tabularelf has ds_* conversion scripts available in a gist: https://gist.github.com/tabularelf/547b62efbcb0beaa8e6478afae8e693f

snap's Issues

JSON with comments

It'd be nice if SNAP could parse JSON with Comments, which are used in Visual Studio Code for example.

sample json string:

// header comment.
{
    // hello, a comment.
    "testprop": 222, // this property controls the amount of pugs.
    "testarr": [ "grumpy pug" /* this pug is very grumpy */, "happy pug" /* :) */],
    /*this string is very important, removing it might lead to serious consequences. */"teststr": "Hello, SNAP!"

    /*
        hello, earth! // yes.


        // exactly.
    */
}
// end comment.

Previous optimisation for 0/1 floats in SnapBufferWriteBinary() now breaks in newer versions of GameMaker

Need to remove the special case and keep datatype consistent.

VDF

https://developer.valvesoftware.com/wiki/KeyValues

Steam uses Valve's KeyValues format (colloquially "VDF" for Valve Data Format, originating from the Source Engine) for app configuration. There are a few reasons this would be useful, most immediately obvious a more thorough Steam Input integration for Input, currently configured only for "Gamepad Emulation" mode.

Another potential use is unifying platform integrations such as achievements, DLC and UGC management, etc.

Not expecting anyone else to implement this, but not going to self-assign for the time being in case someone else is so inclined :-)

WSOD snap_to_json() with HTML5

There's currently a fatal bug in the HTML5 version of json_stringify() for GMS2 2.3, It outputs as expected with Windows, but returns malformed JSON with HTML5. I have filed a bug report with Yoyo, but in the meantime I need to continue development.

I was happy to discover this library as an alternative, However, I have been likewise unsuccessful in getting snap_to_json() to work with HTML5. Again, it outputs as expected in Windows, but produces a WSOD with HTML5. The

The browser console displays a very cryptic and unhelpful error message:

Unhandled Exception - TypeError: _o3(...) is undefined in file undefined at line undefined

Here's the code contained in the Create event of the Controller object:

body = {
	_links : {
		type : {
			href: "http://192.168.1.6/rest/type/score/points"
		},
	},
	title : {
		value : 100
	},
	field_game_reference : [{
		target_id : 3
	}],
	field_player_reference : [{
		target_id : 3
	}],
	field_player_ip : [{
		value : "192.168.0.16"
	}],
	field_score_points : [{
		value : 100
	}],
}

output = snap_to_json(body,true);

I attempted to attach a sample project, but unfortunately github doesn't support the yyz extension.

Function pair to decode GML struct definitions from a string

Top-level is is a struct if variable definitions are used.

//Variables are root-level member variables for the outputed struct
a = 1;
b = true //No semicolon
c = "string"; d = undefined; //Definitions on the same line
e = variable; //Throw exception for non-keyword non-delimited strings

//Arrays supported
rootArray = [
    42, //Spread across multiple lines
    3.14
];

//Structs supported
name = {
    member0 : 1,
    member1 : true,
    member2 : "string", member3: undefined,
    memberArray: [0, "a", false],
    memberStruct: {
        nested1 : .4, //Leading decimal w/o zero
        nested2 : [1, 2, 3, -4, -5.0, -5.6, 0.66666666666666] //Array literals on single lines
     },
    illegal : function() {}, //Throw exception on functions
    test : "done!", //Hanging comma
}

/*Block comments too!*/

Tiny code snippet to allow manual ordering!

Hey Juju, thanks for this awesome extension! I added a feature in __snap_to_json_parser() right after the "if (alphabetise)" part that looks like:

if (variable_struct_exists(_struct, "var_names")) {
    _names = _struct.var_names;
    _count = array_length(_names);
}

I'm exporting JSON files for players to edit (settings, custom levels, etc) and this lets me add a line to ensure the JSON file is nice & organized for the player to play with in Notepad like:

map_data = {
    var_names: ["title", "difficulty", "width", "height", "monsters"],
    //^^^since "var_names" isn't in the _names[] array this line gets left out of the JSON string!

    title:  "Dungeon of Doom",
    difficulty: 3,
    width: 200,
    height: 100,
    monsters: [
        {
            var_names: ["type", "x", "y", "hp", "speed"],
            //^^^never seen in final string, it's just in my monster constructor code

            type: "skeleton",
            x: 100,
            y: 200,
            hp: 30,
            speed: 10
        },
        {
            type: "zombie",
            x: 100,
            y: 200,
            hp: 40,
            speed: 20
        }
    ]
};

...instead of some random order GML comes up with every time which would annoy players manually editing things to have to keep re-locating them, and using the alphabetical sort ends up splitting things unintuitively where height is above a big 100 object list but width ends up way at the bottom:

map_data = {
    difficulty: 3,
    height: 100,
    monsters: [
        {
            hp: 30,
            speed: 10
            type: "skeleton",
            x: 100,
            y: 200,
        },
        {
            hp: 40,
            speed: 20
            type: "zombie",
            x: 100,
            y: 200,
        }		
    ],
    theme:  "graveyard",
    title:  "Dungeon of Doom",
    width: 200
};

Makes it easier to write up documentation for the player too, like "here's the explanation for each variable" and the order in the file will match the doc. Thought you might want to include this!

Allow for function duplication for deep copy

foreach method doesn't execute in scope

Note: This might be the intended behavior, or a limitation of GameMaker. So please feel free to close.

isAnythingHovering = function () {
	var _isHovering = false;
	foreach(things, function (_thing) {
		if (_thing.isHovering == true) _isHovering = true;
	});
	/*for (var _i = 0; _i < array_length(things); _i += 1) {
		var _thing = things[_i];
		if (_thing.isHovering == true) _isHovering = true;
	}*/
	return _isHovering;
}

In the following code, the callback function doesn't have access to and is unable to change the _isHovering local variable.

SNAP edcode messagepack error

Encode:{ x : 300, y : 300, cmd : 10, mid : "WHRdAcBIZ", time : 10 }
error Encode:85 a1 78 cd 2c a1 79 cd 2c a3 63 6d 64 0a a3 6d 69 64 a9 57 48 52 64 41 63 42 49 5a a4 74 69 6d 65 0a
right Encode:85 A1 78 CD 01 2C A1 79 CD 01 2C A3 63 6D 64 0A A3 6D 69 64 A9 57 48 52 64 41 63 42 49 5A A4 74 69 6D 65 0A

Wiki lists "array2D" type instead of "string" for snap_from_csv() function

As per title, snap_from_csv() lists the arguments as array2D, [cellDelimiter], [stringDelimiter] in the preview, whereas it should be string, [cellDelimiter], [stringDelimiter].

"Runner.exe exited with non-zero status" with certain yaml files

Hello,

I'm loading a yaml file at the start of my game using SnapFromYAML, and then after a few minutes of playing the game I get the following error

C:\ProgramData/GameMakerStudio2/Cache/runtimes\runtime-2023.4.0.113/windows/x64/Runner.exe exited with non-zero status (-1073741819)

At first I didn't think this was related to the yaml files but then I read here that the error is usually IO related and that the yaml library was the only IO in my game at the moment.

I found that a small tweak to my yaml made the error stop happening. Here's the 2 yaml varations

Version that caused the error

burn:
- {}
- duration_seconds: 2
  max_stack: 1
  dps: 1
-
  - max_stack+1
-
  - dps+1

freeze:
- {}
- duration_seconds: 2
  max_stack: 1
  slow_rate: .02
-
  - max_stack+1
-
  - slow_rate*1.5

Version that doesn't cause the error

burn:
- {}
- duration_seconds: 2
  max_stack: 1
  dps: 1
-
  - max_stack+1
-
  - dps+1



freeze:
- {}
- duration_seconds: 2
  max_stack: 1
  slow_rate: .02
-
  - max_stack+1
-
  - slow_rate*1.5

I made a git commit with this single change, then I went back and forth. The first variation consistently creates the error after a few minutes of gameplay, and the second one never creates this error

I used this tool to verify that both yaml structures are identical, and I also verified that SNAP compiles the YAML correctly in both cases

My current theory theory is that SNAP doesn't clean up the resources properly after compiling the yaml in some cases, and this can cause errors down the line

I'm sorry I don't have more specific reproduction steps. I might try reproducing the error in a smaller project later this week, but for now I figured I'd report it anyway in case you had any ideas

Function name mismatch deep/shallow add

See images -- the shallow add script holds the wrong function or the name is just mistyped?

Unable to compile for YYP

When I attempt to compile my game for YYP that's using SNAP, I run into issues with the snap_from_messagepack function, like below. I believe it's due to the way the arguments are handled as it uses argument_count.

gml_GlobalScript_snap_from_messagepack.gml.o : error LNK2019: unresolved external symbol "struct YYVAR g_Script_gml_Script____struct___15_anon___snap_from_messagepack_parser_gml_GlobalScript_snap_from_messagepack_275128___snap_from_messagepack_parser_gml_GlobalScript_snap_from_messagepack" (?g_Script_gml_Script____struct___15_anon___snap_from_messagepack_parser_gml_GlobalScript_snap_from_messagepack_275128___snap_from_messagepack_parser_gml_GlobalScript_snap_from_messagepack@@3UYYVAR@@A) referenced in function "struct YYRValue & __cdecl gml_Script____struct___15_anon___snap_from_messagepack_parser_gml_GlobalScript_snap_from_messagepack_275128___snap_from_messagepack_parser_gml_GlobalScript_snap_from_messagepack(class CInstance *,class CInstance *,struct YYRValue &,int,struct YYRValue * *)" (?gml_Script____struct___15_anon___snap_from_messagepack_parser_gml_GlobalScript_snap_from_messagepack_275128___snap_from_messagepack_parser_gml_GlobalScript_snap_from_messagepack@@YAAAUYYRValue@@PAVCInstance@@0AAU1@HPAPAU1@@Z)

I have tweaked my version of the scripts I have locally with it just using named arguments/parameters and checking is_undefined to ensure a default value is set and it compiles as expected.

Just thought I'd flag that if something like that should be fixed so the version works regardless of YYC or VM

Problem with snap_deep_copy

First off, thanks a bunch for creating these scripts, incredibly useful.
My issue is that snap_deep_copy does not copy static variables or methods from the original constructor. I fixed that by doing this:

static copy_struct = function(_source)
{
var _structName = asset_get_index(instanceof(_source));
var _copy = {};
if(script_exists(_structName))
_copy = new _structName();
... etc ...
}

it only runs the new _structName() if _structName is a constructor because of script_exists(), this is to not mess up any normal structs by trying to construct them.

Benchmarks in docs

Since there are built-in methods doing some of the stuff that SNAP does, and since in general it's useful to know what the performance of tools is going to be, it would be great to have some benchmarks in the docs. In particular head-to-heads against built-ins, where available.

Function to convert a 2D array to an array of structs

Do YAML comments need a space after the #?

This is working:

videoediting:
  - l1
  - l2
# putOnTop:
#  - Door: ZuCoffee

Data I am getting: { videoediting : [ "l1","l2" ] }

---

This is not working:

videoediting:
  - l1
  - l2
#putOnTop:
#  - Door: ZuCoffee

Data I am getting: { videoediting : [ "l1","l2" ], #putOnTop : undefined }

---

Do YAML comments need to have a space after #?

Bug with YAML handling 2D arrays

Hello! I imported your library to allow for YAML parsing and it's been super helpful so far :)

I noticed a bug while trying to parse 2D arrays

Input YAML

-
  - 1
  - 2
-
  - 3
  - 4

Expected Output

[
  [1,2],
  [3,4]
]

Actual Output

[
 [ 1 ],
 2,
 [ 3 ],
 4 
] 

I confirmed this using both tabs and spaces

I was also used these source to verify the expected output
https://www.json2yaml.com/
https://stackoverflow.com/a/49725357

I worked around by using the following format, but I figured I'd report it anyway

- - 1
  - 2
- - 3
  - 4

SnapFromJson() omits the end of a path

A json file with this content: {"file":"C:\Users\Michel\Desktop\LDTK\samples\AutoLayers_1_basic.ldtk"} (with double \ \ and not single \ )
is parsed to { file : "C:\Users\Michel\Desktop\LDTK\samples" }
omitting the last part of the path

Test code is:
global.test_me = { file: "", };
var _file = get_open_filename("*.ldtk", "");
global.test_me.file = _file;
show_debug_message("a1", global.test_me);
SnapStringToFile(SnapToJSON(global.test_me), "test_me.json");
global.test_me = SnapFromJSON(SnapStringFromFile("test_me.json"));
show_debug_message("a2", global.test_me);`

CBOR

https://www.rfc-editor.org/info/rfc7049

Rewrite snap_from_json(), and add support for \u0000

Add (simple) YAML parser

Prevent serialisation of methods

Ignore UTF-8 BOM when loading a string from a file

CSV parser improvements

• Row-major / column-major switch
• Fill in rows that aren't max length with sensible value (0 or "" or undefined)
• Add BOM

Add size parameter to SnapBufferReadVDF()

SnapToJSON - edge case with empty struct

While trying to save config. data from Evoleo's LDTKparser, I noticed an error while saving... I think due to the struct having empty values for some keys... I have iterated a bit to simplify the struct.

The code is as follow and returns the error at the end
global.__ldtk_config = {
file: "",
mappings: {
levels: { },
}
}
global.__ldtk_config = SnapFromJSON(SnapToJSON(global.__ldtk_config));

Note that it woks, if I only keep the struct as follow (deleting the "file" key)
global.__ldtk_config = {
mappings: {
levels: { },
}
}
global.__ldtk_config = SnapFromJSON(SnapToJSON(global.__ldtk_config));

The error is
Setting scheduler resolution to 1
Attempting to set gamepadcount to 12
DirectX11: Using hardware device
Collision Event time(microsecs)=2
ERROR!!! :: ############################################################################################
ERROR in
action number 1
of Create Event
for object :
variable_struct_set argument 1 incorrect type (undefined) expecting a Number (YYGI32)
at gml_Script_SnapBufferReadJSON (line 90) - _stackTop[$ _structKey] = _value;
############################################################################################
gml_Script_SnapBufferReadJSON (line 90)
gml_Script_SnapFromJSON (line 12) - var _data = SnapBufferReadJSON(_buffer, 0);
gml_GlobalScript_test (line 13) - global.__ldtk_config = SnapFromJSON(SnapToJSON(global.__ldtk_config));
C:\ProgramData/GameMakerStudio2-LTS/Cache/runtimes\runtime-2022.0.0.12/windows/x64/Runner.exe exited with non-zero status (1)
elapsed time 00:00:20.3054405s for command "C:\ProgramData/GameMakerStudio2-LTS/Cache/runtimes\runtime-2022.0.0.12/bin/igor/windows/x64/Igor.exe" -j=8 -options="C:\Users\Michel\AppData\Local\GameMakerStudio2-LTS\GMS2TEMP\build.bff" -v -- Windows Run started at 12/03/2022 18:55:28

Hjson

https://hjson.github.io/

I would like to see support for Hjson for SNAP. Hjson is a more human friendly They have various implementations which you could reference and it does share a lot of it's syntax with standard JSON so that might assist in implementing this.

Having a format like this that could be used to load in like translations (tModLoader uses Hjson for language files, as a real world example of it's usage), dialog, etc. would be nice, YAML isn't quite my thing and JSON is too strict for my liking.

There's a huge bunch of files you could use to test present in the official implementations. Here's a link to the ones present in the C++ implementation: https://github.com/hjson/hjson-cpp/tree/master/test/assets

A full implementation would be nice but I won't complain if it lacks a few things here and there. I'd be happy even if it only supported reading.

XML being read incorrectly

testfiles.zip

These two XML files have the same issue, being that XML parsing randomly breaks and starts taking some of the XML tag into its text contents. The lines it breaks on are consistent.