Should we build the grammars from a different format than .tmLanguage?

[sgtoj]

Should we build the grammars from a different format than .tmLanguage?

[vors]

Since there are a lot of projects that are taking a dependency (thru git submodule or otherwise), I think it's better to stick with the current format to avoid any additional cross-project coordination. I know that some of them support both json and yaml, but I'm not sure about the original TextMate for example.

For local development, people can easily convert back and forth between the xml and their preferred format.

[daviwil]

Damn, I didn't realize they had done this, that's awesome! Thanks for bringing it to our attention. I'm sure @vors and others would certainly be interested in doing something similar.

[gravejester]

This is nice, I'll take a look into this, if it's ok @daviwil :)

[daviwil]

Go for it!

[gravejester]

For your information, v2 of the EditorSyntax package will generate grammar files in the following formats:

• plist (.tmLanguage)
• JSON
• CSON
• YAML

Editing the grammars are done using the YAML-version, and the build task will generate the other formats.

[omniomi]

I am not married to .tmLanguage as the base from which grammars are built but I am opposed to yaml. Would prefer to use json if we want to switch.

[rjmholt]

For local development, people can easily convert back and forth between the xml and their preferred format.

I would actually take that as an argument in the opposite direction. We should pick the format easiest to develop in and generate the others.

For a consumer, things like ordering and line indentation aren't important, but generating an XML to check in based on working in JSON is likely to cause unnecessary diffs.

The plist format is not very well documented and I'm finding it difficult to work with, compared to the JSON I've seen generated.

[msftrncs]

I purposely created PowerShell scripts to convert between XML and JSON. I work in the JSON file, and convert to XML for the purpose of making PR's to this repo. JSON isn't the best, but its a lot easier than XML.

[vors]

toml ftw

[rjmholt]

I purposely created PowerShell scripts to convert between XML and JSON. I work in the JSON file, and convert to XML for the purpose of making PR's to this repo. JSON isn't the best, but its a lot easier than XML

Yeah JSON would be my preference after looking at the alternatives. TOML would be nice, but it's not as well documented or supported as a TM format as JSON. The VSCode TM grammar extension supports pretty seamless conversion.

@msftrncs have you experienced any issues with the XML -> JSON -> XML rearranging the XML and causing bad diffs (e.g. moving sections around so changes are harder to identify)? I've had this in other automated JSON manipulation scenarios because JSON keys are unordered, and presume that transforming through formats would only exacerbate it.

[msftrncs]

@rjmholt, if I didn't use [ordered], yes. 😄

https://github.com/msftrncs/PwshReadXmlPList

https://github.com/msftrncs/PwshJSONtoPList

https://github.com/msftrncs/PwshOutCSON

https://github.com/msftrncs/PowerShell.tmLanguage/blob/argumentmode_2ndtry/build-plistGrammar.ps1

https://github.com/msftrncs/EditorSyntax/blob/build_grammar.ps1/tools/build-grammar.ps1

https://github.com/msftrncs/EditorSyntax/blob/build_grammar.ps1/tools/build-atomGrammar.ps1