@pdml-lang, I've started working on a PDML syntax for Sublime Text:
https://github.com/tajmone/Sublime-PDML
The repo is just a skeleton, and I'm working on the actual syntax locally for now.
I have some practical questions to help me clarify some parsing issues which are not clear to me. Since this repository has no Discussions, I'm creating an Issue (might be worth enabling Discussions though, since it's a standard/spec, and people might like to share their implementation, etc.).
Whitespace Open Rules
In Basic PDML Specification » Whitespace it says:
There are no whitespace handling rules defined in basic PDML.
Whitespace is preserved when a PDML document is parsed.
and:
Applications reading PDML documents (or customized PDML parsers) are free to implement any appropriate whitespace handling rules, such as:
- skip whitespace nodes
- trim leading and/or trailing whitespace in text nodes
- replace whitespace sequences with a single space (similar to HTML)
Which, if I've understood correctly, is a way to keep the standard open to different applications, according to domain-specific needs — i.e. the Basic PDML Spec lies the foundations for a versatile data and markup scheme, leaving some details open on purpose in order not to preclude any specific use.
Editor Syntaxes vs Open Rules
But when implementing an editor syntax, whitespace can indeed become a problem, since the syntax needs whitespace handling rules in order to work.
Let me illustrate the problem using actual examples from the PDML website...
Whitespace Is Preserved
Again, quoting Basic PDML Specification » Whitespace:
Whitespace is preserved when a PDML document is parsed.
Consider the following PDML snippet:
According to the above example, node [a is parsed as:
{space}foo{space}{space}{space}
which indicates that leading- and trailing-spaces in a text node are preserved (separator space excluded).
Whitespace Is Ignored
In PDML Examples » Tree we find the following example:
[config
[color orange]
[size
[width 1618]
[height 1000]
]
]
where it says:
Node config has two child nodes: color and size. Node size has two child nodes too: width and height.
According to this example, all whitespace between parent and child nodes is simply ignored.
This seems to indicate that there is indeed an implicit whitespace handling rule at play here, i.e. that a text node can't be whitespace only!
If that wasn't the case, then the above example should produce various text nodes too, interpreting any whitespace between the parent nodes and their children as text nodes with spaces only:
config:
- text:
color:
- text:
size:
- text:
width:
- text:
height:
- text:
Do you see my point? According to the EBNF grammar and the railroad diagrams, a separator consists of a single whitespace entity (optional in some cases, like [node1[node2), and everything else is to be considered a text node — which the first of the two examples seems to confirm.
From the standpoint of an editor syntax (based on RegExs definitions) we need a clear rule regarding this.
Impressions…
I get the impression that the second example wishes to illustrate how trees with various levels of nodes nesting can be prettified by using indentation to provide visual cues to structure.
On the other hand, the first example seems to emphasize how PDML parsing is rules-free when it comes to handling literal text, which allows its use for handling raw strings snippets in a lossy-less manner — e.g. when handling code styled listings, where indentation matters.
But you can see that there's a grey area in-between these two extremes, when it comes to whitespace only nodes (e.g. indentation snippets, or sequences on newlines, etc.) — we can either honour them or ignore them, but can't do both at the same time.
Even though the principle of openness stands (i.e. that these details should be handled by the specific applications), editors support needs an explicit rule for this, lest it becomes an opinionated implementation of the Basic PDML syntax.
Proposals
Personally, I think it makes sense to treat whitespace in-between nodes as noise, unless there's some text, in which case we should preserve trailing and leading space entities (mandatory separator excluded), and then leave it to the user to decide whether to trim the spaces on the edges or not, either in the editor or in the final application.
One could add extra functionality to the package, e.g. to trim all leading- and trailing-whitespace from text nodes — but this also means that prettying PDML trees can only be done safely when dealing with trees where text nodes don't span across multiple lines (and when the proposed whitespace-only rule is in place).
What's your advice and guidance on this? how should the syntax handle the whitespace problem.
Also, do you see two potentially different types of the trees when dealing with pretty-formatting (as in JSON) as compared to text nodes spanning across multiple lines?
I guess these type of problems are solved through extension of the Basic PDML, e.g. by introducing specific operators to handle multi-line strings and their whitespace (like for HereDocs, or YAML's use of the pipe | in string blocks).
But right now that we're trying to build some basic editors' support for PDML, we need to address the question in practical terms.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent