thefrontside / platformscript Goto Github PK

5.0 7.0 1.0 464 KB

Bring your YAML to life

TypeScript 100.00%

yaml deno lisp structured-concurrency platform platform-engineering

platformscript's Introduction

PlatformScript

PlatformScript is a declarative, statically typed, secure, and embeddable programming language that uses 100% pure YAML as its syntax.

https://pls.pub

Dive right in

deno install -A https://pls.pub/pls

platformscript's People

Contributors

Stargazers

Watchers

Forkers

wooodhead

platformscript's Issues

binding expressions for function invocation

One of the nice things about PlatformScript function invocation syntax is that we get to have "extra" keys of the invocation. For example:

$(x): "%($x.say), %({ $capitalize: $x.to })"
key1: ignored
key2: also ignored

We can use these keys as $let style binding expressions that enable you to compute subexpressions scoped to the function body.

$(x): "%($say), %($to)"
say: $x.say
to: {$capitalize: $x.to}

Quote Syntax.

There are some cases where you want to have PlatformScript not evaluate a data structure. Consider the following snippet of Open API specification

responses:
  '200':
    description: The response
    schema: 
      $ref: '#/components/schemas/User'

How would we represent this as data in PlatformScript so that we could do things like make it the return value of a function? We cannot do it currently because evaluating the following function:

createResponses():
  responses:
  '200':
    description: The response
    schema: 
      $ref: '#/components/schemas/User'

would raise a ReferenceError: $ref is not defined. This is because normal PlatformScript evaluation rules would dictate that the mapping$ref: '#/components/schemas/Users' is a call to a function named ref passing the string argument '#/components/schemas/Users'.

We need some way to say "don't evaluate stuff", just read it as a raw PlatformScript values, and return it.

Following LISP implementations, the answer is to have a quote form that evaluates to the raw argument. It is represented as a single quote operator '. Thus the expression (sum 1 2 3) evaluates to 6, but the expression '(sum 1 2 3) evaluates to a list containing the symbol sum and the integers 1, 2, and 3.

We can replicate this with our own ' function, so that the createResponses function would be represented as:

createResponses():
  $':
    responses:
    '200':
      description: The response
      schema: 
        $ref: '#/components/schemas/User'

But this is not the end of the story. What if we want to transform this response and actually say that some parts of the data structure should be evaluated, but others should be left alone. To see why, let's consider our createResponses() function again. It's doubtful that it would be useful in this form because we are hard-coding all of the structure, but in reality, we would want to do things like parameterize the description and entity name so that we could call it like:

$createResponses:
  description: find a user by id
  entityName: User

To make this happen, we'd want to define the function with a variable substitution:

createResponses(options):
  $':
    responses:
    '200':
      description: $options.description
      schema: 
        $ref: '#/components/schemas/%($options.entityName)'

but this won't work because we told PlatformScript not to evaluate anything! Again, this is well trodden territory when it comes to LISP. It has the mechanisms of quasiquote and unquote. And it uses the ` and , symbols.

So, `(sum 1 ,(sum 1 1) 3) would evaluate to a list with the symbol sum followed by the integers 1, 2, and 3 because the ,(sum 1 1) tells the interpreter to evaluate the result of this and plug it back into the tree.

By the same token, we can introduce a quasi quote function ` and an unquote function , that can be used to turn on / turn off evaluation in PlatformScript. Our createResponses function could then look like.

createResponses(options):
  $`:
    responses:
    '200':
      description: {$,: $options.description}
      schema: 
        $ref: {$,: '#/components/schemas/%($options.entityName)' }

the quasi-quote and unquote functionality from LISP is equivalent to JavaScript String templating's `` and ${} except instead of producing strings, it produces syntax trees. As such, this syntax can (and will) be used for macros.

Importing Quoted data

What if you have a file on disk or a open api spec that is sitting at a URL that you would like to work with like https://example.com/open-api-spec.yaml? we'd like to be able to just say: "import this as a module, but don't bother interpreting it because I'm going to be transforming it for some purpose" What would that look like? Here are some possibilities:

separate import

This has a separate function $import' (import quoted) for importing modules as quoted PlatformScript.

$import:
  transform: https://pls.pub/x/open-api-gen.yaml
$import':
  spec<<: https://example.com/open-api-spec.yaml

$transform: $spec

parameterized module spec:

Currently, the value of a module mapping is a string corresponding to a url, but this proposal would allow it to be parameterized in order to pass additional attributes describing how the module is to be loaded. In this case, we would add a "quote" option to tell platformscript to just load the module.

$import:
  transform: https://pls.pub/x/open-api-gen.yaml
  spec<<:
    url: https://example.com/open-api-spec.yaml
    quote: true

$transform: $spec

other?

What are other possibilities to tell PS to just read the value, and not interpret it?

Learning

concise writeup of quoting and quasi quoting in scheme https://courses.cs.washington.edu/courses/cse341/04wi/lectures/14-scheme-quote.html
More involved examples of quoting and quasi quoting in Racket https://docs.racket-lang.org/guide/qq.html

Nested Template literals

Our first implementation does not support nested literals. It should.

E.g.

"Goodbye, %( "Cruel %("World")")!"

Functions do not deserialize properly.

There is a bug in print() where the following PS:

greet(thing): Hello %($thing)!
greeting:
  $greet: World

prints as:

greet:
  greet(thing): Hello %($thing)!
greeting: Hello World!

It puts the name of the function as a key in the hash, but then repeats it again, whereas it should just put the body.

Installing `pls` binary without Deno

We love Deno, obviously, but we don't want to force users to have Deno installed to be able to use the pls binary. We'll use this issue to track limitations that prevent us from allowing users to install pls binary directly.

Language Server

MVP should

callout:

bad references.
bad module imports.
bad syntax.

support:

jump to definition
documentation-at-point
completions-at-point

Better template syntax

Sadly, % is a special YAML character for registering tags, and that means that you have to put quotes around every single string that uses them in first position.

(person)=>: "%($person.first) %($person.last)"

This destroys one of the most useful features of YAML, which is that making strings is really easy. Instead, we should choose a template syntax that lets us drop the quotes. Here are a couple of suggestions.

(person)=>: << $person.first> << $person.last>
(person)=>: (= $person.first) (= $person.last)
(person)=>: (% $person.first) (% $person.last)

Template Literals

Right now every string has a set of holes that can be plugged by a reference. Instead, we need the holes to be aribtrary platform script expressions:

$let:
  say: "Hello"
  to: "world"
$do: "Hello, %({$capitalize: $to })"

A string with any number of %() expressions in it should parse not as a PSString, but as a PSTemplate that evaluates to a string.

`pls` executable should output YAML, not JSON

Currently, when you evaluate a platform script program, it outputs JSON-ish stuff. Instead, it should output YAML.

We should have a print() function that prints a PSValue to a string. This is what should use when we evaluate a platform script.

a bare bit of yaml as a module should evaluate to itself.

If I have a file hello.yaml:

hello: world

It currently evaluates to the PlatformScript value false

$ pls ./hello.yaml
false

Instead, it should evaluate to the yaml itself.

$ pls ./hello.yaml
hello: world

Improved function syntax

The current function syntax $(arg): body is not sufficient for two reasons:

it is confusing because $ is the sigil reserved for dereferencing (which is used in function invocation), but the definition of a function is not a dereference.
Only being able to define function properties in terms of an anonymous function is cumbersome and prone to right-ward drift.

For example, to define an <AboutCard> react component, it looks like

<AboutCard>:
  $(props):
    $<>:
      - $<span>: Hello
      - $<span>: Goodbye

The first change, would be to use ()=> for anonymous functions:

<AboutCard>:
  (props)=>:
    $<>:
      - $<span>: Hello
      - $<span>: Goodbye

This clearly marks that there is no de-referencing happening, but rather this is a static value.

The second is the ability to define a function property in a single place:

<AboutCard>(props):
  $<>:
    - $<span>: Hello
    - $<span>: Goodbye

Open Questions

We've talked about using rest options mappings to provide local bindings as a stand-in for destructuring (#15). E.g.

greet:
  (person)=>: Hello %($name)
  name: "%($person.first) %($person.last)"

Would this be possible with "method syntax" where you have multiple function properties in the same map? At first appearance it does not seem so. You could always use $let/$do but then you have a tension of "do I want a convenient function body", or "do I want a convenient function declaration syntax?"

If we were to define this same function using method syntax:

greet(person):
  $let:
    name: "%($person.first) %($person.last)"
  $do: Hello %($name)

Is there a way to have it both ways?

Document modules and `$import` syntax.

Key follow-on to #70 and very doable now that there is a website.

Is `rest` a good name for extra fn mappings?

When invoking a function as a map, you can pass "modifiers":

$call: argument
mod1: "hello"
mod2: "world"

Right now these are represented as rest in the PSValue for a function, but this relates more the the implementation than anything else in the sense that we "spread out the keys of a map when looking if something is a function call":

let [first, ...rest] = map.entries();

What should the platformscript terminology be for this quite unique capability. "keys", "modifiers", "options"

Represent `null` value

null is a valid both as a YAML value, and also as a key in a YAML mapping, but we don not support it.

https://yaml.org/type/null.html

I was hoping to avoid null entirely, but it looks like maybe we can't since lots of YAML in the wild contains null mappings. E.g. this example from the Score documentation

apiVersion: score.dev/v1b1

metadata:
  name: backend

containers:
  container-id:
    image: busybox
    command: ["/bin/sh"]
    args: ["-c", "while true; do echo Hello $${FRIEND}!; sleep 5; done"]
    variables:
        CONNECTION_STRING: postgresql://${resources.db.username}:${resources.db.password}@${resources.db.host}:${resources.db.port}/${resources.db.name}

resources:
  db:
    type: postgres
    properties:
      host:
      port:
        default: 5432
      name:
      username:
        secret: true
      password:
        secret: true

In it, resources.db.host, and resources.db.name are both null literals. These properties are equivalent to saying:

host: null
name: null

host: ~
name: ~

As much as I would like to not have null in PlatformScript at all, if we want all YAML values to be valid platform script, as well as to deserialize and serialize back all wild programs, we will need to support it.

I would love for someone to prove me wrong.

improved module syntax

After working with PlatformScript for some more complex tasks, it is apparent that the module syntax and semantics are suboptimal.

Import Syntax

Most modules have imports, but we currently have 2n+1 lines added to the source file where n is the number of imported modules. So importing from three modules will result in seven lines of preamble. We can do better, and cut in half the number of lines dedicated to managing dependencies.

single named import:

$import:
  suspend: https://pls.pub/std.yaml

multiple imports from the same package:

$import:
  suspend, useContext: https://pls.pub/std.yaml

* import (all properties of a module bound to a value)

$import:
  std<<: https://pls.pub/std.yaml

before

$import:
  - names: [suspend]
    from: https://pls.pub/std.yaml
  - names: [useBackstage]
    from: https://pls.pub/x/backstage.yaml
  - names: [GITHUB_TOKEN]
    from: ./secrets.yaml

after

$import:
  suspend: https://pls.pub/std.yaml
  useBackstage: https://pls.pub/x/backstage.yaml
  GITHUB_TOKEN: ./secrets.yaml

Module Semantics

Symbols

Right now the module has a list of "symbols" and a "value" which are two different things. So the following module:

x: 1
y: 2

has symbols x, and y, i.e. things that can be imported into other modules, but its value is false. This is extremely counter intuitive, and to consume all the symbols as a single value is awkward and requires an accompanying $do statement which is how you define the "value" of a module. So to get "both" x, and y, you would have to define your module like this:

x: 1
y: 2
$do: {x: $x, y: $y }

Otherwise it forces your consumers to do this work which is even worse;

$import:
  x,y: https://pls.pub/x/mymod.yaml

together: {x: $x, y: $y}

Also, it is currently impossible to do something like transparently importing an entity defined with a function. Instead, you have to put it in a "pass through" property which is just a throw-away name. Consider defining a Backstage entity with a defineEntity function.

$import:
  defineEntity: https://pls.pub/x/[email protected]/mod.yaml
entity:
  $defineEntity:
    kind: Component
    meta:
      name: artist-lookup

To consume this, you have to import the "entity" symbol from the module.

$import:
  entity: my-entity.yaml

The proposal then, is to just get ride of the "value" of a module, get rid of the $do syntax at the end of the module, and the value of the module is the value of the module, period. You can import a single symbol with the above import syntax, or get the entire value with the <<name binding. So the above becomes:

$import:
  defineEntity: https://pls.pub/x/[email protected]/mod.yaml
$defineEntity:
  kind: Component
  meta:
    name: artist-lookup

That "merges" the entire entity onto the module definition.

Potential Edge Cases

What do we do with a scalar module value? Like what about the the module defined as:

# hello.yaml
Hello World

This clarifies that the "merge import" name<< is not really a "merge" in the sense of the << native YAML merge key. Instead it is a "give me the whole module value"

$import:
  message<<: ./hello.yaml

If this is defined as a module:

$let:
  to: World
$do: Hello %($to)!

Then the value of the module should be Hello World

Scoping

So that modules can define functions, the module's value is its own scope. So that keys which are already evaluated are in scope when subsequent keys are defined:

to: World
msg: Hello %($to)!

should evaluate to:

to: World
msg: Hello World!

that way, functions inside a module can call each other.

greet(thing): Hello %($thing)
value: {$greet: World}

Conclusions

modules do not have a set of symbols and a distinct value
imports can be named or all-inclusive, and can all be specified on a single line per-module
it is an error to make a named import on a module whose value is not a map

Add tests for `pls` executable

The executable is currently broken with a compilation error. While we should fix that, we should also make sure that it is not allowed to break again.

Migrate to `yaml` npm package

In order to support documentation quick-views inside the language server, and for improved stack traces, we should use the modern, more actively maintained YAML parser. https://eemeli.org/yaml/#comments-and-blank-lines

I tried using this package before and it may require some fixes to make it work with Deno.

Embed external values

We need to represent elements of the platform such as react components, servers, etc inside scripts.

Stack traces

When reducing a syntax tree into an value, PlatformScript does not currently have the concept of tracking where exactly the node being evaluated was from in the event of an error, and furthermore, if that node was called from another part of the tree, it does not have the concept of maintaining the context of where the code being called was called from. This means it does not have the concept of an evaluation stack, and as we all know, debugging without stack trace is nearly impossible. It requires you mentally map any symbols you see in the error message back to where you think those same symbols appear in the source. This mental mapping process works for small scripts, but fails spectacularly for complex projects. In short, we need good stack traces.

PlatformScript needs to handle the following errors gracefully:

Evaluation Errors

Most errors fall into this category. They are what happens when you get a bad reference, or a function body raises an error. However, as a scripting language we need to also handle cases where errors cross the boundaries of a native (JavaScript) function. For example:

How does a native function raise a platform script specific error (like no such reference).
What happens when a native function raises an unhandled, non-platformscript error?
How does a native function catch an error that is thrown from platformscript code that it is calling, evaling, or loading?
What dose a native functions entry in a stack trace look like?

Syntax Errors

Syntax errors are slightly different in that they represent a failure to parse a piece of YAML. As such, they don't have as specific a line number, and they don't have an associated stack. However, when loading modules, there is the possibility that the syntax of the module is bad, and so we need to be able to preserve the stacking context of the code that loaded the module.