ballercat/walt

:zap: Walt is a JavaScript-like syntax for WebAssembly text format :zap:

Walt | Alternative Syntax for WebAssembly | Demo

⚡ Walt is an alternative syntax for WebAssembly text format. It's an experiment for using JavaScript syntax to write to as 'close to the metal' as possible. It's JavaScript with rules. .walt files compile directly to WebAssembly binary format.

Highlights:

• Write "close to the metal" JavaScript!
• No C/C++ or Rust required, just typed JavaScript.
• NO LLVM/binary toolkits required, zero dependencies 100% written in JS.
• Fast compilation, integrates into webpack!

📖 Read the Quick Start Guide

🚀 Try it out in the Walt Explorer.

🙏 Contributions are welcomed! Contributing guide.

🐥 Current status: Alpha

Roadmap

Problem

Writing zero-overhead, optimized WebAssembly is pretty tough to do. The syntax for .wat files is terse and difficult to work with directly. If you do not wish to use a systems language like C or Rust, then you're kind of out of luck. Your best bet (currently) is to write very plain C code, compile that to .wast and then optimize that result. Then you're ready to compile that into the final WebAssembly binary. This is an attempt to take C/Rust out of the equation and write 'as close to the metal' as possible without losing readability.

I feel like this is currently a problem. Most Web engineers are not familiar with the C family languages or Rust. It's a barrier for wide spread adoption of WebAssembly. A competent Front-end engineer should be able to edit WebAssembly as easily as any other systems programmer.

Solution

Provide a thin layer of syntax sugar on top of .wat text format. Preferably porting as much of JavaScript syntax to WebAssembly as possible. This improved syntax should give direct control over the WebAssembly output. Meaning there should be minimal to none post optimization to be done to the wast code generated. The re-use of JavaScript semantics is intentional as I do not wish to create a brand new language.

Here is what an example of a .walt module which exports a recursive Fibonacci function looks like:

export function fibonacci(n: i32): i32 {
  if (n <= 0) return 0;

  if (n == 1) return 1;

  return fibonacci(n - 1) + fibonacci(n - 2);
}

When this code is ran through the walt compiler you end up with a buffer which can be used to create a WebAssembly module with a fibonacci export just as you would expect. All done with familiar JS syntax and without any external binary toolkits! A working demo of this exists in the fibonacci-spec.js unit test file.

Project Goals

The ultimate goal of Walt is to make WebAssembly accessible to average JavaScript engineer by providing a subset of JavaScript syntax which compiles to WebAssembly bytecode directly. That WebAssembly should be easy to make use of and simple to integrate into an existing project with the current build tools.

Use cases

Pretty much everyone who wants a quick-start into wasm can use Walt to get there. The use-cases are not specific to this project alone but more to WebAssembly in general. The fact that Walt does not require a stand-alone compiler and can integrate into any(almost?) build tool still makes certain projects better candidates over others.

• Web/Node libraries, looking to improve performance.
• Games
• Projects depending on heavy real-time computation from complex UIs to 3D visualizations
• Web VR/AR
• Anyone interested in WebAssembly who is not familiar with system languages.

See Wiki for detailed design decisions etc.

Prior Art

• wah - A slightly higher level syntax on top of the wasm text format
• mini-c - Experimental C to WebAssembly compiler

Contributors

ballercat	tbroadley	thomassturm	Baransu	whitecrownclown	balajmarius

hlaaftana	ForsakenHarmony	hamlim	petetnt	novoselrok	Dragas

walt's Issues

Array types in function arguments

Overview

Array types should be acceptable function arguments. Currently, we do not have parser support for them.

Expected

A function can be declared with an array type as argument.

Actual

SyntaxError: 
function testa(arr: i32[]): void {
                       ^ Unexpected token Punctuator
Expected: "Identifier"
  at testa (unknown.walt:7:23)
walt.js:3238:12

Acceptance Criteria

• Tests for the above case
• Tests pass

Records Semantics instead of Objects

As strings has nothing to do with the lookup and usage of the actual compiled result of objects I think it's better to use tokens (like in ReasonML) and not mention strings whatsoever

The "Bounce (canvas)" example is broken

Bug Report

Overview

The "Bounce (canvas)" example is throwing the error and anything isn't showing.

Expected

The working example.

Actual

Uncaught CompileError: WasmCompile: Compiling wasm function #7:<?> failed: immutable global #0 cannot be assigned @+673
    at compile (eval at Explorer.compileAndRun (https://ballercat.github.io/walt/dist/explorer.js:51252:24), <anonymous>:21:22)
    at Explorer.compileAndRun (https://ballercat.github.io/walt/dist/explorer.js:51254:23)
    at CallbackQueue.notifyAll (https://ballercat.github.io/walt/dist/explorer.js:14553:22)
    at ReactUpdatesFlushTransaction.close (https://ballercat.github.io/walt/dist/explorer.js:2148:24)
    at ReactUpdatesFlushTransaction.closeAll (https://ballercat.github.io/walt/dist/explorer.js:6095:25)
    at ReactUpdatesFlushTransaction.perform (https://ballercat.github.io/walt/dist/explorer.js:6042:16)
    at ReactUpdatesFlushTransaction.perform (https://ballercat.github.io/walt/dist/explorer.js:2178:32)
    at Object.flushBatchedUpdates (https://ballercat.github.io/walt/dist/explorer.js:2261:19)
    at ReactDefaultBatchingStrategyTransaction.closeAll (https://ballercat.github.io/walt/dist/explorer.js:6095:25)
    at ReactDefaultBatchingStrategyTransaction.perform (https://ballercat.github.io/walt/dist/explorer.js:6042:16)

Example

Object Spread operator

I think this is a low hanging fruit once you have objects and merging work right. Basically a { a: 2, ...obj } should be transformed to merge({ a: 2 }, obj) and there should be no comma after the spread operator

Enable flow-type eslint plugin rules

Goal

Make the codebase more consistent with clear standards.

Overview

Original issue (#18)

Flow-types helps to reason about the code and maintain a stable API. Unfortunately, some files don't have flow, and the types don't match best-established practices, like avoiding any and *Type type names.

Acceptance Criteria

These are the sequence for how these changes should be applied. The only additional eslint rules to run on are listed below.

• Add eslint-flowtype plugin
  • Add flowtype rules, disabled. #32
• Add validation script to travis-ci #32
• Turn on rule type-id-match, fix type names everywhere
• Turn on rule require-valid-file-annotation and add flow to all files.
• Add npm run flow checks to validation script/travis-ci, fix errors.

Semicolon errors

When omitting semicolons (bad practice very common in JS land) the tokeniser tries to understand the expressions and give funny errors. I think it should indicate a semi colon is missing

Is there a mapping between wasm bytecodes and Javascript keywords?

This would allow to exactly know what bytecodes are still missing to be implemented and where contributions can be done :-) Also this would allow to diferenciate what Javascript features would be considered native code and what can be added as sugar-sintaxis or runtime library to have a full Javascript implementation :-)

Binary encode names into Name section

Goal

Improve debugging

Overview

• WebAssembly has a nifty custom section for Names http://webassembly.org/docs/binary-encoding/#name-section
• Chrome can debug wasm pretty well, it's possible to set breakpoints and even examine the stack.
• Runtime exceptions are hard to deal with, stack traces show plain indexes in the binary.

I'm pretty sure that encoding the names, would improve the DX here. Mainly improving runtime exceptions to actually point to correct functions/variable names which are broken. Also debugging a wasm module in the browser is kind of difficult without original names for locals/variables/functions.

Acceptance Criteria

• Add Name section to the emitter
• Name output should be enabled by default but could be disabled via some flag
• Unit tests. A test ensuring a named section is encoded would be ideal.

Unary Negation

Goal

Unary negations -x are not supported. Current work-around is to manually type these expressions as 0 - x instead.

Acceptance Criteria

• Tokenizer currently treats a string -2 as a single TOKEN Constant. Probably not the best idea, this should be two tokens [Operator, Constant].
• Encode -{identifier} as a binary expression 0 - {identifier}
• Test

Bitwise Operators

Goal

Should be able to do bitwise math

Overview

Binary operators are pretty solid now, but bitwise opcodes are not generated currently. Pretty straightforward change. Nothing should change in the parser, might be as easy as adding it to https://github.com/ballercat/walt/blob/master/src/emitter/opcode.js#L262

Acceptance Criteria

• Unit Test every binary bitwise opcode
• Tests pass

visual glitch on bounce

in examples\walt\animation.walt

change
export function onInit(totalParticles: i32): void {
width = 500;
height = 300; // visual glitches appears with a lot of particles
total = totalParticles;

by

export function onInit(totalParticles: i32): void {
width = 500;
height = 400; //match the real canvas height
total = totalParticles;

Validation

Goal

Make it easier to extend the syntax and make compiler easier to work with.

Overview

This should unblock #28

Currently, the individual node parsing methods perform in-place validation on the syntax. For example, the identifier parser may throw an undefined variable syntax error if it cannot identify the variable.

walt/src/parser/maybe-identifier.js

Line 48 in 296f1cb

ctx.handleUndefinedIdentifier(ctx.token.value);

This is OK, but the more syntax is extended the harder it is to extend the behavior. In the above code, the undefined variable handler is overwritten to silence the error while parsing function arguments(while parsing the parent node). Instead of doing the above, let's have another step before the generator, a static type/validator checker.

The final pipeline will look like this:

Tokenizer -> tokens -> Parser -> ast -> Static Checker -> ast/dag -> Generator -> ir -> Emitter -> bytecode

Initial implementation would only handle Identifier nodes. This means the static checker would do the following:

• Ensure the identifier is valid/in-scope
• Patch nodes to attach correct metadata, like global/local indexes, type definitions, function indexes and table pointers
• throw syntax errors if the identifier is undefined and only do so in correct context. For example, it should be fine for an identifier to be undefined in a function definition because that is the first node it's defined in.
• Code generation needs to be moved entirely to its own step. Currently, functions generate IR in-place but this will need to change.

Mixture of es-modules & node modules and flow & non-flow files?

Is this intentional, or are you transitioning from one to the other?

A few questions ....

This is a really interesting idea. As a JavaScript developer, I also got quite frustrated with having to use C / C++ to create WebAssembly. I also dislike the extra 'baggage' that tools like Emscripten introduces to the code you generate.

Anyhow, I was wondering how your solution compare to AssemblyScript? I am guessing that you want to keep it close to the WebAssembly 'metal', so would not plan to introduce objects, classes, etc ... ?

I do wonder whether it would be worth creating your own TypeScript subset for this project? I notice you are using a TypeScript-style type annotation.

Anyhow, good luck with this project!

Closures

🦄

Placeholder.

I think there is a neat way to implement these within WebAssembly, with dynamic side-modules 🦄

Improve styles for Walt Playground/Explorer

Opening an issue to track some work on improving the styles for the interactive playground.

Some initial questions for @ballercat:

• Do you still want the Material Design styles/javascript?
• Any preference on a color theme? Even just a primary and secondary color?

Function Types, Function pointers and Table

Goal

Support custom function type declarations. This would enable imports of JS functions.

Acceptance Criteria

• Type definitions need to be hoisted above imports, per WebAssembly spec.
• Should behave like flow-type syntax. Ex:

import { setTimeout: Later } from 'env';
import { log: Log } form 'console';
type Log = (i32) => void;
type Later = (i32, i32) => void;

function echo() : void {
   log(42);
}

export function demo() : void {
   setTimeout(echo, 200);
}

The above may not be possible to do in vanilla wasm. There doesn't seem to be a type that could be possible for window.setTimeout. This could be done though if the resulting module has a JS runtime wrapper which maps i32 params to a table entry.

This would require a table import

const buffer = walt.compile(src);
const table = /* create table */
WebAssembly.instantiate(buffer, {
   env: {
      table,
      setTimeout(tableEntry, timeout) {
          window.setTimeout(table.get(tableEntry), timeout);
      }
   },
   console: window.console
}).then(result => result.instance.exports.demo());

For the whole demo to work there probably should be a webpack loader to abstract the table bits.

Webpack Loader - MVP

Goal

Allow engineers to import .walt modules into their JavaScript.

Overview

Currently, it's possible to import a .walt file into your webpack project via raw-loader followed by manually compiling the source into a WebAssembly module.

Here is an example, from docs:

walt/docs/src/examples/memory.js

Line 1 in 296f1cb

import code from "./walt/memory";

And here is the very basic webpack config:

const path = require("path");

module.exports = {
  context: __dirname + "/src",

  devtool: "source-map",

  entry: {
    javascript: "./index.js"
  },

  output: {
    filename: "dist/explorer.js"
  },

  resolve: {
    extensions: [".js", ".jsx", ".json", ".scss", ".css", ".walt"]
  },

  module: {
    loaders: [
      {
        test: /\.js$/,
        exclude: /node_modules/,
        loaders: ["babel-loader"]
      },
      {
        test: /\.html$/,
        loader: "file-loader?name=[name].[ext]"
      },
      {
        test: /\.walt$/,
        loader: "raw-loader"
      },
      {
        test: /\.scss$/,
        loaders: ["style-loader", "css-loader", "sass-loader"]
      },
      {
        test: /\.css$/,
        loaders: ["style-loader", "css-loader"]
      },
      {
        test: /\.(png|woff|woff2|eot|ttf|svg)$/,
        loader: "url-loader",
        query: { mimetype: "image/png" }
      }
    ]
  }
};

Instead, there should be a loader which can read, compile and load .walt files a walt-loader.

Acceptance Criteria

• Write a loader which will resolve a .walt import into a factory method returning a WebAssembly module promise, see https://github.com/ballercat/wasm-loader. Maybe chain the two loaders?

BinaryExpressions, Arithmetic

Goal

Iron out bugs in the binary expressions and provide unit tests for it.

Problem

There is some partial support for binary expressions and math but it's a bit buggy. Some of the expressions are not evaluated correctly.

• 2 + 2 * 5 - 10 - Currently this returns -8 when it should be 2.

Seems like a problem with the parsing of the binary expressions(and the minus operator?). Likely the parser is buggy around operator precedence. See https://github.com/ballercat/walt/blob/master/src/parser/Parser.js#L148

( and ) should also be implemented. Most of the code for this is in various stages of completeness.

Acceptance Criteria

• Fix the binary expression bug above
• Implement the rest of the operator precedence for math only. Currently, the parser treats = separately so they do not play into binary expressions.
• Fix the type problem in binary expressions. _We need to know what types are operated on before we can call the appropriate operator(i32Add vs f32Add for example) inside the emitter. Should be the job of the parser to figure this out IMO.
• Write tests, there is an example of a binary expression test here https://github.com/ballercat/walt/blob/master/src/__tests__/compiler-spec.js#L75
• Make sure local/global references are working in these as well.

Port simple heap for closure plugin

Goal

Stabilize closure plugin.

Overview

Built-in closure plugin doesn't manage memory for closures. I wrote a basic simple-heap in JavaScript a long time ago for emulated malloc/free calls. It should be trivial to port it to a walt file.

Acceptance Criteria

• Implement actual malloc and free
• Write some tests

Loops, Switch

Goal

Support loops.

Detail

• Support for loops
• Support do ... while() loops
• Support while ... do() loops
• Support switch ... case branches
• break

Generators

Placeholder Issue

Reference: #9 (comment)

Generics

https://flow.org/en/docs/types/generics/ can be useful when dealing with either i32, i64, f32 or f64 i/o

Dot operator and Identifiers for Object properties.

Goal

Improve consistency, make Walt object types appear more like structs(which they are).

Overview

Original issue (#22)

As of writing, object property access is performed via array subscript operator and string subscripts like so obj['foo']. This is not the ideal long-term solution and causes some confusion about the semantics of objects in Walt.

The long-term solution is to provide a new dot . binary operator which would be a single way to access properties on an object type.

Acceptance Criteria

• The dot . operator which is not currently supported. We would need the tokenizer to recognize it (easy) and for the parser to convert it down to a plain old array subscript.
• Unit tests
• Removing/converting string property lookups, in examples and tests.
• Identifiers as object definitions should work

Lambda Generic

Goal

Avoid custom keyword lambda

Overview

Instead of a lambda keyword, we could use a Lambda<> generic type to wrap regular function types to convert them to a lambda type.

Acceptance Criteria

This syntax should work

type Fun = (i32) => i32;
type Closure = Lambda<Fun>;

function getCosure(): Closure {
  ...
}

• The above works
• Tests

Website ?

Would you like some help with WALT's website ?
Even if the project pre-alpha, I think it could be nice to have a small website for the project, showcasing its features, its docs, etc.
If you think that's a good idea, I'd be happy to help.

Logical Operators

Goal

We should have logical operator support

Overview

Some research needs to be done on how to best encode these

I'm actually not 100% on what the best implementation approach should be here. There are no native logical operators in wasm, probably because it's just all bitwise &s and |s. Problem is, logical operators have a different semantic from binary ops in most language.

const a: i32 = maybeThis() || maybeThat(); encoded into a bitwise or is not enough here. Probably means that logical operators need to be encoded as blocks that return a value.

Acceptance Criteria

• Support binary logical operators
• Logical ops should return one of the operands
• Support short-circuiting
• Test the world
• Tests pass

Implement Parser

Implement enough of the parser to pass the emitter spec https://github.com/ballercat/walt/blob/master/src/emiter/__tests__/function-section-spec.js#L12

Global array declarations

Goal

Allow for global array declarations.

Overview

It's not possible to declare global arrays. See #86

Acceptance Criteria

• Write a test for the use-case
• Pass the test

Data Section

Goal

Implement all sections, enable static strings in the future.

Overview

Data section is not currently encoded. It should be.

Acceptance Criteria

• Data section is encoded
• It is covered with tests

Nested structs and functions within structs

Goal

Allow for nested structs and functions within structs.

Overview

Depends on (#28)

Closures are cool, but to be useful you'd want to return an object/struct with a closure or another struct as a field.

Basically, this syntax should be possible

type lambda Closure = () => i32;
type Struct = {
   increment: Closure,
   decrement: Closure
};
function getActions(): Struct {
  let x: i32 = 0;
  const result: Struct = 0;
  result = {
     increment: (): i32 => { 
        x += 1;
        return x;
     },
    decrement: (): i32 => {
        x -= 1;
        return x;
    },
    getX: (): i32 => { return x; }
  };
  return result;
}

export function test() : i32 {
  const actions: Struct = getActions();
  actions.increment();
  actions.increment();
  actions.decrement();
  // should be 1
  return actions.getX();
}

This should make the syntax much more expressive and allow for a wide range of applications.

Acceptance Criteria

• Allow nested struct types within other structs
• Allow closure types within structs
• Test everything

Support `else if (...)`

Overview

The lack of else if is awkward, makes it difficult to write some code.

A quick solution might be to encode the node as a literal else block containing a single if.

Sample spec

export function test(x: i32): i32 {
   if (x == 0) {
     return 2;
  } else if (x == 1) {
     return 4;
  } else {
     return -1;
  }
}

That should do it.

• Tests
• Tests pass

Split code generation out of the parser.

Goal

Unblock #49

Overview

There are a few parts of the parser which envoke the generator directly. This is OK, but it's making it difficult to implement a few things. There aren't a ton of areas of code where this happens, but they should all be cleaned up. In the very early days, the generator was baked into the parser and everything(parsing + generation) happened in one pass, it still kind of does. We should fix it.

Here is a search which should point out all the problem areas

https://github.com/search?utf8=%E2%9C%93&q=generate++user%3Aballercat+repo%3Aballercat%2Fwalt+path%3Asrc%2Fparser+language%3AJavaScript&type=Code&ref=advsearch&l=JavaScript&l=

Problem areas

This change should be rather easy to make, but there might be a few issues.

• Function types are generated and referenced in this manner. Not generating function types on the fly will likely break a few things. Something to watch out for.

Acceptance Criteria

• Move all generator calls out of the parser
• Have parser spit out the AST ONLY
• Have generator walk the AST and spit out IR
• Make sure tests pass

Memory, Arrays, sizeof

Goal

Provide the ability to declare and use non-trivial data stored in memory.

TODO

• Memory built-in type and parser
• Custom user built-in types
• Array syntax, parser, generator
• sizeof method

Overview

Memory

Memory import and declaration is explicit.

sizeof

sizeof function is compiled to a static size of a type/array in bytes

sizeof(i32); // 4
sizeof(i64); // 8

Arrays

Flow array shorthand should be supported

let foo: i32[] = 0

import { memcpy: MemCopyType } from 'utils';

type MemCopyType = (i32, i32, i32) => i32;

function test(): void {
   const a: i32[] =  0;
   const b: i32[] = 100;
 
   a[0] = 1;
   a[1] = 2;
   a[2] = 3;
   // memcpy like function can be employed
   memcpy(b, a, sizeof(a));
}

Comments support

Goal

Support comments in source code

• //
• /* */

Problem

The tokenizer does not skip comments, it should. These won't be compiled into final binary but can be useful for debugging purposes.

Function calls, branches

Goal

Function calls are not supported nor are if/then statements. Implement this.

Acceptance Criteria

• Implement function calls in expressions ex return 2 + returnsTwo(); === 4
• Implement function arguments
• Implement comparisons
• Implement if/then branches
• Test everything with a recursive fibonacci example

export fibonacci(n: i32) : i32
{
    if (n == 0) 
    {
        return 0;
     } 
     if (n == 1) {
           return 1;
     }
     return fibonacci(n - 1) + fibonacci(n - 2);
}

Module to Module Imports

Goal

Enable importing between two walt modules.

Overview

I'm pretty sure this should be possible with the loader. File1 has the line import { foo: FooType } from './foo', where ./foo is another walt source File2.

Function imports/exports

Function imports can be regular old import object entries which are exports from another module. That may or may not have a callilng wasm-to-JS, but I think it will. Using shared Table object with shared entries would be awesome.

main.walt:

import { addTwo: AddTwoType } from './foo';

export test(): i32 {
  return addTwo(2);
}

foo.walt:

export addTwo(x: i32): i32 {
   return x + 2;
}

Constants

Constants are just that and wasm does not have mutable scalar exports(pretty sure). Nothing special should just exist on the imported object.

Acceptance Criteria

The whole thing should just work out of the box. The loader should resolve the other modules when it sees a /. in the import. The user should also be able to overwrite the imports via importsObject argument.

Destructuring Assignment

Goal

Allow users to use destructuring assignment.

Overview

Destructuring assignment is pretty helpful and should be implemented. Destructured variables should become locals. Destructured variables should not need type annotation as they can be inferred, so that is super useful.

Acceptance Criteria

• const [x, y] = arrayObject;
• const {x, y} = vectorObject;
• Type annotations for destructured locals are not necessary
• Tests

Some issues with arrays and memory

Hi there!

I have tried out Walt compiler, and I have some questions.
The code, that I have written and run via Node.js

const compileWalt = require('walt-compiler').default;

const buffer = compileWalt(`
import {memory: Memory} from 'js';

let offset: i32 = 0;
//let array: i32[] = 0;

export function init(off: i32) {
    offset = off;
}

export function main(): i32 {
 
  let sum: i32 = 0;
  let i: i32;
  let array: i32[];
  array = offset;

  for(i = 0; i < 10; i+=1) {
    sum += array[i];
  }

  return sum;
}
`);

require('fs').writeFileSync('bin.wasm', new Uint8Array(buffer));

const mem = new WebAssembly.Memory({ initial: 10 });
const importObj = {
    js: {
        memory: mem
    }
};
WebAssembly.instantiate(buffer, importObj).then(({ instance }) => {
    const offset = 100;
    const array = new Uint32Array(mem.buffer, offset, 10);
    for(var i = 0; i < 10; i++) {
        array[i] = 33;
    }
    instance.exports.init(offset);
    console.log(`Main result: ${instance.exports.main()}`);
});

All the code above works as expected, but there are some issues.

• First of all, I found out that it's impossible to declare arrays as global vars. So as you may see I declare an let array: i32[] inside the function main, and the same global declaration is commented out, since with the global declaration and without the local one, there is an error thrown by the compiler. Is it a bug? Or not implemented yet?

• Secondly, as I see the import {memory: Memory} from 'js'; is compiled to (import "js" "memory" (memory $js.memory 1)). Ok, but what if I want to import a memory object with 10 pages? I'm not sure, maybe it will work fine with such import even if I pass a memory object with 10 pages (as I actually do in the code above) and if I work with the memory beyond 64*1024 offset, but I would like you to clarify this moment.

• Thirdly, I found out that if I declare a counter inside the for loop, to wit for(let i: i32 = 0; ... , an error occurs. Is it a bug or it's not implemented yet? The same for ++ operator.

Also, I have seen inside the binary .wasm file and I saw that you count the address of an array element via something like that:

get_local $l2 // array initial offset
get_local $l1 // index
i32.const 4 // since it's i32 
i32.mul // index*4
i32.add // arrayOffset + index*4

It's sane, but I have a suggestion. You can substitute the multiplication with the bitwise shift, which probably will be faster, that is to say substitute arrayOffset + index*4 with arrayOffset + index << 2. What would you say?

Built in Closure plugin into loader

Goal

Make it easy to use .walt files

Overview

A user should not have to pass in an importsObject with a closure plugin by default.

Acceptance Criteria

• Add a compile-and-link option/function to the compiler
• Add closure plugin handling to the loader
• Write a test

Documentation: Getting started.

Documentation is slim, the following needs to be done to help with that:

• There need to be some "Getting started" documentation.
• There needs to be an npm module for simple installation
• raw-loader with the compiler example, like in the /docs

Error when using i64 values in functions

Overview

I get a compiler error when trying to use i64 type values in functions (assigning them to variables, or returning from functions).

When I try to compile this:

const x: i64 = 42;

everything works fine.

But, if this line is in a function:

function test(): void {
	const x: i64 = 42;
}

I get the following error:

CompileError: wasm validation error: at offset 27: type mismatch: expression has type i64 but expected i32

Roadmap?

Hi. This seems like a great project!

What would you think about adding a roadmap or something in the readme about your goals? Might help contributors know where you want to go and where you don’t.

Eg, if the language you’re after isn’t really JavaScript but a version of it, maybe single line comments on multiple lines are fine. 😊

Imports of undefined type should throw

Overview

There is undefined behavior(heh) when importing functions which do not have a matching type definition in the file. Nothing breaks though, it seems like the function gets a random(?) function index assigned to it. I'm not sure how this is working, but this should be a syntax error.

Types are one of the first sections of a WebAssembly module. When we import from the environment we only have an identifier for the type; it get's written as a placeholder to the type index. Once we encounter the type definition, we patch it. When we don't though, we encode the placeholder type, whatever that may be.

Expected behavior

Undefined types should through

Notes

The work for this is slightly tricky since we never parse the tokens twice. And this should be part of a stand-alone sanity check/validation once we are done parsing all of the tokens. This could be the first validation. So far these usually end up in runtime errors.

Add binary debug snapshot to compiler specs

Goal

Improve DX, make testing easier and more stable.

Overview

There are currently a ton of tests ~200. The most useful so far are ones at the entire compiler level. These integration-like tests are good for catching breaks in logic while developing new features. However, it's hard to track down what is actually wrong in the final binary output. Most issues show up as off-by-one errors in the binary, like wrong type indexes or function indexes. While refactoring the parser I found myself using online diff tools to figure out the issues. We might as well use the built-in snapshot in ava to do the same thing.

Acceptance Criteria

This is a pure addition to specs and does not require new logic in the compiler.

• Add snapshots to all existing full compiler specs.

See docs source-code for example use of debug function

walt/packages/walt-docs/src/app.js

Line 67 in de4dc71

const wasm = debugWalt(this.intermediateRepresentation);

Optimizer

Goal

Create an optimizing pass.

Overview

Go nuts? There is zero static optimization currently, everything written is emitted.

Dead code elimination sounds great. AssemblyScript does it, which is dope.

Acceptance Criteria

• It works
• It's tested
• It's opt-in and configurable

npm distribution of compiler version mismatch

The file walt/packages/walt-compiler/dist/walt.min.js is significant older than the non-minified version of the compiler.

The npm download of the compiler does not currently function since it seems several method names don't even exist in the minified version of the file.

Language Feature Document

Goal

A user should be able to read through the features of a language.

Overview

Kind of hard to point others to specs etc., when trying to list out language features. Some of the early documentation is out of date on the wiki. We need an MD-file doc with all features clearly documented.

One thing of note is that, where possible, we can omit documentation for 1:1 JavaScript features. Like object access.

Acceptance Criteria

Think basic language doc, topics should be in the following order

• Basic Types
• Available Math operators
• Control flow (list missing, switch)
• Function declaration
• Custom Function types
• Exports, Imports
• Walt custom types Table, Memory
• Arrays and memory access
• User-defined structs
• Closures

Bonus points

• Document what the syntax above is compiled to in WebAssembly opcodes!

Expose memory.grow, memory.size, memory.dataLength

Goal

Unblock work for a full-blown allocator.

Overview

memory_grow are not available through walt, but can be implemented as members of the Memory and Table types.

Acceptance Criteria

This should work

const memory : Memory = { initial: 1 };

function test() {
  memory.size();
  memory.grow();
  memory.dataSize();
}

• Memory.grow()
• Memory.size() current size of memory
• Memory.dataSize length of the data section, always zero until data senction encoding is implemented.

Support numbers inside identifiers

Bug Report

Overview

Numbers are not supported as part of identifiers. Looks like it was omitted from the tokenizer, should be supported.

Expected

Identifiers like test1 should be valid

Actual

SyntaxError: 
function test1(arr: i32[]): void {
             ^ Unexpected value
Expected: (
  at test (unknown.walt:7:13)

Example