elixirscript/elixirscript

Converts Elixir to JavaScript

ElixirScript

The goal is to convert a subset (or full set) of Elixir code to JavaScript, providing the ability to write JavaScript in Elixir. This is done by taking the Elixir AST and converting it into JavaScript AST and then to JavaScript code. This is done using the Elixir-ESTree library.

Documentation for current release

Requirements

• Erlang 20 or greater
• Elixir 1.6 or greater (must be compiled with Erlang 20 or greater)
• Node 8.2.1 or greater (only for development)

Usage

Add dependency to your deps in mix.exs:

{:elixir_script, "~> x.x"}

Add elixir_script to list of mix compilers in mix.exs Also add elixir_script configuration

  def project do
  [
    app: :my_app,
    # ...
    # Add elixir_script as a compiler
    compilers: Mix.compilers ++ [:elixir_script],
    # Our elixir_script configuration
    elixir_script: [
        # Entry module. Can also be a list of modules
        input: MyEntryModule,
        # Output path. Either a path to a js file or a directory
        output: "priv/elixir_script/build/elixirscript.build.js"
    ]
  ]
  end

Run mix compile

Examples

Application

ElixirScript Todo Example

Library

ElixirScript React

Starter kit

Elixirscript Starter Kit

Development

# Clone the repo
git clone [email protected]:bryanjos/elixirscript.git

#Get dependencies
make deps

# Compile
make

# Test
make test

Communication

#elixirscript on the elixir-lang Slack

Contributing

Please check the CONTRIBUTING.md

elixirscript's Issues

Better handle inner modules

Testing out inner modules and realized there are some issues. The biggest being they export the default object as well creating more than one export default in a file. Same thing goes for the __MODULE__ definition. Need a new way of handling the __MODULE__ translation, and probably need to put inner modules as a property on the outer object. There should only be one export default per file.

Actors, Processes, etc

Always thought of web workers in part for this, but they don't exist in node.js. But there will be web workers in io.js nodejs/node#180. So if the plan stays the same and web workers are used, then actors will work at least on the web and in io.js for the time being.

Translate interpolated strings to JavaScript

This will for sure end up being a template string in JavaScript. I'm thinking about using template strings for strings in general to support multiline strings as well.

Automatic module detection

Currently the modules have to be defined up front for use since every module is an ES6 module.

The idea here would be to go through the module and find uses of other modules within a module. A map would be used to hold the found modules uses to prevent creating multiple imports of the same thing (it wouldn't hurt anything since ES6 modules are singletons). "Module use" would be defined as a call to another module's function(s). When a module is found and it is not in the map, it will be added, and the call ast will be updated to reflect the import. When all are found, aliases will be created for them all and placed with the rest of the imports.

An example

defmodule Test do
  Hello.World.hi()
end

Would be turned into

defmodule Test do
  World.hi()
end

Where Hello.World is saved into a map for making an alias later.

This will turn into the following JavaScript

import World from 'hello/world';
const __MODULE__ = Erlang.atom('Test');

World.hi();

Returning from functions

Everything in Elixir is an expression and functions return the value of the last expression. Would have to figure out how to mimic this in JavaScript. My first thought is to iterate through the ast and wrap the last expression in a return statement.

GenEvent

Protocols

I don't have any solid ideas right now on how to implement protocols. Looking for any suggestions.

Translate unquote to JavaScript

Translate __ENV__ to JavaScript

Create Mix task to output escript client

This will allow for the escript to be used in projects.

Translate __DIR__ to JavaScript

Using ES6 symbols for atoms

I've been back and forth on this one. First they were symbols (before I understood how symbols worked) and not they are a custom class.

After reading about Symbol.for, I'm thinking they could be useful as symbols again. One of my concerns was how to get the description from out of the symbol to return in the to_string function. Right now the answer isn't pretty, but basically using a substring on the toString result of the symbol to get the description out should be enough.

Using symbols would play a lot nicer with a lot things as well. One of them being keys in objects.

GenServer

Few questions regarding current test suite

After taking a look at the current implementation, why we use Atom('ok') instead of just const and we use List(1, 2, 3, 4, 5) instead of an array literal? [1, 2, 3, 4, 5] ?

(function(){
        if(Kernel.__in__(number, List(1, 2, 3, 4))){
          let value = 13;
          return value;
        }else{
          return true;
        }
      }.call(this));

should be

(function(){
        let list = [1, 2, 3, 4];
        if (list.indexOf(number) >= 0) {
          let value = 13;
          return value;
        } else {
          return true;
        }
}.call(this));

Also, why we explicitly call this function with this?

Translate try to JavaScript

I started some work on this. Try is interesting because of the purpose of the different clauses.

From Elixir's docs:
The rescue clause is used to handle exceptions, while the catch clause can be used to catch thrown values. The else clause can be used to control flow based on the result of the expression. Catch, rescue and else clauses work based on pattern matching.

With regards to rescue and catch, one question is does it make sense to support both? The other question is what to do for each one.

My initial thoughts are the rescue can be used to catch errors raised from ElixirScript. These are just structs and they are easy to match one. For example:

Elixir

try do
  UndefinedModule.undefined_function
rescue
  UndefinedFunctionError -> nil
end

can be turned into

try{
    UndefinedModule.undefined_function()
}catch(e){
    if(Kernel.match({'__struct__': Atom('UndefinedFunctionError')}, e){
        return null;
    }else{
        throw e;
   }
}

The throw at the end would be to make sure that if nothing matches that the error is rethrown.

Catch can be used to catch non-ElixirScript errors (i.e. JavaScript errors).

try do
  throw(SomeJavaScriptError)
catch
  :throw, :SomeJavaScriptError ->
    IO.puts "sample thrown"
end

try{
    throw SomeJavaScriptError();
}catch(e){
    if(e instanceof SomeJavaScriptError){
        return IO.puts('sample thrown');
    }else{
        throw e;
   }
}

Else may also be a bit tricky. My only thoughts are to include that clause within the try clause.

So something like this:

x = 2
try do
  1 / x
rescue
  ArithmeticError ->
    :infinity
else
  y when y < 1 and y > -1 ->
    :small
  _ ->
    :large
end

can turn into this

let x = 2;
try{
  let _ref = 1/ x;
  if(_ref < 1 && _ref > -1){
    return Atom('small');
  }else{
    return Atom('large');
  }
}catch(e){
    if(Kernel.match({'__struct__': Atom('ArithmeticError')}, e){
        return Atom('infinity');
    }else{
        throw e;
   }
}

After seems straightforward and just be implemented as a finally clause in JavaScript.

The current pattern matching algorithm may have to be updated to work with the matching in the rescue and catch clauses.

Updating pattern matching

I have been updating the pattern matching implementation and I figured it's best to write an issue for it.

#74

I'm replacing the pattern matching. There is a JavaScript library called funcy that can create pattern matching functions. I forked it here and updated it to suite the needs of the project and added some tests as well. I have been playing with the idea of writing a JavaScript pattern matching library influenced by it, but for now, this will due. The reason I wanted to create a new one is for making something that could be used for binding variables.

The good thing about adding this is it simplifies most of the implementations and I feel more confident that this will work better. I have updated most of the implementations except for the try expressions.The match? implementation needs to be updated as well.

Agent

Maybe implementing the Agent module is the best way to implement updating mutable state?

Reduce size of escript

RIght now it's at 103MB and something feels wrong about that. I'm sure there must be a way to reduce it. Could also be the reason why it's currently taking a while to create the escript right now.

Make Example Project

Make an example project. Since this is targeting es6, a workflow with babel is going to be needed. I will probably make a phoenix project with gulp and jspm . This should be a good test of workflow, interoperability with js libraries, and help add more to documentation.

Translate cond to JavaScript

Will probably end up being represented with if..else if..else statements.

Add __MODULE__ to all modules

__MODULE__

will be turned into

const __MODULE__ = Symbol('module name') 

This will be added to modules right after imports.

Translate __CALLER__ to JavaScript

Investigate translating either Erlang Abstract Format or Core Erlang to JavaScript

I would like to take some time to investigate translating either the Erlang Abstract Format or Core Erlang to JavaScript.

Using the :elixir module, the Elixir AST can be translated into the Erlang Abstract Format. My goal initially was to get to a Core Erlang representation, but maybe the abstract format is a good target. I don't know much about it right now so I'll have to do some reading

Core Erlang may still be an option. If so, it would probably be better to add some PRs to LuvvieScript instead. Then we could update ElixirScript to basically convert Elixir code to Core Erlang and add whatever else is needed.

The idea for translating further down would be so that some features (for instance macros) wouldn't need to be implemented and would probably just work. Features such as pattern matching would also probably be easier to implement as well. Most of the effort would be on implementing features of the Beam VM in JavaScript and less on implementing features of Elixir.

Translate receive to JavaScript

Translate bitstrings to JavaScript?

Question mark for figuring out if bitstrings should be supported. If so, need to figure out what they would be their equivalent in JavaScript.

Add into clause to for comprehensions

Implement standard library

My initial plan is to write the equivalent in JavaScript as modules and include them when converting Elixir to JavaScript.

The awesome path would be to just be able to use the standard library as it already exist today. I don't know right now how complicated that would be, but it could be a stretch goal.

Handling multiple arity of functions

So I read up on how clojurescript handles arity
http://blog.fogus.me/2011/07/21/compiling-clojure-to-javascript-pt1/ (that's from 2011 so it might be an old implementation)

I thought about copying that but with a slight change since the target is ES6.

So in Elixir this:

def example() do
end

def example(one, two) do
end

def example(one, two, three) do
end

Would end up being something like this in JavaScript

function example_0(){}
function example_2(){}
function example_3(){}

export function example(...args){
    switch(args.length){
        case 0:
            example_0.apply(this, args.slice(0, 0 - 1));
        case 2:
            example_2.apply(this, args.slice(0, 2 - 1));    
        default:
            example_3.apply(this, args);            
    }
}

Any thoughts?

Translate unquote_slicing to JavaScript

ESTree.Tools.Builder.identifier/1 not found

Hi,

When trying to process the example "Lions.Tigers.Bears" with version 0.6.5 I get this error:

** (UndefinedFunctionError) undefined function: ESTree.Tools.Builder.identifier/1 (module ESTree.Tools.Builder is not available)
    ESTree.Tools.Builder.identifier(:Bears)
    (elixir_script) lib/elixir_script/translator/utils.ex:102: ElixirScript.Translator.Utils.make_member_expression/3
    (elixir_script) lib/elixir_script/translator/utils.ex:67: ElixirScript.Translator.Utils.make_call_expression/3
    (elixir_script) lib/elixir_script/translator/module.ex:59: ElixirScript.Translator.Module.make_module/2
    (elixir) lib/enum.ex:977: anonymous fn/3 in Enum.map/2
    (elixir) lib/enum.ex:1261: Enum."-reduce/3-lists^foldl/2-0-"/3
    (elixir) lib/enum.ex:977: Enum.map/2
    (elixir_script) lib/elixir_script.ex:65: ElixirScript.transpile_path/2

Apparently I only have identifier/2 in my code path.

iex(1)> ESTree.Tools.Builder.identifier [tab]
identifier/2

I have estree 2.0.0 and inflex 1.4.0 installed.

Do you have any pointers that could help me track this down?

Translate case to JavaScript

Revisiting Maps

right now maps are turned into objects and struct are turned into classes. I'm thinking maybe both can be turned into ES6 Maps.

Implement map/struct update

Add way to update mutable JavaScript items

This would be useful for times when one needs to actually update an existing JavaScript variable. Already implemented a module named Mutable with a function named update, but maybe if it's just one function it can be placed in another module.

Performance

The performance is not the greatest right now. Using erlang_js, the tests did seem to run a bit faster, but I ran into issues when using it outside of the project (couldn't find erlang_js_drv.so). It may just be something I have to figure out and eventually get working.

I do like using node though. I haven't used ports much, but maybe connected to node via a port willbe a better solution than what is currently used.

Translate for to JavaScript

Just from testing initially, I don't think translating to a for..of statement is right or will be enough. Array.prototype.map may be wrong too.

Macros

So I think the simple idea is to expand the everything and parse that.

Translate module attributes to JavaScript

my first thought is to translate them to const variables.

Add pattern matching for bitstrings

Bitstrings are the last data structure without pattern matching support (unless I missed something else). I'm not sure how to handle matching on sizes of parts of the bitstring.

Translate quote to JavaScript

Name?

Any suggestions?

I think everyone so far assumes "ElixirScript", but I'm not 100% sold on that. It may end up being the name though.

I like "ExScript" but a quick Google search showed there is already a project with that name. I also like "ElixirJS".

Handling Elixir identifiers with characters that are not allowed in JavaScript identifiers

In Elixir you can create variables or function names such as the following:

allowed?
allowed!

In particular using a ? or a !. And there are some other examples as well. Using ? or ! aren't allowed in JavaScript unless you don't use the dot operator

user.allowed? //Error
user["allowed?"] //Ok

Using that syntax with variables isn't an option. In #57, I have done some work so far in calling functions using the bracket notation and updated the stdlib functions with ? and ! in them to use the name from Elixir (I was originally omitting the ? and ! from the names).

The question is how to handle invalid characters in variable and function names?

One option is to throw an error when the translator sees a variable or function name with any invalid JavaScript characters.

Pattern Matching

Figure out how to mimic pattern matching in JavaScript.

Translate super to JavaScript

http://elixir-lang.org/docs/stable/elixir/Kernel.SpecialForms.html#super/1
http://elixir-lang.org/docs/stable/elixir/Kernel.html#defoverridable/1

Data Structures

Need to figure out how to represent the data structures. The main concern is whatever is chosen, they are able to be destructured. Whether that means using some of the destructing features in ES6 or implementing some destructing feature ourselves We have:

• Tuple
• Bitstring
• List
• Map
• Struct
• Keyword
• Dict
• Set

I've gone back and forth on tuples. What I'm currently working on has tuples going from this in Elixir

{ value1, value2}

to this in JavaScript

{'0': value1, '1': value2 }

The reason being so that they look different than lists (currently both lists and tuples are turned into arrays) and to allow destructing using the new feature in ES6. Looking on any feedback on this change.

I still haven't figure out how bitstrings would carry over.

Maps and Structs are difficult too. Currently, maps are turned into objects and Structs turned into classes. The idea being that when you defined a struct, you could have default values and that is taken into account. I would prefer to have maps and structs both be turned into the same thing (maybe objects). If that were to happen would need a way to take into account default values set on structs. If they are both turned into objects then that makes destructuring for both an easy task.

Figure out how to handle types and specs

This isn't a priority but I would like to support both eventually.

On Imports and Exports

One of the things I've uncovered while creating test projects is handling imports from a parent directory. In my local repo I made it so prepending Parent to an import, alias, or require statement would add ../ to the import path. I'm not a fan of it, but it's an idea.

What I think will end up happening is keeping functionality similar to how it is now, but adding a check for a from: clause on import and alias. from: would take the path of to the import if so desired.

import Values.Colors, from: "../values/colors"

turns into

import * as Colors from "../values/colors";

Without the from clause, it would just build the path starting from the current directory

import Values.Colors

turns into

import * as Colors from "values/colors";

My only issue with the change would be that it would not compile with Elixir. I think that's fine as there will be references to javascript libraries that don't exist in Elixir's scope anyway. The alternative is to make a macro and that may still be a solution. some like:

import ElixirScript, only: :macros
import_from Values.Colors, from: "../values/colors"

As I'm writing this, I'm starting to like that a bit more, but I'm not a complete fan yet. It may be something to look into down the road if sharing modules between Elixir and ElixirScript is more of a thing.

Another thing I've noticed is that a lot of javascript libraries return an object as their default export. I think it makes sense to do the same here. For example

defmodule Values.Colors do
    def red() do
    end
    defp some_private_function() do
    end
end

into this

function red(){
    return null;
}

function some_private_function() {
return null;
}


let Colors = {
    red: red
}

export default Colors;

This would make it follow the pattern that seems to be occurring with both ES6 modules and node modules.

The other change I thought about making is to the alias implementation. The idea would be to make it function how require currently functions, and add the from clause like above. In this scenario, alias and require would be interchangeable. I think that's ok.

Following the example for imports this would be how it would function.

alias Values.Colors, from: "../values/colors"
alias Values.Colors
alias Values.Colors, as: Pizza

turns into

import Colors from "../values/colors";
import Colors from "values/colors";
import {default as Pizza} from "values/colors"

Any feedback would be appreciated.

Implement Logger with console.log

Using erlang_js instead of node

I created a branch using erlang_js. Which would remove the node dependency. It sort of works right now, but the problem is that when erlang_js starts, it writes it logs to standard out. So it works when using it stand alone, but if you were to use it in something like gulp, it causes issues.

Also currently running into issues when running it as an escript.

Update for comprehensions to work with bitstrings

This would probably have to wait until #51 is figured out.