facebook/regenerator

Source transformer enabling ECMAScript 6 generator functions in JavaScript-of-today.

regenerator's Issues

new Expressions not supported

The following input causes regenerator to abort:

function *_() { new String(yield 1) }

with

Error: unknown Expression of type "NewExpression"

Provide a pluggable Browserify transform

As suggested on Hacker News by shtylman:
https://news.ycombinator.com/item?id=6594727

Clarify the license of /runtime file

Should I add LICENSE and PATENTS files in the root of any project where I am using regenerator and include /runtime file from it inside project code? When I use /runtime file it's also brings this license text:

This source code is licensed under the BSD-style license found in the
LICENSE file in the root directory of this source tree. An additional grant
of patent rights can be found in the PATENTS file in the same directory.

which is not valid due I don't have original LICENSE and PATENTS files

Hoist function declarations to the beginning of their enclosing blocks

Background: an important first step when transforming a generator function is to turn all declarations into variable assignments, then declare those variables in the scope of the outer function, so that their values can persist between invocations of the generator.

This applies to function declarations, too. For example,

function *gen() {
  yield foo();
  function foo() { ... }
}

should become

function gen() {
  var foo;
  return wrapGenerator(function *(context) {
    foo = function foo() { ... };
    yield foo();
  }, this);
}

The problem right now is that we don't move the foo = assignment to the beginning of the block, so foo is undefined when we try to call it.

how to include the runtime for regenerated builds?

if i have a node package i want to regenerate, what is the suggested way to include the runtime? the easiest way is to include it with every file.

bin: Add an option that prepends the runtime to the output

The binary could use an option that prepends the runtime to the output. Something like this, perhaps:

regenerator es6.js > es5.js # es5.js cannot be used on its own; it needs the runtime

regenerator --include-runtime es6.js > es5.js # es5.js now includes the runtime and can be used on its own

This would boil down to prepending runtime/dev.js to the output if the --include-runtime flag (or whatever you would prefer to call it) is present.

Add more implementation comments

Especially: explain what I mean by "leaps."

Defining $ctx in a generator function breaks conversion

For example:

function *range(max, step) {
  var $ctx = {};
  var count = 0;
  step = step || 1;

  for (var i = 0; i < max; i += step) {
    count++;
    yield i;
  }

  return count;
}

var gen = range(20, 3), info;

while (!(info = gen.next()).done) {
  console.log(info.value);
}

console.log("steps taken: " + info.value);

Implement a live-updating in-browser sandbox for playing with the transformer

Probably want to use https://github.com/marijnh/codemirror like http://facebook.github.io/react/ does.

The generated code should also be executable in a console.

There should also be a button to submit a one-click bug report if you notice something wrong.

cc @petehunt

When PhantomJS is available, use it to run test/tests.browser.js

If I'm not mistaken, Travis CI provides phantomjs by default in the $PATH of the test environment.

Generators should be allowed to be named yield

The following code does not transform as expected:

function* yield(undefined) {
    return yield yield void typeof null instanceof this;
}.call(Function);
x=yield(yield);
x.next();
x.next();

Consider using context.dispatchException for both static and dynamic exceptions

Right now we have different code paths for handling static throw statements (see LeapManager.prototype.emitThrow) versus handling uncaught/dynamic exceptions.

The dynamic dispatch code should be able to handle static throw statements as well. That is, instead of emitting code to jump directly to the enclosing catch through any intervening finally blocks, we could just emit an actual throw statement which would be caught by the runtime and dispatched with the same semantics.

Static throw statements are pretty rare in actual code, so the possible loss of performance from exiting/reentering the function is really not worth worrying about.

The maintainability advantage of relying on one code path instead of two should be obvious.

Fix the sandbox page in IE8

IE seems to be choking on defaults.null and similar expressions in ast-types, so probably just need to switch to defaults["null"] and bump the dependency version.

Support the arguments object

In order to support generator functions that access the arguments object, we need to save a reference to the arguments of the outer function in a freshly-named variable, and then rename all free references to arguments in the inner function.

It's tempting to just store the arguments object as context.args, but I'm not sure that would preserve the parameter aliasing behavior of arguments (i.e., if you modify arguments, you're also modifying the actual parameters passed to the function, and vice-versa).

How can I detect for GeneratorFunctions in code?

So existing code out there that I've seen checks the constructor.name property of the object to see if it's a GeneratorFunction:

function isGeneratorFunction(obj) {
  return obj && obj.constructor && 'GeneratorFunction' == obj.constructor.name;
}

Currently, regenerator-transpiled code fails this test, since the constructor property is still set to the Function object.

Can we rectify this somehow? Ideally making the above test function pass as-is, so that existing generator code out in the wild doesn't need to be tweaked.

Problem with object literals containing multiple yield expressions

The following code does not transform as expected:

var get = async(function *(){
  var left = readJSON('left.json')
  var right = readJSON('right.json')
  return {left: yield left, right: yield right}
})

With two yields in the same expression, they both get the same value.

Runtime: Object returned by wrapGenerator should have "[object Generator]" string representation

Both FireFox and V8 generators implementations have this behavior:

function *test(){ yield 1 }
// Note: FireFox support generators without "*" before function name

var gen = test();
gen.toString() === "[object Generator]";// true !!!

It's allow to check is object is Generator or not.

Pull in as much of the Mozilla generators test suite as possible

https://github.com/mozilla/mozilla-central/tree/master/js/src/tests/ecma_6/Generators

Thanks to Mike Shaver for this suggestion.

Infinite loop in .throw()

The following call to .throw() never returns (or throws).

function *gen(){
   yield 1;
}
var it = gen();
var one = it.next();
it.throw("inifinte loop");
console.log("I never get run!");

Rename context parameter to guarantee it doesn't capture any free variables

The only new variable introduced in the scope of the inner generator function is the context or $ctx parameter, and while it might be acceptable to say "just don't name any variables context or $ctx," it would really be ideal to synthesize the name of the parameter such that we can be completely confident of its freshness.

The great thing about having the context parameter and knowing that it doesn't collide with any other variables is that the generated code can then add arbitrary properties to it without any concern for scope pollution.

In Node v0.11, execute tests natively using --harmony-generators

That is, in addition to transforming test/tests.es6.js into test/tests.es5.js, we should run the tests natively using the following command:

mocha --reporter spec --harmony-generators test/tests.es6.js

This will ensure that we maintain fidelity with the V8 implementation of generators, and (perhaps most importantly) allow Travis CI to verify fidelity in Node v0.11 with every push.

Does no install with npm command

Here is the error when "npm install grunt-regenerator --save-dev"

npm ERR! git clone git://github.com/ariya/esprima.git Cloning into bare repository '/Users/jorgegarridooval/.npm/_git-remotes/git-github-com-ariya-esprima-git-9077e517'...
npm ERR! git clone git://github.com/ariya/esprima.git 
npm ERR! git clone git://github.com/ariya/esprima.git fatal: unable to connect to github.com:
npm ERR! git clone git://github.com/ariya/esprima.git github.com[0: 192.30.252.130]: errno=Operation timed out
npm ERR! Error: Command failed: fatal: unable to connect to github.com:
npm ERR! github.com[0: 192.30.252.130]: errno=Operation timed out
npm ERR! 
npm ERR! 
npm ERR!     at ChildProcess.exithandler (child_process.js:637:15)
npm ERR!     at ChildProcess.EventEmitter.emit (events.js:98:17)
npm ERR!     at maybeClose (child_process.js:735:16)
npm ERR!     at Socket.<anonymous> (child_process.js:948:11)
npm ERR!     at Socket.EventEmitter.emit (events.js:95:17)
npm ERR!     at Pipe.close (net.js:466:12)
npm ERR! If you need help, you may report this *entire* log,
npm ERR! including the npm and node versions, at:
npm ERR!     <http://github.com/isaacs/npm/issues>

npm ERR! System Darwin 13.0.0
npm ERR! command "node" "/usr/local/bin/npm" "install" "grunt-regenerator" "--save-dev"
npm ERR! not ok code 0

ES5 doesn't require quoted/bracketed IdentifierNames that are also Keywords

generator["throw"] = function(exception) {

This isn't necessary if you're targeting ES5 runtimes.

Don't bother parsing or transforming when source contains no generator functions

Thanks to the requisite function * syntax, it should be very easy to detect when a string of code does not contain any generator functions:

if (!/function\s*\*/.test(source)) {
  return source;
}

Fix shadowing behavior of catch parameters

The parameter caught by a catch block is supposed to shadow any variables of the same name in the outer scope, as if the catch block introduced a new scope.

Currently, setting the catch parameter just overwrites any outer variables.

Probably want to define the parameter as a context property and then rename free references to it within the catch block.

Implement a build process using Grunt or something similar

Off the top of my head, here are the items that need building:

• test/tests.es6.js to test/tests.es5.js
• test/tests.es5.js to test/tests.browser.js
• runtime/dev.js to runtime/min.js (and maybe update size claims in README.md automatically)
• automatically copying mocha.js and mocha.css from node_modules/mocha so we don't have to check those files in

Simple try/catch causes compiler to abort

The following:

function *_() {
    try {
        yield 1;
    } catch(e) {
    }
}

fails with:

AssertionError: null === {"isGlobal":false,"didScan":true}
emit.js:668  assert.strictEqual(catchScope.lookup(catchParamName), catchScope);

Support source maps for generated code

The best place to implement source map support is probably in https://github.com/benjamn/recast, which is a dependency, but I wanted to have an issue here so that people know source maps are on the roadmap.

Add an appropriate LICENSE file

cc @jamesgpearce @daveman692

Remove dependency on browserify

We don't need browserify for anything except running the test suite in a browser, and it pulls in a ton of dependencies that slow down npm install considerably.

tag releases in git please

pretty pls

Runtime: wrapGenerator.isGeneratorFunction does't work in IE and after minification

The current isGeneratorFunction based on non-standard Function#name property. This is not working in IE 11 or lower and after code has been minified.

1. IE11 and lower doesn't support Function#name according ECMAScript extensions compatibility table
2. Function#name will be changed after minification. You can verify this by running runtime/min.js and trying to use wrapGenerator.isGeneratorFunction. You will get failure result.

Integrate with Travis CI

Should be able to test in all major Node versions (v0.6, v0.8, v0.10, v0.11).

Not sure if this is possible until the repository is public, but it's an absolute must once we start taking pull requests.

Also want to put the little green "build:passing" badge in README.md.

function*(){} fails to compile

The following fails to parse:

echo 'function*(){}' | ./bin/regenerator /dev/stdin

with:

Error: Line 1: Unexpected token (

Track transformation performance to prevent regressions (and validate optimizations)

Maintain path-to-root information throughout emitter.explode

Using https://github.com/benjamn/ast-types/blob/master/lib/node-path.js of course.

Losing catch scope in nested generators in try-catch

function *foo () {
  try {
    b;
  } catch (e) {
    yield function* () {
      alert(e);
    }
  }
}

g = foo();
bar = g.next();
bar.value().next()

Expected: ReferenceError: b is not defined
Actual: undefined

Support let/const block binding via https://github.com/olov/defs

This synthetic example demonstrate the issue:

function *test() {
    let a = 3;

    {
        let a = 1;
        yield a;
    }

    {
        let a = 2;
        yield a;
    }

    yield a;
}
// Should be: [ for(x of test()) x ] == [1, 2, 3]

Regenerator doesn't differentiates let/const and var, and translates it all into var which is wrong translation in many cases.

I believe that regenerator should throw an Error if code contains let or const. At least until regenerator will support block binding.

Another solution is to added block binding into regenerator. I suggest awesome @olov defs project. I use it in my project es6-transpiler and it works perfectly.

Sandbox bug report

The following code does not transform as expected:

function *add($ctx, $abc) {
  yield $ctx + step;
}

Should probably add a rename for variables named $ctx to avoid clashes.

Do a better job of pruning dead code

Tools like UglifyJS can do a lot of this, but we should try harder upstream, too.

We already attempt to prune obviously unreachable code in Emitter.prototype.getDispatchLoop (see isSwitchCaseEnder and alreadyEnded), but I disabled that temporarily because that extra cleverness made it harder to verify the generated code.

Problem with npm install

I freshly cloned regenerator, ran npm install and got the following error:

npm ERR! Error: No compatible version found: ast-types@'>=0.3.15-0 <0.4.0-0'
npm ERR! Valid install targets:
npm ERR! ["0.1.1","0.1.2","0.1.3","0.1.4","0.2.0","0.2.2","0.2.3","0.2.4","0.2.6","0.2.7","0.2.8","0.2.9","0.2.10","0.2.11","0.2.12","0.2.13","0.2.14","0.2.15","0.2.16","0.2.17","0.2.18","0.2.19","0.2.20","0.2.21","0.2.22","0.2.23","0.2.24","0.2.25","0.3.0","0.3.1","0.3.2","0.3.3","0.3.4","0.3.5","0.3.7","0.3.8","0.3.9","0.3.10","0.3.11","0.3.12","0.3.13"]
npm ERR!    <whatever>
npm ERR! node -v v0.10.15
npm ERR! npm -v 1.2.18

Did I miss something?

yield* does not have correct return value

The following code does not transform as expected:

function* foo() {
  yield 3;

  return yield* bar()
}

function* bar() {
  yield 3;

  return 4
}

function runGen(gen) {
  var o = gen()
  while (true) {
    var res = o.next()
    if (res.done) return res.value
  }
}

console.log("foo", runGen(foo))
console.log("bar", runGen(bar))
window.baz = foo()

Running bar directly gives me the expected result of 4 from the second call to the bar generator.

Running foo directly gives me the wrong result undefined. This is because yield* bar() returns undefined, not the return value of bar

yield* is not working

The following code does not parse:

function* g() {
  yield 1;
  yield 2;
}

function* g2() {
  yield* g();
}

Nested catches with the same variable cause compiler to abort

function *_() {
    try {
    } catch(e) {
        try {
            yield 1;
        } catch(e) {
        }
    }
}

No code emitted for empty while loops

Empty while loops with simple conditions seem to get lost in translation:

function *bug(x) {
  while (x) {

  }
}

The generated code is essentially the same as it would be for an empty generator function:

function bug(x) {
  return wrapGenerator(function($ctx) {
    while (1) switch ($ctx.next) {
    case 0:
      return $ctx.stop();
    }
  }, this);
}

next(value) call to newly created generator should throw TypeError

Equivalent V8 issue https://code.google.com/p/v8/issues/detail?id=3099
Firefox implements proper behavior.

Relevant es-discuss post: https://mail.mozilla.org/pipermail/es-discuss/2014-January/035766.html

This will require reconsidering the check function of the test harness.

How can I detect for Generator objects in code?

Currently I use

function isGenerator(obj) {
  return obj && Object.prototype.toString.call(obj) === "[object Generator]"
}

To detect a generator object which works in harmony but not in regenerator.

Is there an or clause I can add to feature detect generator objects from regenerator or should I duck test for .next() and .throw() ?

This is similar to #48

yield as an expression is not working

The following code does not parse

function* g() {
  var x = (yield 1) + (yield 2);
  return x;
}

.throw() throwing in the wrong place?

If I understand correctly, in the following it.throw should be caught by the try/catch.

function *gen(){
    try {
      yield 1;
    } catch(e) {
      console.log("I caught you!");
   }
}
var it = gen();
it.throw("Catch me if you can");

Parentheses unnecessarily required in || expression

The following code does not transform as expected:

function* fn(){
  var y = x || yield 10;
}