livelykernel/lively.modules

JavaScript package and module system for interactive development

This repository has moved to https://github.com/LivelyKernel/lively.next/tree/master/lively.modules

lively.modules's Issues

Module state recorder does not allow to (re)define certain globals

E.g. var name = "foo" as a top level variable throws an Error. This is b/c the module.recorder objects actually inherit from the global object, window in the browser. The global (at least in browsers in nodejs it sees to work) has a guard that certain names are not allowed to be redefined. This guard even applies when used in an object inheriting from the global, hence the problem.

update module dependencies in code with side effects

export var i = 0;
function inc() { i = i+1 }

Running inc should
a) update the exported value for i (what you get when running System.import(...)) and
b) update all dependent modules that import i

This is currently not working b/c we trigger re-exports using a evaluationDone() call at the end of module executions (compiled in as part of rewriting) and evals via lively.vm.

Update instances when class definition changes after module reload

When using modules in workspaces such as in the example below, changes to the foo method are automatically applied to the instance a in the second workspace when evaluating the whole workspace content.

However, this does not work when just working with modules. In particular, the combination of moduleWrite and reloadModule only reloads modules that depend on the updated module but instances of classes will still use the old method definition.

As a workaround, it is possible to simply re-evaluate the entire module with lively.vm but it seems this should happen automatically with reloadModule.

persistent module state

How can a module keep and manage its own state persistently while being updated and reloaded.

Example: lively4's components module needs to keep track of its templates and prototypes...

new module creation + import issue

Steps to reproduce:

1. in existing module foo.js add import for non-existent module import Bar from "./bar.js"
2. create bar.js with export default class Bar {}
3. return to foo.js. expected: Bar is defined but it isn't right now

imported modules are not updated in workspace modules

Contents of workspace:

import foo from "foo.js"
foo.bar()

After changing the bar in "foo.js" and reloading it, the workspace still executes the old bar method.
Did we use the wrong method to update a module? We currently use lively.modules.reloadModule(urlToFooJS);.

manual import via doit doesn't work for non-packages

When evaluating an import statement such as import { h } from "virtual-dom"; from a module that is not contained inside a lively package, the import will not resolve the module correctly, i.e. h in the example will be undefined.

Config code:

"systemjs": {
    "map": {"virtual-dom": "./node_modules/virtual-dom/dist/virtual-dom.js"},
    "meta": {"node_modules/virtual-dom/dist/virtual-dom.js": {"format": "amd"}}
}

It seems that the import statement doesn't use the parent module for lookup:
await System.normalize("virtual-dom") => http://localhost:9001/virtual-dom
vs.
await System.normalize("virtual-dom", lively.modules.module("lively.morphic/index.js").id) => http://localhost:9001/node_modules/lively.morphic/node_modules/virtual-dom/dist/virtual-dom.js

"fetch" should not be captured

Invalid invocation error: __lvVarRecorder.fetch(__lvVarRecorder.lively4url + src)

Works fine: fetch(__lvVarRecorder.lively4url + src)

It seems that "fetch" should not be called as a method.

Transform fails for `export default 3;`

Transform for export default 3; is failing with an 'unexpected token' error -- will investigate deeper. It works for const X = 3; export default X; though...

Dealing with new exports of module

When adding a new export to a module, moduleSourceChange is not able to propagate the new export to dependencies correctly b/c it does not trigger a complete reload, only updates dependent module setters*. We should either detect that case automatically and reload or make module instances volatile by hooking into System.newModule to allow module modifications later.

*well, and re-executes the dependent modules, in turn triggering multiple re-exports...

System notifier for module and package events

As @levjj pointed out in #20 we need a notification mechanism for changes of the module, package, and System state. Main use for the notifications would be tools that depend on lively.modules, e.g. when having multiple browsers open, one browser changes the definition of a module then the other browsers (might) need to update their views.

notify.js has a first implementation of a notification system currently this is only used for module changes change.js.

Let's make a list of use cases from which we can deduct what functionality the notifier needs to include.

1. What kinds of events do we need? notifying about doits, module (un)loads, package loads, ...?
2. What would a good interface to the notifier look like? How can notifications be submitted, how can dependents subscribe? There is currently a custom (un)subscribe mechanism which has the advantage of using ids for listeners so duplicated subscriptions can easily be avoided. Alternatively we could switch to nodejs-like event handlers that are available to us cross-platform via lively.lang/events.js.

Opinions?

/cc @tylervernonsmith @merryman

class statement as expression does not work....

var bar = class Foo {

}

Original:

Error:SyntaxError: https://lively-kernel.org/lively4/ContextJS/Layers.js: Unexpected token (749:52)
      747 | }
      748 | __lvVarRecorder.LayerableObject = LayerableObject;
    > 749 | __lvVarRecorder.cop.COPError = class __lvVarRecorder.COPError {
          |                                                     ^
      750 |     constructor(message) {
      751 |         this._msg = __lvVarRecorder.msg;
      752 |     }
        at Parser.pp.raise (https://lively-kernel.org/lively4/lively4-core/src/external/babel-browser.js:65149:13)
        at Parser.pp.unexpected (https://lively-kernel.org/lively4/lively4-core/src/external/babel-browser.js:66379:8)
        at Parser.pp.expect (https://lively-kernel.org/lively4/lively4-core/src/external/babel-browser.js:66373:33)
        at Parser.pp.parseClass (https://lively-kernel.org/lively4/lively4-core/src/external/babel-browser.js:65991:8)
        at Parser.pp.parseExprAtom (https://lively-kernel.org/lively4/lively4-core/src/external/babel-browser.js:64582:19)
        at Parser.eval [as parseExprAtom] (https://lively-kernel.org/lively4/lively4-core/src/external/babel-browser.js:67755:22)
        at Parser.pp.parseExprSubscripts (https://lively-kernel.org/lively4/lively4-core/src/external/babel-browser.js:64413:19)
        at Parser.pp.parseMaybeUnary (https://lively-kernel.org/lively4/lively4-core/src/external/babel-browser.js:64394:19)
        at Parser.pp.parseExprOps (https://lively-kernel.org/lively4/lively4-core/src/external/babel-browser.js:64340:19)
        at Parser.pp.parseMaybeConditional (https://lively-kernel.org/lively4/lively4-core/src/external/babel-browser.js:64322:19)
        at Parser.pp.parseMaybeAssign (https://lively-kernel.org/lively4/lively4-core/src/external/babel-browser.js:64289:19)
    Error loading https://lively-kernel.org/lively4/ContextJS/Layers.js
Line:undefined Col: undefined
Source:undefined
Error:null

@j4yk

Superclass not correctly initialized in modules with circular dependencies

See https://github.com/LivelyKernel/lively.modules/blob/circular-module-superclass-bug/tests/instrumentation-test.js#L97-L106

When class Bar gets initialized in module b.js class Foo isn't defined yet and as such Bar inherits from undefined = Object. Circular dependencies work fine when imported definitions can are accessed via module bindings "later", when defining a class we need to have its superclass at load time, however...

We might need to fix this with defining a "forward declaration" (a named function that gets initialized later) for classes as part of the class transform of https://github.com/LivelyKernel/lively.ast/blob/master/lib/class-to-function-transform.js#L165. Downside of this is that makes the class transform more complicated.

Add a Windows CI integration

This project does not work on Windows. I think it would be great to use CI to try to improve portability, especially when for most developers getting access to the platform may be extra work (so developers don’t have to bother with trying to fix win32 support until after a CI build fails at which point they can boot up a VM or whatever).

Appveyor is a nice travis-like CI, if you are willing to integrate with Appveyor I would be willing to write an appveyor.yml as a PR. Please just request.

package.json is not checked for validity

I forgot a coma and my package.json and got the weirdest issues... after loading the module a second time.

Automatically re-capture circular reference exports

If we load module1 first then moduleEnv("module1").recorder.y will be undefined, not 3.

module1:

import { y } from 'p2'
export var x = 3;

module2:

import { x } from 'p1';
export var y = 2;

export default async function foo() does work any more in a module

export default async function env(env, args) {

export default calass Foo() is not rewritten

Lazy definitions

Entities in the top-level such as class and object definitions currently get re-defined on moduleSourceChange and reloadModule operations. Often modifications of those objects that keep the original identity reflect the users intentions better.

• identify objects whose identity should be maintained
• for those objects, implement an update strategy*
• use those updates via options, e.g. options.resetEnv = false in reloadModule could signal to keep objects, maybe a more explicit behavior is better

*The tricky part are deleted sub-elements such as methods and properties for classes. Given that an object or class can be extended in multiple places, we cannot simply remove all props/methods that don't occur in the latest class / obj definition. What's needed is to track the location of these sub-elements and compare them with the location of the re-definition to figure out if a prop/method should be removed.

Including lively modules bundle as test dependency

When importing the lively.modules bundle lively.modules-with-lively.vm.js as dependency, we get the following error, because acorn is undefined.

Chrome 51.0.2704 (Windows 10 0.0.0) ERROR
  Uncaught TypeError: Cannot read property 'plugins' of undefined
  at http://localhost:9876/src/external/lively.modules-with-lively.vm.js:10705

(We include the bundle via script tag for testing)

allow binding of "this" in runEval

return lively.modules.runEval(str, {targetModule: "workspace://1", context: this}).then( x => x.value)

}

Improving compile performance

Compilation time of modules takes way too long, more than 30-45secs for a bootstrapped lively.morphic.

A first step to address that was added in commit 5406707 that introduced an IndexDB-based cache for module translation when lively.modules is used browser side. Loading lively.morphic fully cached now takes ~4secs.

We plan to improve that further by adding a customized transpiler to SystemJS (instead of "plugin-babel"). An example of the setup:

import * as babel from "https://cdnjs.cloudflare.com/ajax/libs/babel-standalone/6.17.0/babel.js";

var myTranspiler = {
  translate: function(load, traceOpts) {
    console.log(`translating ${load.name} [${load.metadata.format}]`);
    load.metadata.format = 'register';
    return babel.transform(load.source, {plugins: ["transform-es2015-modules-systemjs"]}).code
  }
}

System.set("my-transpiler", System.newModule(myTranspiler))
System.config({transpiler: "my-transpiler"})

By using either completely our own lively.ast-based translator or using a customized babel that only adds compilation steps required by the platform (feature detection) we should be able to improve compilation speed further.

Remove methods from class prototype when changing class definition

As pointed out in #13, methods and accessors are not actually removed from classes when the class definition changes. It may seem like an edge case but there have been multiple confusing situations caused by out-dated methods and accessors being called after renaming them in the class definitions. This also causes problems with a method got deleted with the intention of calling the superclass. In that case, leaving the out-dated method prevents the super call.

As a side-note, this could be problematic if classes could be extended but as long as there are no mixins, it should be fine to simply remove the method or accessor from the prototype with delete.

constructor without super call not removed

bug reports in german!

1. subclasse mal einen morph und überschreibe den constructor ohne super zu rufen. 2. speicher das module. 3. lösche den constructor wieder. 4 -> jetzt wird, obwohl der constructor gar nicht mehr überschrieben sein sollte, der fehlerhafte constructor immer noch aufgerufen

how can packages be used?

We tried out:

lively.modules.importPackage('https://lively-kernel.org/lively4/lively4-dummy-a/package.json')

and got the error:

lively.modules-with-lively.vm.js:24878 Uncaught (in promise) TypeError: System.getConfig is not a function(…)

that pointed to:

   var packageInSystem = System.getConfig().packages[packageURL] || {};
    if (!packageInSystem.map) packageInSystem.map = {};

transform source with babel in runEval

runEval("2**3") should work

Update systemjs once global loader issue is fixed

Revert a73cf29 once systemjs/systemjs#1276 is fixed.

standalone does not work (under relative path)

we checked out and installed this package (npm install)
and hosted its files statically under:

https://lively-kernel.org/lively4/lively.modules/examples/browser-standalone/index.html

This pages seems to fail, because the System.js configuration is not suited for this case?

How can we load lively-modules standalone or with a running System.js installation in Lively4?