alexisvincent/systemjs-tools

(dev)tools for working with SystemJS

SystemJS Tools

systemjs-tools is a collection of powerful, customizable build tools, to help build compelling development and production stories for projects that rely on SystemJS. Think figwheel or webpack, for SystemJS.

Quick Start

Install the cli

yarn global add systemjs-tools

Navigate to your frontend root and initialise the config

THIS STEP DOESN'T WORK YET, YOU NEED TO MANUALLY CREATE THE CONFIG

systemjs init

Start the development server

systemjs serve

Features

systemjs-tools provides you with the following

• Snappy Page Refreshes
• Hot Module Replacement
• Cache Persistence (across process restarts)
• Automatic Configuration and Smart Defaults
• Bundling and Builds
• Contribution Friendly Codebase
• Automatic JSPM Interoperability
• Development Feedback Console
• node_modules package resolution (beta)
• Project Templates
• IDE Analysis Engine

items in bold are unfinished

Motivation

Current development workflows for SystemJS leave much to be desired. Projects end up clobbering together a slow, haphazard subset of their desired workflow and the frustration this causes contributes to developers leaving the SystemJS ecosystem. That's ridiculous, SystemJS provides EXCELLENT primitives to build seamless development workflows and we should be leveraging that. systemjs-tools is my contribution towards tooling for the SystemJS ecosystem, and I would encourage you, if you have the time, to help contribute towards filling the gaps that exist in our ecosystem.

Usage

systemjs-tools exposes a cli tool and a server-side library and has deep integration with systemjs-hot-reloader. Each exposes layered abstractions and hooks for you to describe your unique environment, while still providing a largely automatic experience at all levels.

Client

systemjs-tools uses systemjs-hot-reloader as its frontend client. As such you should follow the instructions listed in the README.

General Usage

systemjs-tools (cli and server) upon initialization, searches upwards for a project root directory (indicated by a systemjs-tools.js file or a systemjs-tools key in your package.json). It then loads the relevant config, describing your environment (eg. baseURL directory and port to serve on).

If you do not already have a config file, navigate to the directory containing your frontend source files and run systemjs init (not ready yet), to create a config describing your project.

Typically one would then run a command such as systemjs serve, to start up a development server.

Setting up trust for the SPDY server

It may be helpful to generate your own localhost certificates for the SPDY server and trust that with your browser. Recently, Chrome and Firefox have begun rejecting certificates that do not specify a Subject Alternative Name. Here is a one-liner to generate your own localhost.crt and localhost.key:

  openssl req -x509 -newkey rsa:2048 -keyout localhost.key -out localhost.crt -days 365 -nodes -subj '/CN=localhost' -reqexts SAN -extensions SAN -config <(cat /etc/ssl/openssl.cnf <(printf "[SAN]\nsubjectAltName=DNS:localhost"))

To use this, specify a configuration within your systemjs-tools.js:

fs = require('fs')

// Specify keys for localhost
module.exports.config.serve.keys = {
  key: fs.readFileSync('localhost.key', 'utf-8'),
  cert: fs.readFileSync('localhost.crt', 'utf-8'),
  ca: fs.readFileSync('localhost.key', 'utf-8'),
}
module.exports.config.channel = {
  keys: module.exports.config.serve.keys
}

API

For an in-depth look at the API, checkout the links below.

1. Config
2. Server
3. CLI

Roadmap (currently usable for development)

• basic development bundling with file busting
• cli - boilerplate generation
• cli - config generation via systemjs-config-builder
• cli - serve wrapper
• promise construct for serialization of builder operations
• rxjs core api (for plugin communication)
• cross session generic cache with file busting
• development mode
• sourcemap support
• hot module replacement
• handlers - [static, bundle]
• basic documentation
• handlers - [named bundle mappings]
• handlers - [compile, http2 server push]
• logging via debug package
• preemptive file loading
• production build story
• hmr - [preemptive sources]
• development console with error catching
• configuration schema and validation (using ajv)
• cli - use config schema to automatically expose options
• generic dependency tree mapping for hmr of things that have their own dependency resolvers (eg. sass and pcss)

systemjs-tools's Issues

proxy support

Would love to use this without having to hack my own server-side JS. I use jspm-dev-server and it has a proxy feature I need. Could this behavior be merged in via optional config?

I use:

--proxy http://localhost:8182 --port 8183 --ignore smc/jspm_packages

Also, I found the --ignore required because of the directory watch overhead scanning the large source subtree I have. To illustrate, my app sources look like this:

smc/jspm.config.js
smc/jspm_packages
smc/dir1
smc/dir2

The jspm_packages directory is so large that watching it for changes uses considerable CPU.

sourcemaps not preserved when serving app bundle

When serving up the app bundle after the app "entries" trigger, the application source maps do not get served. I noticed that the file served by systems serve actually does a wrapper around the generated js, such that //# sourceMappingURL does not appear on the last line.

Example end of generated bundle:

//# sourceMappingURL=data:application/json;base64,eyJ2ZXJzaW9uIjozLCJzb3VyY2VzIjpbXSwibmFtZXMiOltdLCJtYXBwaW5ncyI6IiIsImZpbGUiOiJvdXRwdXQuanMiLCJzb3VyY2VzQ29udGVudCI6W119
})(System, System);

I think the sourceMappingURL has to be the last line of the file or it won't work.

This makes debugging nearly impossible on a large application.

custom cert support (fix deep merge)

I want to do something like this in my systemjs-tools.js:

var fs = require('fs');

  module.exports.config.serve.keys = {
    key: fs.readFileSync('localhost.key'),
    cert: fs.readFileSync('localhost.crt'),
    ca: fs.readFileSync('localhost.key'),
  }

This way, I can customize the localhost key which is being served from some project that does not supply a localhost key with Subject Alternative Names set. When I do this, the file is corrupted by the lib/config processing with merge/conform somehow, causing the cert to become stringified instead of being passed to spdy.createServer() as Buffer form.

Support optional bundle expression config

As discussed regarding performance of rebuilding my large app and retain HMR from issue #14, I decided I really needed to be able to specify a bundle expression as part of the config.

I will submit a PR for this tweak.

Can't install package

Hi,

I'm trying to install systemjs-tools with yarn but get the following error. I'm not sure if this is a problem on yarn's side or systemjs-tools side:

$ yarn global add systemjs-tools
yarn global v0.19.1
warning No license field
[1/4] Resolving packages...
[2/4] Fetching packages...
error An unexpected error occurred: "https://registry.yarnpkg.com/systemjs-hmr/-/systemjs-hmr-0.1.8.tgz: invalid tar file".
info If you think this is a bug, please open a bug report with the information provided in "/home/xxx/.config/yarn/global/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/global for documentation about this command.

Test windows support

Make sure this works with windows

Handlers documentation/support

I need CORS support in my projects, thus I've been trying to enable CORS through the handler API. But so far I've been unable to get it working. It seems that the documentation is a bit ambiguous. Is there a way to get this functionality working right now?

So far I just managed to get this working by manually hard coding CORS to the server.

jspm.config.js not found if in a subdirectory

For whatever historical reason I cannot remember, my project's jspm.config.js location is in a source subfolder. When this occurs, the systemjs serve configuration sniffer can't find it.

I was able to workaround this by using manual builder configuration, but it would be really nice to have this work out-of-the-box.

  builder: {
    configFiles: ['smc/jspm.config.js'],

stale compiled output served by builder on subsequent rebuilds

I am performing the following workflow:

• start with no cache in systemjs-tools
• hit refresh on browser, see the state of the markup (1)
• wait 45s for build to complete
• change application .html file (2)
• witness HMR hotswap in the browser--yeah!
• hit refresh on browser
• wait 5s for build to complete, which is NICE and FAST, but uses old .js and/or .html
• see browser has reverted to remove the change in (2) above, and you will see (1) instead

In looking at this, it seems the _.cache.builder is stale. I tried to evict the .html from _.builder.cache.compile.loads and _.builder.cache.trace, but no dice. It seems keeping the _.cache.builder reuse is broken, which is unfortunate because my app could really use the 5s reload!

Serve doesn't work with Node.js 6 LTS

I noticed that systemjs serve throws an error after a file has been changed.

TypeError: Object.values is not a function

It seems that the support for this ES2017 functionality is added in nodejs 7.x versions, which are not yet LTS. Would it be possible to add a shim or do I just need to update to 7.x or 8.x?

Bundling failed

I am trying to use the "entries" feature to cause the app to bundle. When my entrypoint to my application is hit, I receive the following error:

:: bundling app/main.js...
Unhandled rejection ReferenceError: entry is not defined

This is because the error handler is referring to "entry" when it should refer to "expression":

_.error(`failed to bundle ${expression}`, err)

Support externalized sourcemaps

The builder serves up inline sourcemaps; however, this does not work as well within IntelliJ, and perhaps other scenarios. It would be better to support *.js.map.

What I am seeing in my debugger is that sometimes my app stops in dependencies.js (bundled) instead of the source mapped sources, other times it works fine. The only functional change here is the sourcemaps being inlined.