alexisvincent / systemjs-config-builder Goto Github PK
View Code? Open in Web Editor NEWGenerate SystemJS config files from node_modules
Generate SystemJS config files from node_modules
Hey @alexisvincent nice libray! Should be nice if the repo has LICENSE file
I followed the brief installation directions:
Then I got:
Unhandled rejection TypeError: Cannot read property 'name' of null
at ~/.config/yarn/global/node_modules/systemjs-config-builder/dist/index.js:126:42
at Array.forEach (native)
at ~/.config/yarn/global/node_modules/systemjs-config-builder/dist/index.js:125:25
at tryCatcher (~/.config/yarn/global/node_modules/bluebird/js/release/util.js:16:23)
at Promise._settlePromiseFromHandler (~/.config/yarn/global/node_modules/bluebird/js/release/promise.js:510:31)
at Promise._settlePromise (~/.config/yarn/global/node_modules/bluebird/js/release/promise.js:567:18)
at Promise._settlePromise0 (~/.config/yarn/global/node_modules/bluebird/js/release/promise.js:612:10)
at Promise._settlePromises (~/.config/yarn/global/node_modules/bluebird/js/release/promise.js:691:18)
at Promise._fulfill (~/.config/yarn/global/node_modules/bluebird/js/release/promise.js:636:18)
at Promise._settlePromise (~/.config/yarn/global/node_modules/bluebird/js/release/promise.js:580:21)
at Promise._settlePromise0 (~/.config/yarn/global/node_modules/bluebird/js/release/promise.js:612:10)
at Promise._settlePromises (~/.config/yarn/global/node_modules/bluebird/js/release/promise.js:691:18)
at Promise._fulfill (~/.config/yarn/global/node_modules/bluebird/js/release/promise.js:636:18)
at Promise._resolveCallback (~/.config/yarn/global/node_modules/bluebird/js/release/promise.js:431:57)
at Promise._settlePromiseFromHandler (~/.config/yarn/global/node_modules/bluebird/js/release/promise.js:522:17)
at Promise._settlePromise (~/.config/yarn/global/node_modules/bluebird/js/release/promise.js:567:18)
Hi Guys,
Wondering your thoughts on this ...
I can see an usecase for generating the systemjs from an npm shrinkwrap file (or yarn.lock file) and not by walking the node_modules dependencies. If you want a consistently repeatable systemjs config file then - it's a better option than trusting developers (or C.I environments) node_modules installs.
When people use optimistic semver in node modules, it can obviously have the side effect that different devs are developing on the same NPM module, but with a different set of downstream dependencies. This can be a disaster for sane release management.
If the systemjs config file was produced from a stable dependency tree that can be versioned, e.g. yarn.lock or npm shrinkwrap - then I could see this been more useful.
Good work on this package so far!
I have breifly looked through the code and are trying to understand the details of how it works. In doing so I thought I would write my findings down and maybe it could be added to a "How it works" section of the readme in the future. It may also help in discussion of improvements to the algoritm. So here it is, please let me know what I have misunderstood :-).
SystemJS needs three pieces of information to load a package:
P1. Extended package.json information (eg. format: cjs
).
P2. The URL to the package root.
P3. Map to dependent packages.
systemjs-config-builder
provides the above information for each package by this algoritm:
A1. Walk node_modules
recursively and look for package.json
. Build a registry (map) of packages with key name@version
and value {location}
. The recursive part ensures that packages that has private dependencies in their own node_modules
folder are included in the registry.
A2. Iterate the registry and call jspm
to generate the extended package.json information needed (P1). The registry values now have {location, config}
.
A3. Loop through the registry and write information to a generated.config.js
file.
A3.1 Output path
section (P2). This section is hardcoded to this:
"paths": {
"nm:": "node_modules/"
}
A3.2 Output map
section. For each package (1...N) set name
to registry key minus @version
part and output:
"map": {
"[name1]": "nm:[name1]",
...
"[nameN]": "nm:[nameN]",
}
A3.3 Output packages
section. For each package (1...N) set name
to registry key minus @version
part, and config
from the registry value and output:
"packages": {
"nm:[name1]": {
[config1]
},
...
"nm:[nameN]": {
[configN]
}
}
The systemjs plugin-json package, for example, uses define
but gets "format": "cjs"
.
When trying to use plugin-typescript the chrome tab hangs with 100% CPU usage.
I assumed this was done in jspm/npm, but it isn't
Hey,
Ive been running out of memory when trying to generate System config for large modules. Increasing the Node max mem is not really helping. Problem seems to be in JSPM-NPM.
Could I contribute a enhancement that only scans "dependencies" and "peerDependencies" and skips "devDependencies" ?? Will really cut down the tree walking!
I guess some command line arg could be used to determine when to ignore devDepends.
Interested?
In case a package is published under a scope, it is not added to the generated config. An example for this are the Angular >= 2.0.0 packages that are published under the @angular
scope.
The resulting node_modules
directory structure looks like this:
node_modules
├── @angular
│ ├── common
│ │ ├── README.md
│ │ ├── bundles
│ │ ├── index.d.ts
│ │ ├── index.js
│ │ ├── index.js.map
│ │ ├── index.metadata.json
│ │ ├── package.json
│ │ ├── src
│ │ └── testing
│ ├── compiler
│ │ ├── README.md
│ │ ├── bundles
│ │ ├── index.d.ts
│ │ ├── index.js
│ │ ├── index.js.map
│ │ ├── index.metadata.json
│ │ ├── package.json
│ │ ├── src
│ │ └── testing
│ ├── core
│ │ ├── README.md
│ │ ├── bundles
│ │ ├── index.d.ts
│ │ ├── index.js
│ │ ├── index.js.map
│ │ ├── index.metadata.json
│ │ ├── package.json
│ │ ├── src
│ │ └── testing
│ ├── platform-browser
│ │ ├── README.md
│ │ ├── bundles
│ │ ├── index.d.ts
│ │ ├── index.js
│ │ ├── index.js.map
│ │ ├── index.metadata.json
│ │ ├── package.json
│ │ ├── src
│ │ └── testing
│ └── platform-browser-dynamic
│ ├── README.md
│ ├── bundles
│ ├── index.d.ts
│ ├── index.js
│ ├── index.js.map
│ ├── index.metadata.json
│ ├── package.json
│ ├── src
│ └── testing
...
At the moment, everything works if you use jspm to install the @node polyfills. But this still requires you to have JSPM installed.
Normally we would just be able to install the jspm-nodelibs-* packages, but these don't follow the npm conventions. For example, jspm-nodelibs-buffer has the following package.json
:
{
"main": "buffer.js",
"name": "jspm-nodelibs-buffer",
"version": "0.2.0",
"registry": "github",
"map": {
"./buffer.js": {
"node": "@node/buffer",
"browser": "buffer-browserify"
}
},
"dependencies": {
"buffer-browserify": "npm:buffer@^4.0.0"
},
"jspmNodeConversion": false
}
Notice the dependency mapping. This causes npm to fail and yarn to do weird things. So I need to create forks of all these libs, and ideally make this automated so that whenever the underlying package changes, we just rebuild.
In the generated.config.js file, all the module locations (in node_modules) are with escaped back slashes. The imports in the code themselves, though, use forward slashes. This causes at least one issue I noticed with systemjs, resolving packages for an import, since the backslashes don't match. (See https://github.com/systemjs/systemjs/blob/master/dist/system.src.js#L2205. the substr
check will always be false because the slashes are different)
The natural solution seems to be normalizing all backslashes to be forward slashes.
I saw there's an item on the checklist in the readme about this. How about (optionally?) excluding all devDependencies?
Just wanted to let you know I'm doing some experimenting over here with using the yarn.lock file as we originally discussed. Not sure if it will be useful but at least I am learning a lot :-). I would really like to make the system config a pure function of the lockfile. But I discovered that is not possible as we need to add package config from from the jspm-node conversion stuff. This part is also quite slow, would be nice to find a way to make it faster. Anyway, the system config is now a pure function of the lockfile and the extra jspm-config.
I'm basically doing the same thing as in this repo, just divided the steps a little different to get info from the lockfile rather than the node_modules folder. I was thinking we could add some code that looks for a yarn.lock file and uses it if present, otherwise falls back to scanning through node_modules. Maybe I could contribute something like this but we would have to re-arrange a bit to make that work. And as I said, not sure if it will be useful as the slow part is probably not scanning node_modules.
One thing I noticed is that I did not have to hard-code the node-libs packages using the short name in the map. It worked with the long names. What I mean is that this worked:
SystemJS.config({
"paths": {
"nm:": "node_modules/"
},
"map": {
"jspm-nodelibs-assert": "nm:jspm-nodelibs-assert",
"jspm-nodelibs-buffer": "nm:jspm-nodelibs-buffer",
"jspm-nodelibs-child_process": "nm:jspm-nodelibs-child_process",
"jspm-nodelibs-cluster": "nm:jspm-nodelibs-cluster",
...
If I understood correctly the code in this repo will map with the short names like "assert", "buffer" and so on.
Installing via npm works. Yarn does not.
The problem is that Yarn installs the module with thispackage.json
{
"name": "string_decoder",
"version": "0.10.31",
"description": "The string_decoder module from Node core",
"main": "index.js",
"dependencies": {},
"devDependencies": {
"tap": "~0.4.8"
},
"scripts": {
"test": "tap test/simple/*.js"
},
"repository": {
"type": "git",
"url": "git://github.com/rvagg/string_decoder.git"
},
"homepage": "https://github.com/rvagg/string_decoder",
"keywords": [
"string",
"decoder",
"browser",
"browserify"
],
"license": "MIT"
}
to node_modules/string_decoder-browserify
instead of node_modules/string_decoder
.
This is different to the npm behaviour, which works as expected.
@thejameskyle maybe you can help with this?
to reproduce,
npm install
cd test
npm install or yarn install
jspm install
npm run config
the above breaks if you installed via yarn
I follow the example to do:
// in an empty folder.
yarn global add systemjs-tools
yarn add systemjs systemjs-nodelibs systemjs-plugin-babel react react-dom
systemjs config
But the systemjs config
just exit and no generated.config.js
is created.
I am did steps from guide, but command do nothing. Its works?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.