Build and compile npm packages to run in the browser. This API is capable of
building modules through different build systems. The aim is to have full
cross-build-system API that serves a single file to be used in the browser.
Note that this API should only be hit from warehouse.ai.
git clone [email protected]/godaddy/carpenterd.git
npm install
Make sure BFFS is configured against a running NoSQL database. Development, staging and test configurations assume this instance is available on the localhost. Without a database builds will not be stored.
npm startThe API consists of two methods. Running this as an API allows the entire
build process to run independantly as a microservice. POST routes only
accept application/json.
Trigger a new build for the package specified in the payload. Configuration
properties are merged in with the provided specification. For example the
registry that is used to install the package will be merged in. This route
expects a POST payload that is similar to npm publish.
Payload:
{
"_id": "test",
"name": "test", // Used as key for storage.
"description": "A builder test",
"main": "index.jsx", // Entry file if not defined in build system.
"dist-tags": {
"latest": "0.0.0" // Used to extract the version.
},
"build": "webpack", // Overrule the build system type.
"main": "index.jsx",
"_attachments":{
"test-0.0.0.tgz": {
"data": "...", // base64 encoded tarball of npm pack.
"length": 665
}
}
}The route will stream whiteline delimited JSON as response. The id is the
unique v4 id generated that can also be used to cancel the build.
Example:
curl -vX POST -H "Content-Type: application/json" -d @payload-0.0.0.json http://localhost:1337/build
Accept: application/json
Accept-Encoding: gzip, deflate
Content-Type: application/json; charset=utf-8
Host: localhost:6064
{"event":"task","message":"start","progress":0,"id":"95cf09e6-3a4b-42b2-a3ef-d52b8a3e9ae0","timestamp":1438958247119}
{"event":"task","message":"init","progress":14,"id":"95cf09e6-3a4b-42b2-a3ef-d52b8a3e9ae0","timestamp":1438958247120}
{"event":"task","message":"unpack","progress":29,"id":"95cf09e6-3a4b-42b2-a3ef-d52b8a3e9ae0","timestamp":1438958247120}
{"event":"task","message":"exists","progress":43,"id":"95cf09e6-3a4b-42b2-a3ef-d52b8a3e9ae0","timestamp":1438958248603}
{"event":"task","message":"read","progress":57,"id":"95cf09e6-3a4b-42b2-a3ef-d52b8a3e9ae0","timestamp":1438958248605}
{"event":"task","message":"install","progress":72,"id":"95cf09e6-3a4b-42b2-a3ef-d52b8a3e9ae0","timestamp":1438958249945}
{"event":"task","message":"assemble","progress":86,"id":"95cf09e6-3a4b-42b2-a3ef-d52b8a3e9ae0","timestamp":1438958250210}
{"event":"task","message":"finished","progress":100,"id":"95cf09e6-3a4b-42b2-a3ef-d52b8a3e9ae0","timestamp":1438958250226}
carpenterd will orchestrate builds as specified in the package. Builds are
distributed to NSQ. carpenterd-worker instances
subscribe to NSQ and perform the actual builds. To maximize the developer
experience it will use the same configuration you use locally. In any case
the result should equal the local build output, with the exception of
additional minification, etc. Minification will only be performed if the
env is set to prod, e.g. for npm dist-tags 'package@version' prod.
The following builds systems are currently available.
-
Browserify: will read the
mainfile as determined from thepackage.jsonand bundle all modules that are imported/required. Configuration is usually part of any dependantpackage.json. This build has no explicit configuration, it will simply execute browserify. The complete output file with the CommonJS require wrapper is exposed to BFFS. -
Webpack: will read the default webpack configuration (
webpack.config.js). There are no imposed limitations on the configuration. However, the output directory will have to be./distby our convention. All files in the output directory will be published to BFFS.
Specify a build system in package.json with the build keyword or use
any of the following terms in the keywords:
- Webpack:
webpack - Browserify:
browserify
Alternatively specifying the build system name on the package.json with
the relative path to the configuration file will also classify the build
system, for example: webpack: '/path/to/config.js'.
If a published package should not run any builds at all, provide a
build: false flag in the package.json.
{
"name": "package",
"version": "1.0.0",
"build": false,
...
}Note: the module/package can also be published directly to a module registry. However, if you want to ensure dependents are build whenever your module is published this flag can be useful.
Each environment specifies a different set of default options for the builder.
For instance which registry to run npm install against. Each build instance
has a maximum runtime of 15 minutes. This value can be changed in the
configuration.
By default carpenterd runs as an service over http and has no
authentication in place. Setup the configuration to have Slay use https
and use authentication middleware, for example authboot.
Store API keys and tokens in an encrypted config with whisper.json.
Variables and specifications required for a build are discerned from a
combination of package.json, build system configuration files and
defaults from Carpenters configuration.
type: can be supplied as build property on the package.json or is
extracted from the keywords. Defaults to Webpack.
target: writes the package and its dependencies to a temporary folder
named after build.id a unique v4 id. After building this folder is
removed from the file system to save disk space.
version: read from the package.json dist-tags.latest. Has no default.
name: Defaults to the package.json name property, e.g. the modules name.
locale: Uses the locales specified on the package.json. Each unique
locale triggers a new build. The build will have the environment variables
LANG and LOCALE set for each build. These values default to en-US.
The files listed here need to be relative of the root project so that they can be properly read from disk. This gives you more fine tune control over what files get published to the CDN in any given environment.
[files]
prod = ['dist/js/app.min.js', 'dist/css/app.min.css']
test = ['dist/js/app.js', 'dist/css/app.css']
dev = ['dist/js/app.js', 'dist/css/app.css'];Carpenterd supports posting messages to the warehouse.ai status-api via NSQ. It will post messages to the nsq topic configured at:
{
// ...other configuration
"nsq": {
"statusTopic": "an-nsq-topic", // topic that you choose for the status-api to consume
// ...other nsq setup
},
// ...other configuration
}The NSQ payloads will be object that take the form:
{
eventType: "event|queued|error|ignored", // The type of status event that occurred
name: "package-name",
env: "dev", // The environment that is being built
version: "1.2.3", // The version of the build
locale: "en-US", // (Optional) The locale that is being built
buildType: "webpack", // The type of the build (typically just webpack)
total: 5, // (Optional) The number of builds that were queued
message: "Description of what happened"
}In the status-api NSQ payload there is a field called eventType.
The possible values that carpenterd will send are:
event- Used for interim statuses that a user might care about, but doesn't affect/progress the overall build statusqueued- Used to indicated how many builds were queued withcarpenter-workererror- Used to indicate thatcarpenterdencountered an error and wasn't able to queue all the buildsignored- Used to indicate that the build was ignored and no builds were queued. Typically this is because the package was not configured to have a build or was set to not build.
Run an AWS local cloud stack, pull latest [localstack].
This requires docker to be setup.
docker pull localstack/localstack:latest
npm run localstackRun tests in a separate terminal.
npm test
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent