snodgrass23/base12

12factor.net web application boilerplate for node.js built on express.js

base12

12factor.net web app platform for node.js, built on express.js

$ sudo npm install -g base12
$ base12 new projectname && cd projectname
$ make open

Make commands

Install all modules and bower components

  $ make setup

Run the default build command using Grunt

  $ make build

Run the app with one process and without file change restarts

  $ make simple

Run the app with default settings and open browser to app url

  $ make open

Run the app with default number of processes (num cpu's in machine)

  $ make run

Run the app, limiting to a single process

  $ make run 1

Run the app and cycle on file changes

  $ make cycle

Run the app and cycle on file changes, limiting to a single process

  $ make cycle 1

Run the app with single process and node profiling

  $ make profile

Run the app with single process and node profiling using prof_lazy option

  $ make profile-lazy

Run the app with default settings with node debugging enabled (use node-inspector to view source and debug)

  $ make debug

Run the app with default settings with node debugging enabled and immediately break on first line

  $ make debug-brk

What you get

Production-ready

• Painlessly follow Ryan Dahl's 'gospel' for node.js apps (12factor.net by Adam Wiggins).

Cloud Deployments

• Deploy to the cloud easily out of the box to Heroku using built in Procfile. Built in Mongoose and Redis support for connecting to RedisToGo and MongoLab addons.

Structure

• Always know where things go. An easy to use component driven layout with proven MVC architecture all on top of express.

Express 3

• Leverage the newest version of the most popular app framework for node.js.

Not Rails

• We believe that, if Rails is best for your project, you should use it. Instead, base12 embraces the node.js way: light processes, shallow inheritance, simple interfaces, and the chain-of-responsibility pattern.

Where stuff goes

assets                -- place to store assets for project (graphics, src files, etc.)
components            -- place to store components for small piecs of functionality in app
  /dashboard          -- default dashboard example component
  /errors             -- default component for handling server errors
  /user               -- default component for user functionality using mvc pattern (signup, signin, signout, settings)
doc                   -- documentation
lib                   -- app specific and non-npm-published node.js libraries
  /balance            -- uses cluster to create and blance multiple processes
  /config-load        -- loads available config files and environment variables
  /flash              -- flash messaging
  /inject             --
  /locals             -- add resuable local helpers to app views
  /middleware         -- sets up express middleware (stylus, sessions, logs)
  /mongoose           -- connects mongoose to mongodb
  /mongoose-util      -- provides mongoose helpers (validations, plugins, etc)
  /redis              -- provides app-wide redis connection
  /reload             -- watches for file changes and reloads app
public                -- static files are hosted here
scripts               -- scripts (eg admin, deployment, migrations)
test                  -- tests (mocha by default)
tmp                   -- your app can store temporary files here

app.js                -- runs your app
config.default.js     -- default config (no sensative passwords or location specific options)
config.local.js       -- local config (ignored by git, create to store sensative information and location specific options)
config.test.js        -- config for running tests
Makefile              -- automated task makefile
package.json          -- npm package.json

Writing new components and libs

All base12 components have the same signature:

module.exports = function(app) {
  // ...
  return my_module;
}

The component or lib is responsible for supplying the app with the needed interface hooks. For example, a component might look like:

module.exports = function(app) {
  app.get('/dashboard', function(req, res) {
    return res.render(require('path').join(__dirname, 'dashboard'), {
      user: req.session.user
    });
  });
};

You will then need to add the component to the main app file for loading.

var user = require('./components/user');

user(app, config);

Updating constants and config

Application constants (values that do not change from machine to machine) are located in config.default.js.

module.exports = {
  port: 3000,
  cluster: true,
  request_timeout: 100000,
  session_secret: "base12secret",
  log_requests: false,
  stylus_compress: 1,
  stylus_debug: 1,
  stylus_force: 1,
  test: false,
  redis: {
    host: "localhost",
    port: 6379,
    auth: "",
    debug: false
  },
  mongoose_url: "mongodb://localhost/base12"
};

Environment config (values that can change from machine to machine) are located in config.local.js, which is not tracked by git. You can create this file whenever needed and it values will override the defaults if both exist.

module.exports = {
  port: 80,
  mongoose_url: "mongodb://username:[email protected]/base12"
};

All system environment variables will also act as local config object and will also override default settings of the same name. This leaves it up to the developer and environment which way they want to specify location dependent settings.

The 12 Factors

1. Codebase

"One codebase tracked in version control, many deploys."

Base12 uses git-based deployments exclusively.

2. Dependencies

"Explicitly declare and isolate dependencies."

Base12 uses npm install both locally and in deploys to manage dependencies. Manage your dependencies in package.json.

3. Config

"Store config in the environment."

Base12 uses the untracked config.local.js file to manage environment config. It will also pull in available environment variables for instant compatibility with cloud providers like Heroku.

4. Backing services

"Treat backing services as attached resources."

Backing service configuration is stored in config.local.js or environment variables on each host.

5. Build, release, run

"Strictly separate build and run stages."

make build builds a base12 app's assetes, while make run executes it. make cycle watches local files and cycles between build and run phases for rapid development.

6. Processes

"Execute the app as one or more stateless processes."

Base12 apps are stateless. The built-in session manager is backed by redis, and apps can be run as any number of independent processes forked from app.js. The directory structure provides /tmp for temporary file manipulation, but provides no permanent file storage mechanism since that should be done through a backing service.

7. Port binding

"Export services via port binding."

Ultimately, base12 relies on node's built-in http server to field requests. No http container or helper is needed.

8. Concurrency

"Scale out via the process model."

Using deployment-specific process managers (eg, upstart), base12 keeps the master node.js process running. In make run base12 uses cluster to spawn and monitor multiple processes on a single machine.

9. Disposability

"Maximize robustness with fast startup and graceful shutdown."

Base12 uses a crash-only design. Uncaught errors exit the process, triggering the balancer to replace it. Startup is nearly immediate.

10. Dev/prod parity

"Keep development, staging, and production as similar as possible."

We encourage you to keep your config.local.js configurations as similar as possible across machines to maximize parity.

11. Logs

"Treat logs as event streams."

Base12 logs events directly to stdout and stderr.

12. Admin processes

"Run admin/management tasks as one-off processes."

All admin processes are handled with scripts in the /scripts directory. Built-in scripts include provisioning and deployment, tests, dependency management, and generators.

System Requirements

• node.js >= 0.8.x
• npm >= 1.1.x
• redis (for default session management)
• mongodb (if using default user component)

base12's Issues

Needs automatic view helpers for resources

Not sure about the best API for this, consider these:

routes.js:

app.resource('users'); 

view.jade:

resource_uri('users', 'create');
resource_url('users', 'show', 'uuid')

or maybe, since app is already injected into the view:

app.controllers.users.create.url()
app.controllers.users.show.url('uuid')

app.controllers.users.url() // generic "/users" so you can concat custom routes to the end

default configs should update to match project name instead of default 'base12'

Need a subdomain example for Heroku

Cannot run with debug

I want to debug, but trying to run node --debug run / cycle results in:

Failed to open socket on port 5858, waiting 1000 ms before retrying
Failed to open socket on port 5858, waiting 1000 ms before retrying
Failed to open socket on port 5858, waiting 1000 ms before retrying

Is there an alternative for debugging?

Needs a resource generator

$ npm run-script generate resource user(name, email) with (password, timestamp)

would use /scripts/generate/templates/* to...
-> create /app/models/user.js
-> create /app/controllers/user.js
-> create /test/models.user.test.js
-> output to the terminal:

To access your new user resource, add this to your routes file:

app.resource('users', app.controllers.user);

(or something like this)

Needs an index for generator scripts

$ npm run-script generate

available generate scripts:
- twitter-bootstrap
- html5bp
- resource resourcename(property, list) with (plugin, list)

Needs a solution for /shared JavaScript packages

move mongoose libs to separate node module

also add functionality to module to handle things:

• easy file uploads to local or s3
• helper functions like findByIdAndUpdate that also trigger pre/post hooks

Needs a cycle/watch system for cycle.js

...that watches files (but not /public files) and runs build.js then run.js on changes.

Needs a good 'expires' header middleware

Either as a standalone lib, or as a pull request into Connect or Express, ala:

http://developer.yahoo.com/performance/rules.html#expires

Perhaps with an API like this?

app.use(connect_expires(0)) // expires immediately
app.use(connect_expires(1000 * 60 * 60 * 24 * 7)) // expires one week after last access
app.use(connect_expires(new Date('24 august 2014')) // expires on a specific date

...where subsequent calls would just override old calls, so that you could do:

app.use(connect_expires(0)); // markup expires immediately
app.use(express.router);
app.use(connect_expires(1000 * 60 * 60 * 24 * 30)); // static files expire in a month
app.use(express.static(__dirname + '/public'));

Update for Node 4.0 and clear out now irrelevant issues.

add default mailer component

allow component to have templates and also allow it to send mail directly or through sendgrid on heroku environment

register error throws text of [Object object]

click register button without filling in any fields.

nimbus.json error

I have an app built with a previous minor release of base12 (my package.json is asking for 0.3.x), but this time when I ran an update of all my modules, I apparently pulled a new version which errors on run b/c it's missing a nimbus.json file.

Looking at the nimbus project, it looks like something I don't need to bother with, so why does the base12 app care if I am using it or not?

Further, if I do want to correct my error, what's the nimbus.json file need to have to work? Can't find docs on that either.

Added features (like mongoose) should check config

...on server start to see if their key (mongoose: { ... }) is present. If not, they should throw an error that says they need config info. This will quickly alert team members when new required config is not present.

Needs a standard lib of stylus mixins

look into usefulness of using yeoman generators

http://yeoman.io/generators.html

Remove bloated dependencies

I believe base12 would be a much more useful module without having things like mongoose and redis as default options. What if someone decided to go with a Postgres database? Removing nib is also a potential. We should provide the option to use these packages easily as we use them the most, but not assume they'll be used for everything.

Side note: bcrypt is required twice in package.json @ two different versions.

base12 or make command for copying remote config to heroku

should this use nimbus.json file, or should we update nimbus to use a separate remote config that could be used for this process as well?

Needs a build system for build.js

Possibly using ender?

Shouldn't have to sudo.

I have to run base12 command as sudo to get things to work properly and even when they do everything is owned by root.

env/defualt.env.js

changing the port number in this file doesnt seem to load into the server

Steps Taken

1. followed install instructions
2. created Sample project
3. cd into Sample project
4. change port number in env/defualt.env.js
5. node run

Expected Result: Server runs on changed port number
Actual Result: Sever runs on defualt port 3000

Notes: also I tryed changing the env/defualt.env.js in the root base12 dir with the same results

Needs a cycle/watch system for cycle.js

...that watches files (but not /public files) and runs build.js then run.js on changes.

Cycle Not Quite Working

When I run node cycle and make a change to the controller. That change is not reflected in the view.

grunt build scripts

Implement grunt build scripts to handle things like JS minification or even things like s3 deployments.

Needs resourceful routing (or express-resource to be upgraded to Express 3)

Considering an API like this -

routes.js:

  var ac = app.controllers;

  app.resource('projects', ac.project, function() {
    this.resource('pages', ac.page, function() {
      this.resource('comments', ac.comment);
    });
    this.resource('reviews', ac.review);
  })
  .map('stats', ac.project.stats)
  .map('collaborators', ac.project.collaborators);

Add reset password functionality to user component

copy functionality over from Client Dashboard

pull render methods from components and move to global location

License isn't posted

Maybe I'm missing something, but I can't find what license base12 is released under

New base12 script handler doesn't work

Works when you test it with an absolute path locally, but once it's installed base12 to /usr/local/bin/base12, the $dir stuff breaks since it's looking in /usr/local/scripts to find the scripts that of course aren't there (eg 'new' and 'generate').

Looking into a way to get the right $dir after npm install.

load config from env vars when needed

Update lib/config-load to also check for environment variables for local config. This will help with deployment to cloud services like Heroku.

Needs to lock version of base12-server

In package.json, the version of base12 should be locked:

"base12":"0.2.0" <-- or whatever

...so that it expects the same directory structure and API that the cloned template is using. Otherwise, you can clone a project with "base12":"latest" and a breaking change in a new version of base12 can screw up the project on npm install.

Needs other generator scripts

eg:

npm run-script generate twitter-bootstrap
npm run-script generate html5bp

and more complex stuff like:

npm run-script generate auth-passport

-> creates /app/middleware/authentication.js (providing user middlewhere like .must_be_user())
-> creates /app/lib/authentication.js (configuration for auth mechanisms)
-> outputs:

you should add

var authentication = require('./lib/authentication') 

to app/index.js and then add authentication to your app's 'init' array.
Then, you should use the methods in middleware/authentication wherever you would like to authenticate your controller actions. For example:

app.post('/messages', app.middleware.authentication.must_be_user, app.controllers.message.create);

npm install failing

Getting errors left and right trying to do npm install lately. This worked fine a couple weeks ago, but the last few times I've tried installing it (once on a centos box, several times on OSX - Lion) I've been getting errors like this:

npm ERR! error installing [email protected]
npm http GET http://registry.npmjs.org/mkdirp/0.3.0
npm ERR! error installing [email protected]

npm ERR! Error: ENOENT, no such file or directory '/Users/pvencill/.nvm/v0.6.9/lib/node_modules/base12/node_modules/___connect.npm/package/lib/public/icons/page_white_find.png'
npm ERR! You may report this log at:
npm ERR! http://github.com/isaacs/npm/issues
npm ERR! or email it to:
npm ERR! [email protected]
npm ERR!
npm ERR! System Darwin 11.4.0
npm ERR! command "node" "/Users/pvencill/.nvm/v0.6.9/bin/npm" "install" "-g" "base12"
npm ERR! cwd /Users/pvencill/workspace
npm ERR! node -v v0.6.9
npm ERR! npm -v 1.1.0-3
npm ERR! path /Users/pvencill/.nvm/v0.6.9/lib/node_modules/base12/node_modules/___connect.npm/package/lib/public/icons/page_white_find.png
npm ERR! fstream_path /Users/pvencill/.nvm/v0.6.9/lib/node_modules/base12/node_modules/___connect.npm/package/lib/public/icons/page_white_find.png
npm ERR! fstream_type File
npm ERR! fstream_class FileWriter
npm ERR! code ENOENT
npm ERR! message ENOENT, no such file or directory '/Users/pvencill/.nvm/v0.6.9/lib/node_modules/base12/node_modules/___connect.npm/package/lib/public/icons/page_white_find.png'
npm ERR! errno {}
npm ERR! fstream_stack Object.oncomplete (/Users/pvencill/.nvm/v0.6.9/lib/node_modules/npm/node_modules/fstream/lib/writer.js:204:26)
npm ERR! error rolling back [email protected] Error: UNKNOWN, unknown error '/Users/pvencill/.nvm/v0.6.9/lib/node_modules/base12/node_modules/express/node_modules/___mkdirp.npm/package/examples'

npm ERR! Error: failed to fetch from registry: commander/0.6.1
npm ERR! at /Users/pvencill/.nvm/v0.6.9/lib/node_modules/npm/lib/utils/npm-registry-client/get.js:139:12
npm ERR! at cb (/Users/pvencill/.nvm/v0.6.9/lib/node_modules/npm/lib/utils/npm-registry-client/request.js:32:9)
npm ERR! at Request._callback (/Users/pvencill/.nvm/v0.6.9/lib/node_modules/npm/lib/utils/npm-registry-client/request.js:137:18)
npm ERR! at Request.callback (/Users/pvencill/.nvm/v0.6.9/lib/node_modules/npm/node_modules/request/main.js:109:22)
npm ERR! at Request. (/Users/pvencill/.nvm/v0.6.9/lib/node_modules/npm/node_modules/request/main.js:198:58)
npm ERR! at Request.emit (events.js:88:20)
npm ERR! at ClientRequest. (/Users/pvencill/.nvm/v0.6.9/lib/node_modules/npm/node_modules/request/main.js:195:10)
npm ERR! at ClientRequest.emit (events.js:67:17)
npm ERR! at Socket. (http.js:1134:11)
npm ERR! at Socket.emit (events.js:67:17)
npm ERR! You may report this log at:
npm ERR! http://github.com/isaacs/npm/issues
npm ERR! or email it to:
npm ERR! [email protected]
npm ERR!
npm ERR! System Darwin 11.4.0
npm ERR! command "node" "/Users/pvencill/.nvm/v0.6.9/bin/npm" "install" "-g" "base12"
npm ERR! cwd /Users/pvencill/workspace
npm ERR! node -v v0.6.9
npm ERR! npm -v 1.1.0-3
npm ERR! message failed to fetch from registry: commander/0.6.1
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR! /Users/pvencill/workspace/npm-debug.log
npm not ok

I've tried several versions of node to see if maybe it was a version problem, but have not had different results.

Needs a solution for /shared JavaScript packages

base12 generate mongoose -- needs to run 'npm install'

Needs a standard lib of locals (view helpers)

mongoose generator

This line needs to be changed to mongoose instead of mongo...

mongoose.connect('mongodb://' + app.config.mongo.host + '/' + app.config.mongo.db, function(err)

Make each component's public folder content accessible

For example the "components/user/public/" contents are not accessible when the view references them. I am adding BackboneJS widgets to each component and would like to keep the static files like .js separate without the need of recreating them in the main public folder.

git: don't ignore case!

Needs to automatically run this:

git config core.ignorecase false

Because otherwise deployment from OSX -> Linux can be a real pain in the ass.