npm/ndm

ndm allows you to deploy OS-specific service-wrappers directly from npm-packages.

ndm's Issues

Detect node bin path using `which node` in config.

I'd like to use exec-sync to detect the appropriate Node.js bin, rather than having it set from CLI.

look into restart/stop customization

NodeSource uses lock file restart detection?

API for writing self-installing modules

Continuing tangential discussions from #39 and #48.

Given an app, allow it to require('ndm') and use it to implement its own service installer. The app could use a combination of it's own services.json for the interview phase and custom code for additional setup like creating necessary files/directories/users or whatever else such an installer might need to do.

Refactor Logging Class

• It would be nice to have different log levels.
• It would be nice to abstract the coloring logic out of the various classes that log.
• the number of new-lines I use are all over the place.

look at emulating `npm run-script`

see: https://github.com/bmeck/npm-script

allow programmatic usage

Too much stuff gets junked with require('./config'), checking cwd(), argv, and reading current host state.

blocking #87 #88 #89

support for forever-monitor

I've done some commits to help forever to work as service manager for NodeOS, only keeps ndm to export services to its format :-) This would need to have render functions instead of templates, since the idea is to have all that config in a json file, with a list of objects one for each installed service, so an API is necesary. Where could we start looking for?

SmartOS Support + Thoughts about Service Generation

It would be awesome to support SmartOS, we've had a few people request it.

As we add more service wrappers, It would probably be smart to abstract things a bit, so that an integration can be written as a self contained module. Here's where a new service is added currently:

• OS-specific default settings are defined in config.js (e.g., logs) this would be easy to pull out into another module:

https://github.com/npm/ndm/blob/master/lib/config.js#L53

• The actual service generation step is performed here, and simply involves rendering a different template depending on the OS:

https://github.com/npm/ndm/blob/master/lib/config.js#L53

A couple thoughts about this approach:

• It would be better to pull the rendering into its own JS class for each service, this could be where we also place the OS default settings.
  • we could have a base class that template-based service wrappers inherit from.
• this abstraction would let us support an OS like NodeOS, where service generation is done through an API rather than through plopping a file somewhere on disk.
  • as part of this we should probably make the generation step async, and make it so you can reference an external module for service generation.

CC: @mmalecki, @rmg

pre/post script hooks and node_modules/.hooks support

ndm interview with command line parameters

prompt allow to define command line parameters so it doesn't do that questions later, it would be a good feature for ndm interview so it doesn't block asking the configuration parameters.

init does not work properly for scoped packages

Generate/Install Should Raise An Exception If Permissions Are Bad

On Ubuntu, an exception does not get raised when /etc/init/ cannot be written to.

Add ./node_modules/.bin to path

It would be nice if ./node_modules/.bin was in the search path for bins.

use a process-queue for running remote stuff

ndm <cmd> <npm-package>

I would love to be able to ship an npm package (with a service.json) that can be installed as a service, right away, like so:

npm install -g <my-package>
ndm generate <my-package>

Same goes for start/stop etc.

Currently one has to create a wrapper around services that are dependencies. I can understand this setup in a devops environment where you want to package together and distribute services, and in addition with specific args for ports etc.

But I think it would also be cool if in addition there was a more basic use case where you could just start an individual service directly from the npm package, without creating a wrapper at all.

If I'm not mistaken to pull this off you would need to allow top-level scripts in the service.json, right next to the top-level args and env.

I'd be more than willing to send a PR if this sounds like something that could get merged.

escape spaces when generating run-scripts

We should escape spaces in strings when generating run scripts, so that users don't have to quote their variables.

Add `interview` feature

It would be nice to be able to prompt the user for variables when generating the service.json:

inquirer looks great for this.

Infer module, description, and scripts.start from package.json

Things would be a bit more DRY in service.json if ndm would read from package.json in the absence of certain properties. Instead of this:

{
  "newww": {
    "module": "newww",
    "description": "A total rewrite of npm-www using hapijs",
    "scripts": {
      "start": "node server.js"
    },
    "env": {
      "PORT": "8081"
    },
    "args": {}
  },
  "newww2": {
    "module": "newww",
    "description": "A total rewrite of npm-www using hapijs",
    "scripts": {
      "start": "node server.js"
    },
    "env": {
      "PORT": "8082"
    },
    "args": {}
  },
  "newww3": {
    "module": "newww",
    "description": "A total rewrite of npm-www using hapijs",
    "scripts": {
      "start": "node server.js"
    },
    "env": {
      "PORT": "8083"
    },
    "args": {}
  },
  "newww4": {
    "module": "newww",
    "description": "A total rewrite of npm-www using hapijs",
    "scripts": {
      "start": "node server.js"
    },
    "env": {
      "PORT": "8084"
    },
    "args": {}
  }
}

I would prefer something like this:

{
  "newww": {
    "env": {
      "PORT": "8081",
      "NODE_ENV": "PRODUCTION"
    }
  },
  "newww2": {
    "env": {
      "PORT": "8082",
      "NODE_ENV": "PRODUCTION"
    }
  },
  "newww3": {
    "env": {
      "PORT": "8083",
      "NODE_ENV": "PRODUCTION"
    }
  },
  "newww4": {
    "env": {
      "PORT": "8084",
      "NODE_ENV": "PRODUCTION"
    }
  }
}

For personal reference, these spots in the code are related:

• https://github.com/npm/ndm/blob/master/test/service-test.js

• ndm/lib/service.js

  Line 217 in 386cb63

  exports.allServices = function(serviceNameFilter) {

• ndm/lib/service.js

  Line 271 in 386cb63

  function parseServiceJson(serviceNameFilter, serviceJson) {

Document which platforms are available in cli help

Took me a while to figure out it was 'ubuntu', 'centos' and 'initd', and the default 'linux' wasn't a thing.

accept full set of node-flags

It would be good to accept a full set of node flags, perhaps we could have a special suffix in the package.json for indicating flags that should be passed to the node bin?

Fallback to package.json

Currently we require that a service.json file is present in modules which are installed using ndm. I think we should fallback to using the package.json if none is found. I suggest that we, by convention, look for the following fields in package.json:

{
  "name": "package-name",
  "description": "description of services",
  "scripts": {
    "start": "the-service-script"
  },
  "env": {
       "PORT": 8000,
       "uid": {
          "description": "user to run as"
        }
   },
   "args": {
        "--foo": "args-instead-of-envs"
   }
}

This reuses a lot of fields from package.json, which I think is awesome, and fits well with the discussion around service manifests started here: Service Descriptions

CC: @piranna, @seanewest, @groundwater

run multiple copies of the same process

Args parser has trouble with shorthand, e.g., -p

bad exit codes

When an error happens starting/stopping/generating a script, we should exit with a non 0 code.

Refactor Configuration Class

• I'd like command-line argument generation to be automatic.
• there's some repeated code when setting up configuration for different OS' I'd like to get rid of.

not resolving relative paths appropriately

When a script is executed using ndm:

ndm run-script foo --path=./foo.txt

Paths are not resolved properly.

ideas for collaboration

Hey @bcoe, nice to meet you. I ran across this project and it looks like we are solving very similar problems with npm. For the past few months, I've been building this: https://github.com/xtuple/xtuple-server.

I ended up building a semi-generic task-based deployment system. The one piece I've struggled to do well is the management of the deployed processes. If you have a minute, I'd be interested to see what you think. I can see some opportunity for combining forces, if that interests you.

Add support for `crontab`

Add support for crontab on Linux as node-autostart does. This is more simple to manage that SystemD.

Rename 'remove' for 'uninstall'

An 'uninstall' command would be more simetric with the 'install' one and also make it orthogonal to NPM.

Support uid on Centos

This is easy on ubuntu:

setuid ubuntu.

Setting uid does not seem to work as well on Centos, I'm thinking perhaps we could put:

su user in the script clause on Centos.

test args parsing with popular libraries

• yargs.
• minimist.
• dashdash.
• nopt.

Wrong logs path reported when installing global service

Log directory reported as:

/Users/benjamincoe/Library/Logs/ndm.log

Should be:

/Users/benjamincoe/Library/Logs/service-name.log

More/less religious logging

It would be nice if ndm's generated Upstart jobs allowed Upstart's logging to be used. Using the console log stanza directs the process's stdout/stderr to /var/logs/upstart/.log with automatic log rotation.

Disclaimer: I wrote a rather opinionated blog post on how Node apps should log to stdout instead of managing log files.

Windows support?

Are there any plans (wishlist) for Windows support? Sorry if this was mentioned somewhere else, searched issues and didn't find much. Thanks.

Look into adding replify, other debugging helpers

scripts should start on reboot

.plist files already start on boot, but we should ensure that Centos/Ubuntu upstart scripts do the same.

yargs woes, does not handle `--foo=bar` handles `--foo bar`.

Discuss usefulness of Upstart for service coordination

Carrying from a tangent conversation on #48.

Upstart (and systemd) has support for connecting services as triggers. This could be used to model in Upstart the service dependency implied by listing multiple services inside services.json. Foreman does this based on a Procfile, but ndm's services manifest looks like it is even more suited to this task.

Using self-install with private npme modules

Following is a workaround to make it possible for npme private modules to be self-installed.

Assuming private module is @private/myapp

1. On the "bin" script, Include the private repo prefix, after the require statement:

var argv = require('yargs').argv
, ndm = require('ndm')('@private/myapp');

The docs say that you should use the name of the module inside the service.json, but without the prefix, the service.json is not found.

1. On the service.json, the "module" property has to be included for the service instantiation path to be set correctly.

"myapp": {
"description": "myapp",
"module": "@private/myapp",
"scripts": {
"start": "node ./lib/myapp.js"
},
........

Without the "module" property the script change-directory (cd) command changes to an incorrect path.

.ndmrc

It would be nice to have a .ndmrc file, so that users can place their service.json, logs, etc, in different locations.

Run scripts in foreground

It would be great for docker if scripts could be executed in the foreground.

refactor config singleton to be real singleton

Use ndm to manage the containing module

It would be great to be able to use ndm on the containing module. I want to be able to use ndm to create service wrappers for the module itself, rather than modules in node_modules. Obviously I can use a wrapper module but this isn't a good solution for my use case.

Start/Stop/Restart Specific Services

• it would be awesome to be able to start/stop/restart specific services.

environment variables should be quoted in upstart scripts

To reproduce:

Create a service.json with an env stanza that looks like this:

"env": { "MY_VAR": "i am a string with a lot of spaces in it" }

Generate upstart configs in an ubuntu environment. Observe that the service is invoked as something like this:

PORT=8080 MY_VAR=i am a string with a lot of spaces in it /usr/bin/nodejs server.js

Sadness ensues.

I would like to be able to run a command in the context of a service

It would be awesome to be able to run a CLI command in the context of the variables set in service.json.

systemd support

As common as Upstart@~0.6.5 (which lacks useful features from 1.4) support is, systemd looks like it is or will be taking over the role of "most common non-sysv-init" since Ubuntu and Debian are moving to it and almost every other distro already supports it.