gerhard/deliver

Pure bash deployment tool with customisable strategies.

deliver's Issues

SSH remote hosts key authentication

Initialized empty Git repository in /home/dashboard/app/assets/_lib/d3/.git/
Host key verification failed.
fatal: The remote end hung up unexpectedly

Option to sync / check git remote parity before deliver

If working in teams, it's important to keep your remote repo in sync when deploying releases. As deliver uses forced updates to push refs to the $HOSTS, it risks overwriting the target with an old working copy if the remote was not synced beforehand.

It would be useful to have an option such as $SYNC_REMOTE=origin which will perform a git pull, deliver, git push.

Alternatively, an option to exercise caution by checking your working copy is up to date with remote before a deliver is performed, because you don't always necessarily want deliver and automatically push code you've just pulled from the remote before you check it.

Deliver requires /usr/bin/sh to be bash

__remote() uses ssh syntax, which will evaluate the entire quoted command with env sh, which unfortunately on SmartOS is defined as ksh. The side effect is input redirection fails. It would be nice if __remote could explicitly tell ssh to execute bash and then the command. Not sure how to do this.

Handle interrupts gracefully

Currently, ^C just stops right there, no message.

Pre & post function hooks

When delivering, it would be nice if we could notify Campfire of fail/success. We might want to track successful/failed deploys via Graphite, librato metrics or any other endpoint (be it HTTP or some random process listening on a port, locally or remotely).

I'm thinking something along the lines:

declare -f pre_init_app_remotely > /dev/null
if [ "$?" = 0 ]
then
  pre_init_app_remotely
fi

This needs to be a generic function that checks all strategy functions.

SIGINT during git push does not stop gracefully

tag v0.8.0

time to tag v0.8.0?

When SERVERS are missing, deliver doesn't fail

Calista ➜  q git:(master) deliver check

APP             q
APP_ROOT        /Users/gerhard/code/q
STRATEGY        ruby
APP_USER        q
DELIVER_TO      ~q/app
SERVERS         

CAN'T DELIVER, WE'RE MISSING A FEW THINGS

Calista ➜  q git:(master) deliver -V
DELIVERING Q WITH RUBY STRATEGY
Q DELIVERED!

Foreman, exports and init control

This is just a quick thought dump:

Should the launch() command be in common, and just use foreman start/status/stop?

Should the launch() command be factored out to common, and have a switch/case for each export format supported by foreman?

PORT variable doesn't work

#!/usr/bin/env bash

APP="myapp"
BRANCH="node"
SERVERS="12.34.56.78"
PORT="2222"
STRATEGY="nodejs"

results in:

-----> Ensuring remote is ready to accept git pushes
ssh: connect to host 12.34.56.78 port 22: Connection refused

Why won't it go for the right port? or is there something I am misunderstanding?

foreman export upstart tmp

If for some reason the server is restarted, will the upstart scripts automatically start on boot? And is there a reason why it's exported to /tmp rather than /etc/init? Thanks!

SSH timeout

Handle SSH timeouts properly. Currently, if the SSH can't be executed, the script won't timeout.

Stop from delivering if there are no new commits

bug in launch

The launch task will not utilize APP_USER. All the other tasks prior to launch will ssh APP_USER@HOST but launch will ssh HOST, causing it to use the local username by default..
I'll try to fix it, but my bash scripting is VERY rusty so I don't know how long it'll take me.

Request: make it installable through npm

Installing through git clone is ok, but I would greatly appreciate if we could install deliver through npm install...

Multiple apps in a single repository

Think many small and self-contained ruby services (APIs? background workers?) in a single repository.

Overload-able strategy function

If I need to tweak the setup function from the nodejs strategy, I shouldn't need to create a custom strategy. Overloading specific strategy functions in the config file makes most sense in this scenario.

Does deliver support rollback?

One thing I expect from deployment tool is ability to easily rollback to previous version in case of some failure (e.g. app deployed to production introduces bug and we want quickly revert to older, working version).

Does deliver supports that?

Usage output

-h|--help

Authorizing multiple hosts on initial deliver freezes

Authorizing hosts happens in the background. Only one of the bg jobs accepts STDIN, the other one will wait indefinitely.

Authorize hosts in sequence, NOT in parallel.

Illumos / smf support

Illumos support would be greatly appreciated, for those using SmartOS or the like. Modifications are required for launch to check if `SUPERVISOR="smf", and use the following commands to check status, stop and start a service:

svcs: lists enabled services, even if they're not running. To see all services use 'svcs -a'
svcadm enable <foo>: start the service named <foo>
svcadm disable <foo>: stop the service named <foo>
svcadm restart <foo>: restart the service named <foo>
svcadm clear <foo>: if a service is in maintenance mode, this clears it so it can be enabled
svccfg export <foo>: shows the service manifest about what binary is being run and how

The real work will be in generating a service definition file, which is described here

Wordpress strategy

Any interest in this?

Overload-able functions

If I need to tweak the setup function from the nodejs strategy, I shouldn't need to create a custom strategy. Overloading specific strategy functions in the config file would make the entire experience a lot neater.

Authorize remote hosts explicitly

When using a private npm module, sourced from bitbucket.org for example. Same for a ruby gem.

'deliver check' displays all config vars

Currently, only the STRATEGY_VARS & REQUIRED_VARS are displayed.

Deliver any branch

Currently deliver only supports pushing master branches. As we're abstracting git push, we should be able to push any branch to the remote.

generated strategy

You know, for your Jekyll, Octopress, Toto etc. statically generated sites.

No sudo when using upstart

Both the ruby & nodejs strategies leverage foreman. Deliver is primarily used on Ubuntu with upstart, and this requires sudo privileges to start/stop upstart jobs.

The way I think this should work is to have a custom foreman upstart template which monitors files in the app itself and takes appropriate action. For example, touch tmp/restart-timeseries-api triggers a restart of /etc/init/timeseries-api.conf.

Host keys are breaking in git_submodules()

git_submodules() needs to be able to handle:

https://github.com/user/repo
[email protected]:user/repo
git://github.com/user/repo

See the submodules-fix branch.

Staging / Testing

Just started testing, created config file and a strategy file for composer ... all is fine.

The question I have is how do we handle multiple environments (testing / staging / production)? I can't see anyway to do this other than to completely change the config file each time.

error in git push

remote: error: refusing to update checked out branch: refs/heads/master

I can't push local master to nodejs01-myapp master, it says that it needs to be a bare repo for this to work.

Rename SERVERS to HOSTS

I clearly wasn't in my best of naming moods. Deprecate SERVERS as of 0.7.0.

Improve args parsing

https://github.com/wayneeseguin/rvm/blob/master/binscripts/rvm-prompt#L46

If foreman fails, the remote command doesn't

In my specific case, if -p option is given an empty argument, foreman command fails silently (double-check this), and the upstart files are not generated. When the resulting conf files are meant to be moved into place, the command fails without an indication as to what exactly happened.

Generated strategy tries to push wrong revision

Think this was introduced in 9579a8c - when delivering using the generated strategy, deliver will attempt to reset the remotes to the revision of the repo before the content was generated, which (as it's on a different branch) doesn't get pushed to the remotes, so fails with a fatal: Could not parse object: '123456...' error.

Multiple deliver strategies per project

Think gh-pages & ruby.

Make available via brew

Follow rbenv's and/or riemann's example.

Static assets pushed to separate hosts

In a properly distributed service, the front-end servers are separate from the back-end ones. All static assets should be pushed to the front-end servers. This means nothing more than doing the same push as to the back-ends, the difference being that all app-related tasks need dropping. So no dependency installs (npm, gems etc.) & no app restarts.

The next step would be to allow assets to be pushed directly to CDNs. I'm thinking CloudFront to start with (possibly Rackspace Files).

I still need to understand how useful this really is with the new services such as Cloud Flare and Fastly.

Timeout during npm install

I've probably done something wrong but:

This is the output of deliver check