southclaws/machinehead

A docker-compose application manager that deploys and maintains a set of compose projects and provides secret management for them via Vault.

Machinehead

GitOps for single-server deployments!

Deprecated! Replaced with Wadsworth

A docker-compose application manager that deploys and maintains a set of compose projects and provides secret management for them via Vault.

Machinehead is designed for single-server hobbyists who want to make use of containers and modern GitOps practices but can't since most of the tools (such as kube-applier) focus on cluster technology such as Swarm and Kubernetes.

In addition to this lack of tooling, managing sensitive secrets such as database credentials on single-server deployments doesn't have many solutions that integrate with Docker nicely.

And so, Machinehead was born to solve both these problems!

Architecture

Machinehead is essentially a background process that is given one or more Git repositories that contain docker-compose.yml files. It will periodically attempt to pull from each reository and, if there is a change it will execute docker-compose up for it.

This lets you update the configuration of your containerised applications simply by doing a git push!

Tip: pairs really nicely with Watchtower!

Usage

Machinehead is current-working-directory ("CWD") sensitive rather than binary-path sensitive, this means you can install it with go get and run it from any directory.

It doesn't currently have any official daemonising methods so it's up to you to write your own systemd/upstart/whatever configs. You could also just use tmux or screen and detach from the session.

Configuration

When Machinehead is started, it will check the CWD for machinehead.json which looks like:

{
  "targets": [
    "git@domain:username/my-project1",
    "git@domain:username/my-project2"
  ],
  "check_interval": "10s",
  "cache_directory": "./machinehead_cache"
}

For best results, the directory that contains machinehead.json should also be a git repository, if it is, Machinehead will also watch that repository for changes and, if there are any, it will pull them and automatically update its own configuration if there are changes to machinehead.json.

Global Environment Variables

Machinehead will also search the CWD for a file named .env, if it finds one, it will attempt to read it as key=value format and pass the fields to all instances of docker-compose which means you can set globally shared variables for all your projects.

Vault

Vault isn't currently supported but it's planned. The idea is, you'll be able to create per-project secrets inside Vault that contain sensitive configuration variables such as database credentials etc.

Using the example targets above, Machinehead will read kv values from /secrets/my-project1 for my-project1 and export them as environment variables for the docker-compose call.

machinehead's Issues

Fully implement and test Vault support for secrets

Currently, some Vault code is in but it doesn't really do anything and hasn't been fully tested.

Load configuration from a JSON, YAML, TOML file

Configor would be a good choice for this.

However, Configor brings its own logs, which I'm against. So I may fork it and either remove logs or provide an event stream for logging.

Make all daemon errors non-fatal

Currently, if the daemon encounters an error with

• loading secrets
• running compose
• receiving an error from gitwatch
  It will exit.

In production, this is not desired behaviour so these errors should be logged but not cause an exit.

However, there should be some way of signalling an exit inside start in the rare case that it's not possible to continue operating after an error.

Move configuration to JSON, make .env pass through

Environment variables read from .env should pass through to all applications, as a way to set global variables.

Add command line interface

It would be useful to provide a CLI for options such as config location and version.

Expand `targets` to contain more options

targets should not just be a list of URLs (as simple and elegant as that was) but a list of objects where additional properties can be set.

• Per-project environment variables
• Auto-generate a .env file for each project for testing - so the user can run docker-compose commands manually while still using all the secrets that were loaded from vault (#7), the global .env and the per-project environment variables section above
• Disable automatic refreshing of this service

Hot-reload of configuration

Dependent on #2

Configuration should be reloaded at runtime (fsnotify) in order to facilitate zero-downtime reconfigurations.

Reloaded configuration should not cause a failure, if the new configuration results in an error, the original configuration should be used as a fallback.

Authentication for private git repositories

In order to support private git repos, Machinehead requires basic auth at the very least.

SSH would be a nice to have, not sure on the complexities of implementation yet.