bloomberg/consul-cluster-cookbook

Wrapper cookbook which installs and configures a Consul cluster.

consul-cluster cookbook

Cluster cookbook which installs and configures a Consul cluster with TLS certificates.

This cookbook utilizies the Consul application cookbook to bootstrap and maintain Consul server infrastructure. It is intended as an example usage of the wrapper cookbook pattern and should hopefully serve as a basis for production infrastructure.

Usage

The default recipe in this cookbook is designed to install and configure the Consul agent to bootstrap a cluster. The minimum number of nodes necessary for the infrastructure is three. After three nodes have been setup and configured Consul will bootstrap the Cluster.

Chef Vault and TLS

By default, the Consul Cluster cookbook uses Chef Vault to encrypt the TLS certificates and keys that are deployed to the nodes. This data should only be capable of being decrypted by the nodes the Consul Cluster itself.

{
    "ca_certificate": "-----BEGIN CERTIFICATE-----\nMIIE [...]",
    "certificate": "-----BEGIN CERTIFICATE-----\nMIIE [...]",
    "private_key": "-----BEGIN RSA PRIVATE KEY-----\nMIIE [...]"
}

Preparing the Instance

There are many configuration options that are exposed using the Consul application cookbook that can be set in either a wrapper cookbook, a Chef environment or a Policyfile. But it is good form to not keep secret information inside of a repository (or not encrypted in the Chef Server).

The following configuration file can be written to the instances to set cluster's datacenter and encryption key for the gossip protocol. This file should be written to /etc/consul/conf.d/server.json on each of the nodes.

{
    "datacenter": "us-east-1",
    "encrypt": "RxNBbMMl3m/cSOdjKDhATg=="
}

Using Policyfiles

It is extremely simple to use Chef Policyfiles to configure your three nodes which formulates your Consul cluster. Using the chef update and chef push commands you can push the below policy to a Chef Server. A more detailed explanation of how to use these commands is for the chef CLI documentation.

name consul-cluster'
default_source :community
cookbook 'consul-cluster', '~> 2.0'
run_list 'consul-cluster::default'

override['consul']['version'] = '0.6.4'
override['consul']['config']['bootstrap_expect'] = 3
override['consul']['config']['start_join'] = %w{10.0.10.2 10.0.10.3 10.0.10.4}

Test Kitchen

If you are looking to execute the integration tests you must first configure the policyfile that ships with the test fixtures.

~/Projects/consul-cluster-cookbook % chef install test/fixtures/policies/default.rb
~/Projects/consul-cluster-cookbook % kitchen converge ubuntu-1404

consul-cluster-cookbook's Issues

consul-cluster-cookbook still reports version 2.0.0 in chef supermarket

I am only able to pull version 2.0.0 from the chef supermarket, which causes me to have to pull version 2.1 of consul.

About override data bag item

How can override the data bag item for sanely override the data bag item name in our wrapper cookbook.
We keep following items in data bag and will like to know what are the changes required to use this by overriding data bag items.
['consul-cluster']['tls']['ssl_key']['item']
['consul-cluster']['tls']['ssl_cert']['item']
['consul-cluster']['tls']['ssl_chain']['item']

Your quick help will be appreciated.

Thank you,
Sanjay

Specifying start_join in wrapper cookbook recipe fails to populate consul.json

If I specify override in attributes/default.rb, the override takes place and the IPs show up in consul.json.

If I specify the attribute in recipes/default.rb, the attributes show up with knife show, but not in consul.json.

Example...

vault_nodes=search(:node, 'recipes:"role-vault-server" AND chef_environment:"' + node.chef_environment + '"')
vault_ips=vault_nodes.map(&:ipaddress)
node.default['consul']['config']['start_join'] = vault_ips

Results in knife show ...

consul:
config:
..
start_join:
10.0.0.1
10.0.0.2
10.0.0.3

But, they do not appear in /etc/consul/consul.json

I assume something screwy is happening here with poise? Any ideas?