Giter Site home page Giter Site logo

docker-consul's Introduction

About this Repo

This is the Git repo of the Docker official image for consul. See the Docker Hub page for the full readme on how to use this Docker image and for information regarding contributing and issues.

The full readme is generated over in docker-library/docs, specificially in docker-library/docs/consul.

See a change merged here that doesn't show up on the Docker Hub yet? Check the "library/consul" manifest file in the docker-library/official-images repo, especially PRs with the "library/consul" label on that repo. For more information about the official images process, see the docker-library/official-images readme.

docker-consul's People

Contributors

aldiyen avatar alvin-huang avatar analogue avatar bastiaanb avatar danstough avatar david-yu avatar dnephin avatar eculver avatar ffmmm avatar freddygv avatar hanshasselberg avatar hashi-derek avatar hashicorp-copywrite[bot] avatar ishustava avatar jmurret avatar kpenfound avatar kyhavlov avatar lkysow avatar markan avatar mikemorris avatar mkeeler avatar mterron avatar nathancoleman avatar pearkes avatar preetapan avatar rboyer avatar ryansch avatar slackpad avatar tryan225 avatar zkanda avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

docker-consul's Issues

Adding runtime config

Hi there,

I've run into an issue recently with deploying docker-consul with my use case and would like to offer to add a feature that address this if it's desired by hashicorp / this repo's maintainers.

Issue: environments that dynamically schedule containers (Mesos, Kubernetes, Rancher) are becoming common but dealing with state in them is hard. Using volumes to inject config's is not an option in these cases as the volume must either be distributed to every host or scheduling of the container must be limited to those which have the volume.

Proposed feature: many people have began using applications like ContainerPilot to do runtime configuration, upkeep, monitoring, and metrics (in fact ContainerPilot is often paired with Consul Templates to dynamically render config files). I will be creating a container that allows for runtime configuration via ContainerPilot which can be enacted by an environmental variable and thought that I would offer to contribute to the official Consul repo first before rolling my own.

Please let me know if this is something that would be accepted into this repo!

Thanks,
code-ape

Docker socket permissions for health checks?

I'm currently using a locally-built Consul image, that runs as root, and that includes a couple of Docker health checks. I'm trying to migrate to using the official image, and running into permission issues.

Say I run Consul as

docker run --name consul \
  --net host \
  -v $PWD/consul:/consul/data:Z \
  -v $PWD/etc/consul:/consul/config:Z \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -e CONSUL_ALLOW_PRIVILEGED_PORTS=true \
  consul:0.7.2 agent

where the health checks, recursor settings, etc. are in the $PWD/etc/consul directory.

If the Docker socket on the host (/var/run/docker.sock) is world-writeable, this works fine. If I'm on an Ubuntu 16.04 host, where the socket is mode 0660 owned by user 0 group 16, the Docker health checks silently fail, since this container (via gosu) runs its process as user 1000 group 1000.

Are there best practices for giving Consul permission to docker exec? I'm not comfortable with the broader implications of making the Docker socket world-writeable or opening a TCP version of it, I only want to give permission to Consul.

Can't seem to get the api to work? Try this

curl http://localhost:8500/v1/catalog/nodes

Connection refused???

Probably bound to the wrong network.

When running a client agent it kept binding to lo by default

Set these bad boys in your environment variables and all of your problems will go away

CONSUL_BIND_INTERFACE=eth0
CONSUL_CLIENT_INTERFACE=eth0

(problems with docker-entrypoint.sh)

credit: @pauzed

Multi-arch docker images for consul

Hi, I'm looking to enable docker containers for packages like consul for multiple architectures on dockerhub, starting off with a focus on ppc64le. Wanted to check on the work that will be involved to do that

I have locally been able to successfully test ppc64le changes to the dockerfiles in this repo - changes for enabling additional arch would be minimum.

The official images have this method for enabling multi-arch : https://github.com/docker-library/official-images#multiple-architectures, as mentioned there they recommend having a single dockerfile with swithches for arch-specific changes, want to understand if ppc64le additions would be accepted to the dockerfile here (can raise a PR) and if the ppc64le image creation process can be initiated subsequently under docker-library/official-images (would mostly need changes to the metadata file https://github.com/docker-library/official-images/blob/master/library/consul)

Thanks!

Container exits instantly after starting with net parameters

I tried several ways to start the official image on my system:

docker run -d consul

works just fine.

docker run -d consul -P

doesn't. The container exits immediately. Enumerating the ports separately has the same effect. --net=host won't do the trick neither.

Docker logs says this:

Invalid flags before the subcommand. If these flags are for
the subcommand, please put them after the subcommand.

usage: consul [--version] [--help] <command> [<args>]

Available commands are:
    agent          Runs a Consul agent
    configtest     Validate config file
    event          Fire a new event
    exec           Executes a command on Consul nodes
    force-leave    Forces a member of the cluster to enter the "left" state
    info           Provides debugging information for operators
    join           Tell Consul agent to join cluster
    keygen         Generates a new encryption key
    keyring        Manages gossip layer encryption keys
    leave          Gracefully leaves the Consul cluster and shuts down
    lock           Execute a command holding a lock
    maint          Controls node or service maintenance mode
    members        Lists the members of a Consul cluster
    monitor        Stream logs from a Consul agent
    reload         Triggers the agent to reload configuration files
    rtt            Estimates network round trip time between nodes
    version        Prints the Consul version
    watch          Watch for changes in Consul

I'm on CentOS 7 with Docker 1.12.0.

Cannot bind to port 53 for DNS with host network mode

The documentation mentions the possibility to map the DNS port:

docker run -d --net=host -p 53:8600/tcp -p 53:8600/udp consul

But when using --net=host the port mappings are not used by docker. So what is the recommended way to use the default DNS port? I've tried with

-e 'CONSUL_LOCAL_CONFIG={"skip_leave_on_interrupt": true, "ports": {"dns": 53}}'

but this leads to

==> WARNING: Expect Mode enabled, expecting 3 servers
==> Starting Consul agent...
==> Starting Consul agent RPC...
==> Error starting dns server: dns udp setup failed: listen udp 10.32.24.16:53: bind: permission denied

Could not find IP for interface

hi, folks

I was running docker inside of the vbox ubuntu 14.04 VM.
the consul docker image is the latest (0.6.4)

I got following error when start the consul container:

vagrant@vagrant:~$ docker run --rm --net=host --name=consul -e CONSUL_BIND_INTERFACE=192.168.59.104 consul
ip: can't find device '192.168.59.104'
Could not find IP for interface '192.168.59.104', exiting

IP 192.168.59.104 is the docker host IP (aka. the ubuntu VM IP)

if run ip addr list command in the consul container, I can see 4 interfaces:

# ip -o -4 addr list
1: lo    inet 127.0.0.1/8 scope host lo\       valid_lft forever preferred_lft forever
2: eth0    inet 10.0.2.15/24 brd 10.0.2.255 scope global eth0\       valid_lft forever preferred_lft forever
3: eth1    inet 192.168.59.104/24 brd 192.168.59.255 scope global eth1\       valid_lft forever preferred_lft forever
4: docker0    inet 172.17.0.1/16 scope global docker0\       valid_lft forever preferred_lft forever

if run the command, got the error:

# ip -o -4 addr list 192.168.59.104
ip: can't find device '192.168.59.104'

the relate source code is here https://github.com/hashicorp/docker-consul/blob/master/0.6/docker-entrypoint.sh#L14

Consul Swarm mode network configuration

I'm using the official consul docker image version 0.9.3 with the following compose file:

version: '3'

services:
  consul:
    image: consul:0.9.3
    environment:
      CONSUL_LOCAL_CONFIG: '{"skip_leave_on_interrupt": true}'
      CONSUL_BIND_INTERFACE: eth0
    command: agent -ui -data-dir /consul/data -server -bootstrap-expect 3 -client 0.0.0.0 -log-level debug -retry-join 10.10.0.3 -retry-join 10.10.0.4 -retry-join 10.10.0.5
    deploy:
      mode: replicated
      replicas: 3
    networks:
      - consul
    ports:
      - "8500:8500"
    volumes:
      - consul_data:/consul/data

networks:
  consul:
    driver: overlay
    ipam:
      driver: default
      config:
        - subnet: 10.10.0.0/24

volumes:
  consul_data:

This appears to work okay for me in a Docker Swarm with 3 nodes when deploying this as a stack using docker stack deploy

My question is this: without the -retry-joins the cluster can't bootstrap. Per the bootstrap documentation I believe this is expected to prevent split-brain, etc. So what's the best way to bootstrap a consul cluster running as a Docker Swarm Service?

I am only able to make the cluster bootstrap after having added the -retry-join statements to the container command. It seems non-ideal to have to specify these hard-coded IPs. For example, what if another container started up in the consul network first? Presumably it would interfere with the resulting IPs of the consul server containers.

Is there a recommendation on how to deal with this? The only other thing I can think of would be this:

  1. Deploy the consul servers via docker stack deploy
  2. At this point, all the nodes of the service are running but the cluster isn't bootstrapped
  3. Now figure out the IP's of each of the container nodes running consul server
  4. Manually issue consul join commands to one of the nodes, supplying the IP's of the other containers

Suggestion to add CONSUL_ADVERTISE as a parameter to containers; defaults to the IP listed in /etc/hosts

As noted from the current documentation, the -bind and -advertise parameters are closely related. One can bind to all interfaces, but needs to advertise to a specific one. Otherwise, Consul will fail to startup.

When using Consul in a Docker Swarm spanning multiple hosts, containers will always have more than one interface, eth0, eth1, eth2, etc. These include the container IP, the swarm ingress bridge, loopback, etc. On top of that, container aliases and service names may not respond to the same IP as the one listed in /etc/hosts.

An obvious workaround is to bind Consul to all interfaces of the container using -bind 0.0.0.0, but this immediately conflicts with the -advertise parameter. When faced with multipe bind interfaces, we need to specify which one will be advertised. Finding out this value at runtime has showned to be a challenge.

There is an option to provide CONSUL_BIND_INTERFACE to the container, but by itself, this parameter is not as flexible as we exepct it to be.

Another option we found browsing the issue history here is to include calls to the YML compose file that returns the IP address of the desired interface. For exmaple:

command: ... -advertise='{{ GetInterfaceIP "eth2" }}'

However, given different swarm environments, the bind interface might change from eth0 to eth2, etc. The value of the bind interface is not portable across environments.

We found a way around this by replacing the image's original docker-entrypoint.sh with our own custom-entrypoint.sh. Our script expects and optional CONSUL_ADVERTISE variable to be present. When it's not available, it will deduce the advertise address from the container's /etc/hosts, which will have its public IP address.

#!/bin/sh

if [ "$CONSUL_ADVERTISE" == "" ]; then
  CONSUL_ADVERTISE="$(grep $(hostname) /etc/hosts | awk '{print $1}')"
fi
CONSUL_ADVERTISE="-advertise $CONSUL_ADVERTISE"

exec /usr/local/bin/docker-entrypoint.sh $@ $CONSUL_ADVERTISE

I'm leaving the suggestion to include this parameter as part of the image. Of course, always open to ideas to make this any better. I'm curious to understand any reason why this would not be a good idea.

Kind regards.

expect mode cannot be enabled when dev mode is enabled

I am trying to run consul on openshift container platform. I am using official consul docker image. I use CONSUL_LOCAL_CONFIG environment variable to get a 3 node consul cluster.

My json file like this : {
"data_dir": "/var/consul",
"log_level": "INFO",
"node_name": "node1",
"server": true,
"ui": true,
"bootstrap_expect": 3
}

I got an error like " Expect mode cannot be enabled when dev mode is enabled". How can I disable dev mode via using CONSUL_LOCAL_CONFIG variable?

Development examples aren't working on Docker for Mac

❯ uname -a && docker -v
Darwin 15.6.0 Darwin Kernel Version 15.6.0: Mon Aug 29 20:21:34 PDT 2016; root:xnu-3248.60.11~1/RELEASE_X86_64 x86_64
Docker version 1.13.1, build 092cba3
❯ docker run -d --name=dev-consul consul
0f3919f4a5c72a18fd23808703d62c71d8aa30badb2ba663e3c3fa926c0d3f4e

❯ docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                                                                  NAMES
0f3919f4a5c7        consul              "docker-entrypoint..."   5 seconds ago       Up 3 seconds        8300-8302/tcp, 8400/tcp, 8500/tcp, 8301-8302/udp, 8600/tcp, 8600/udp   dev-consul
❯ docker logs 0f3919f4a5c7
==> Starting Consul agent...
==> Starting Consul agent RPC...
==> Consul agent running!
           Version: 'v0.7.5'
           Node ID: '42144b13-15ae-4f76-94e0-2cb7d2abef8a'
         Node name: '0f3919f4a5c7'
        Datacenter: 'dc1'
            Server: true (bootstrap: false)
       Client Addr: 0.0.0.0 (HTTP: 8500, HTTPS: -1, DNS: 8600, RPC: 8400)
      Cluster Addr: 127.0.0.1 (LAN: 8301, WAN: 8302)
    Gossip encrypt: false, RPC-TLS: false, TLS-Incoming: false
             Atlas: <disabled>

==> Log data will now stream in as it occurs:

    2017/03/01 19:47:34 [DEBUG] Using unique ID "42144b13-15ae-4f76-94e0-2cb7d2abef8a" from host as node ID
    2017/03/01 19:47:34 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:127.0.0.1:8300 Address:127.0.0.1:8300}]
    2017/03/01 19:47:34 [INFO] raft: Node at 127.0.0.1:8300 [Follower] entering Follower state (Leader: "")
    2017/03/01 19:47:34 [INFO] serf: EventMemberJoin: 0f3919f4a5c7 127.0.0.1
    2017/03/01 19:47:34 [INFO] consul: Adding LAN server 0f3919f4a5c7 (Addr: tcp/127.0.0.1:8300) (DC: dc1)
    2017/03/01 19:47:34 [INFO] serf: EventMemberJoin: 0f3919f4a5c7.dc1 127.0.0.1
    2017/03/01 19:47:34 [INFO] consul: Adding WAN server 0f3919f4a5c7.dc1 (Addr: tcp/127.0.0.1:8300) (DC: dc1)
    2017/03/01 19:47:41 [ERR] agent: failed to sync remote state: No cluster leader
    2017/03/01 19:47:42 [WARN] raft: Heartbeat timeout from "" reached, starting election
    2017/03/01 19:47:42 [INFO] raft: Node at 127.0.0.1:8300 [Candidate] entering Candidate state in term 2
    2017/03/01 19:47:42 [DEBUG] raft: Votes needed: 1
    2017/03/01 19:47:42 [DEBUG] raft: Vote granted from 127.0.0.1:8300 in term 2. Tally: 1
    2017/03/01 19:47:42 [INFO] raft: Election won. Tally: 1
    2017/03/01 19:47:42 [INFO] raft: Node at 127.0.0.1:8300 [Leader] entering Leader state
    2017/03/01 19:47:42 [INFO] consul: cluster leadership acquired
    2017/03/01 19:47:42 [DEBUG] consul: reset tombstone GC to index 3
    2017/03/01 19:47:42 [INFO] consul: member '0f3919f4a5c7' joined, marking health alive
    2017/03/01 19:47:42 [INFO] consul: New leader elected: 0f3919f4a5c7
    2017/03/01 19:47:44 [INFO] agent: Synced service 'consul'
    2017/03/01 19:47:44 [DEBUG] agent: Node info in sync
==> Failed to check for updates: Get https://checkpoint-api.hashicorp.com/v1/check/consul?arch=amd64&os=linux&signature=&version=0.7.5: x509: certificate signed by unknown authority
  "Networks": {
                "bridge": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": null,
                    "NetworkID": "173125c66a565b1c772c7748c201ee5e95b6445ffa060d666ca92486bf605c46",
                    "EndpointID": "228ca883b38f3d23f3135d639b578f3f5c7b6993e6dacb56f99dcb48cebb30f8",
                    "Gateway": "172.17.0.1",
                    "IPAddress": "172.17.0.2",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:ac:11:00:02"
                }
            }
❯ docker run -d consul agent -dev -join=172.17.0.2
6a736806a0b4e0659fbd472d8d6e614db3f8ef740954e8057691bb04b1fef169

❯ docker ps -a
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS                     PORTS                                                                  NAMES
6a736806a0b4        consul              "docker-entrypoint..."   7 seconds ago       Exited (1) 5 seconds ago                                                                          amazing_haibt
0f3919f4a5c7        consul              "docker-entrypoint..."   2 minutes ago       Up 2 minutes               8300-8302/tcp, 8400/tcp, 8500/tcp, 8301-8302/udp, 8600/tcp, 8600/udp   dev-consul

❯ docker logs 6a736806a0b4
==> Starting Consul agent...
==> Starting Consul agent RPC...
==> Joining cluster...
==> 1 error(s) occurred:

* Failed to join 172.17.0.2: dial tcp 172.17.0.2:8301: getsockopt: connection refused

Conflicting Node ID when using Docker

When using Consul with Docker I noticed with 0.8.0 the following message appear:

Failed to join 172.17.0.2: Member '330e1df099ea' has conflicting node ID 'a8a108a5-c071-49d0-84ef-b774af85c15f' with this agent's ID

When I use version 0.7.5 this message doesn't appear. Although Node IDs are still the same.

CONSUL_BIND_INTERFACE not working?

I am trying to set my bind interface through CONSUL_BIND_INTERFACE and it is pulling the correct IP for the interface but it is still giving me an error:

[fakeuser@fakeserver fakedir]$ docker run -it --rm --net=host -e "CONSUL_BIND_INTERFACE=eth1" consul consul agent -client=0.0.0.0 -ui -retry-join=server1.us.local -retry-join=server2.us.local -retry-join=server3.us.local -data-dir=/tmp/consul
==> Found address '100.73.14.171' for interface 'eth1', setting bind option...
==> Starting Consul agent...
==> Error starting agent: Failed to get advertise address: Multiple private IPs found. Please configure one.
When I run it and fill in the bind address manually it works:

[fakeuser@fakeserver fakedir]$ docker run -it --rm --net=host consul consul agent -ui -client=0.0.0.0 -bind=100.73.14.171 retry-join=server1.us.local -retry-join=server2.us.local -retry-join=server3.us.local -data-dir=/tmp/consul
==> Starting Consul agent...
==> Starting Consul agent RPC...
==> Consul agent running!
Version: 'v0.7.1'
Node name: 'server1.local.us'
Datacenter: 'dc1'
Server: false (bootstrap: false)
Client Addr: 0.0.0.0 (HTTP: 8500, HTTPS: -1, DNS: 8600, RPC: 8400)
Cluster Addr: 100.73.14.171 (LAN: 8301, WAN: 8302)
Gossip encrypt: false, RPC-TLS: false, TLS-Incoming: false
Atlas:
I'm not sure why its not working as its getting the right bind address to pass to -bind.

Why does the consul docker container require me to add a volume as rw?

I would like to add a config directory as a volume:

-v /etc/myconfig/with/options/not/supported/by/the/cli:/consul-server/myconfigdir

When I run this with:

-e CONSUL_CONFIG_DIR=/consul-server/myconfigdir

I get ==> config: Readdir failed on /consul-server/config. lstat /consul-server/myconfigdir/consul-server.json: permission denied

I believe there's a chown missing. Which actually leads me to another issue. Why do I have to declare a config volume as rw.

About -bind=<external ip> -retry-join=<root agent ip>

Hi, I'm newer for consul.
Can you tell me what is external ip and root agent ip?

When I run the command:

$  docker run -d --net=host -e 'CONSUL_LOCAL_CONFIG={"leave_on_terminate": true}' consul agent 
-bind=<external ip> -retry-join=<root agent ip>

just generate a docker container id and exit.

I tried external ip:

192.168.99.100   -- my docker default ip  
10.48.114.90     -- my computer ip  
172.17.0.2       -- dev-consul ip  
127.0.0.1

And the root agent ip, I just try

172.17.0.2
127.0.0.1

Thanks for you help and reply.

DNS listen port issue

In fact your netstat -an output confirms that - I can see it listening on 127.0.0.1:53 but not on 0.0.0.0:53
[root@red ~]# netstat -ntlp | grep "22"
tcp 0 0 192.168.122.1:53 0.0.0.0:* LISTEN 1472/dnsmasq
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1009/sshd
tcp6 0 0 :::22 :::* LISTEN 1009/sshd

[root@red ~]# netstat -ntlp | grep "53"
tcp 0 0 127.0.0.1:53 0.0.0.0:* LISTEN 7046/named
tcp 0 0 192.168.122.1:53 0.0.0.0:* LISTEN 1472/dnsmasq
tcp 0 0 127.0.0.1:953 0.0.0.0:* LISTEN 7046/named
tcp 0 0 0.0.0.0:49153 0.0.0.0:* LISTEN 1382/samba
tcp6 0 0 :::49153 :::* LISTEN 1382/samba
[root@red ~]#
i wants configure 0.0.0.0:53
how i do.... anyone have idea, please reply as soon as possible
##############this named.conf file
listen-on port 53 { any; };
listen-on-v6 port 53 { ::1; any; };
directory "/var/named";
dump-file "/var/named/data/cache_dump.db";
statistics-file "/var/named/data/named_stats.txt";
memstatistics-file "/var/named/data/named_mem_stats.txt";
allow-query { localhost; any; };

core dump at /core in official image

Hi,

there seems to be an 8MB core file left on the root of the consul container that... probably shouldn't be there! (you might want to check no credentials leaked?).

How to reproduce:
docker run --rm -it consul ls -l /core

Service Advertised with internal Docker ip

I am having an issue where my service is being advertised with the internal Docker ip. (172.17.0.2)
I need the consul agent to use its advertised IP, for the service location, instead of the IP that the service contacted it on.
Is there a way to do this?

Can't use ping in script checks

I migrated recently from progrium/consul image. In that image, consul is run as root user and all my ping script checks were working.
With this new image I get this:
PING 192.168.0.36 (192.168.0.36): 56 data bytes
ping: permission denied (are you root?)

I think ping is a very typical check script and it should work by default IMHO.

Is there a way to make consul user to execute pings?
What about other script commands?

Thanks in advance.

docker-compose dns_search option does not propogate

when writing a docker-compose such as the following:

version "3"

services:
  consul:
    image: consul:latest
    dns_search: "somewhere.example.com"

the service consul does not have an entry in /etc/resolv.conf for search somewhere.example.com

this is working on other images so I'm not sure if i'm just too much of a consul newbie or what.

i can give any additional info. thank you very much!

Unable to read config directory files in Openshift/Kubernetes when mounting from a config map

After upgrading to version 1.0.1, I was not able anymore to start consul on Openshift. Right after starting up there was on single line, that the configuration directory /consul/config/..data is a directory. The POD does not start up. I did not pass any special configuration command line items.

The directory looks like:

drwxrwsrwx 3 root 10001900 100 Nov 22 13:58 . drwxrwxr-x 1 consul root 20 Nov 4 08:24 .. drwxr-sr-x 2 root 10001900 45 Nov 22 13:58 ..119811_22_11_13_58_17.821227557 lrwxrwxrwx 1 root root 33 Nov 22 13:58 ..data -> ..119811_22_11_13_58_17.821227557 lrwxrwxrwx 1 root root 19 Nov 22 13:58 encrypt.json -> ..data/encrypt.json lrwxrwxrwx 1 root root 18 Nov 22 13:58 master.json -> ..data/master.json

The config map looks like:

apiVersion: v1 kind: ConfigMap metadata: name: consul-config data: encrypt.json: |- { "encrypt": "vjSSeXnpTcOuiAszAIgbKw==" } master.json: |- { "acl_datacenter":"dc1", "acl_default_policy":"deny", "acl_down_policy":"deny" }

The deployment config looks like:

` volumeMounts:

  • name: generali-consul-volume-1
    mountPath: /consul/data
  • name: consul-config
    mountPath: /consul/config
    dnsPolicy: ClusterFirst
    restartPolicy: Always
    securityContext: {}
    terminationGracePeriodSeconds: 30
    volumes:
  • name: generali-consul-volume-1
    persistentVolumeClaim:
    claimName: generali-consul
  • name: consul-config
    configMap:
    name: consul-config`

How to populate image with default data?

Hi!
We are starting to work with Consul and after a while we'd love to have our own image setup with a pre-default data (in our case at least some key/values already stored when we launch a container with that image).
What we're trying to do is, based in your container, populate it when it boots up with our predefined data. In our case, we are using our own Dockerfile that already extends yours, something like this:

FROM consul:latest
MAINTAINER xxxx
EXPOSE 8300 8301/udp 8302 8302/udp 8400 8500 8600 8600/udp
ENV BASE /xxx
RUN mkdir -p ${BASE}/data
COPY populate_keys.sh ${BASE}/
COPY data/ ${BASE}/data
RUN chmod +x ${BASE}/populate_keys.sh
CMD ${BASE}/populate_keys.sh

In our case, the script that try to populate the server tries to run many curl PUT operations just to fill the server with our specific data. In our case, populate_keys.shis something like this:

#!/bin/sh
set -x -e
HEADERS="Content-Type: application/json;charset=UTF-8"
URL=http://localhost:8500/v1/kv/config
/usr/bin/curl -X PUT -H \""$HEADERS\"" --data @hello_kv.txt \""$URL/hello/data\""

The problem is that is seems that consul server does not start as it is in your container image.

What we are missing? How we can try to "improve" your consul docker image just by adding our own specific data and also have the consul elements running (agent, UI, etc)?

Best,

`k8s` update breaks consul deployment

The new security update 1.7.14, 1.8.9, and 1.9.4 makes configMaps readonly by default: https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.7.md#other-notable-changes

Changes secret, configMap, downwardAPI and projected volumes to mount read-only, instead of allowing applications to write data and then reverting it automatically. Until version 1.11, setting the feature gate ReadOnlyAPIDataVolumes=false will preserve the old behavior. (#58720, @joelsmith)

This breaks a deployment that uses a configMap to define the consul config file. From consul logs:

chown: /consul/config: Read-only file system

Why does consul need to chown the config file?

data_dir cannot be empty

Hi all
I created a Dockerfile that looks as follow:

FROM consul

ENTRYPOINT ["consul", "agent"]

COPY config-file.json /consul/config

The config( config-file.json) file that looks as follow:

{
  "datacenter": "microservices",
  "data_dir": "/tmp/consul",
  "log_level": "INFO",
  "disable_host_node_id" : true, 
  "server": true,
  "bootstrap_expect" : 2,
  "client_addr": "0.0.0.0",
  "ui": true,
  "bind_addr": "192.168.70.20",  
  "watches": [
    {
        "type": "checks",
        "handler": "/usr/bin/health-check-handler.sh"
    }
  ],
  "telemetry": {
     "statsite_address": "127.0.0.1:2180"
  }
}

then run the container and got following:

dockerizer@docker1:~/consul$ docker run --rm -ti --network=host servery
==> data_dir cannot be empty

What am I doing wrong?

Thanks

Binding DNS port 53 on the host

I use the Docker image from Gliderlabs and would like to replace it with Hashicorp's image. But ran into an issue when running a consul:v0.6.4 container in "host" network-mode and binding the Consul DNS server to port 53. This works for the Gliderlab's image because consul agent is run as 'root'.

It would be great if there was an option to override Hashicorp consul's default user and group settings ( consul:consul) by setting an environment variable. Something like:

docker run -itd consul:v0.6.4 -e CONSUL_USER=root -e CONSUL_GROUP=root ...

It might be better to limit the option to the 'root' user or group because that is the only other user/group guaranteed to exist in the container.

wdyt?

docker run - Exposing Consul's DNS Server on Port 53 - failure on Ubuntu

I am running the command suggested (for exposing consul dns server on 53) on Ubuntu VM and ran into docker run and container creation failure. There are no relevant daemon logs for this, suggesting there might be issue in command parsing for CONSUL_ALLOW_PRIVILEGED_PORTS or issue in binding the ports.

Appreciate any help in this regard.

es_consul@consul-vm-1:/$ uname -a
Linux consul-vm-1 4.4.0-59-generic #80-Ubuntu SMP Fri Jan 6 17:47:47 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

es_consul@consul-vm-1:/$ docker version
Client:
Version: 1.13.0
API version: 1.25
Go version: go1.7.3
Git commit: 49bf474
Built: Tue Jan 17 09:58:26 2017
OS/Arch: linux/amd64

Server:
Version: 1.13.0
API version: 1.25 (minimum version 1.12)
Go version: go1.7.3
Git commit: 49bf474
Built: Tue Jan 17 09:58:26 2017
OS/Arch: linux/amd64
Experimental: false

es_consul@consul-vm-1:/$ docker run -d --net=host -e 'CONSUL_ALLOW_PRIVILEGED_PORTS=' consul -dns-port=53 -recursor=8.8.8.8
f364f735afdadf752c679e8c9d96c91594f67f6ff88f3e14e9699231de877474

es_consul@consul-vm-1:/$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
f364f735afda consul "docker-entrypoint..." 2 seconds ago Exited (1) 1 second ago gracious_poincare

es_consul@consul-vm-1:/$ docker logs f3
error: exec failed: text file busy

es_consul@consul-vm-1:/$ journalctl -u docker.service | grep f364f735afda
Jan 26 02:21:31 consul-vm-1 dockerd[18550]: time="2017-01-26T02:21:31.535104422Z" level=error msg="containerd: deleting container" error="exit status 1: "container f364f735afdadf752c679e8c9d96c91594f67f6ff88f3e14e9699231de877474 does not exist\none or more of the container deletions failed\n""

How to write docker-compose setup?

It is a slight disappointing that docker-compose setup is completely nonexistent in the current official image.

Could you please provide information about how to specify the following docker-compose image:

  • client mode
  • in a cluster (when cluster is not in Docker)

gpg: keyserver receive failed: No keyserver available

In OSX with Docker Version 17.09.0-ce-mac35 (19611)

gpg: directory '/root/.gnupg' created gpg: new configuration file '/root/.gnupg/dirmngr.conf' created gpg: new configuration file '/root/.gnupg/gpg.conf' created gpg: keybox '/root/.gnupg/pubring.kbx' created gpg: keyserver receive failed: No keyserver available

I have had to remove all the gpg commands to make the build work. And I have had to replace all the wget calls with curl -O . All other calls via the proxy works fine.

Support -data-dir and -config-dir in default docker entrypoint script

Hello,

Please change the default script to read in the passed arguments (-data-dir, -config-dir) to agent, instead of the hardcoded paths here:

if [ "$1" = 'consul' ]; then
# If the data or config dirs are bind mounted then chown them.
# Note: This checks for root ownership as that's the most common case.
if [ "$(stat -c %u /consul/data)" != "$(id -u consul)" ]; then
chown consul:consul /consul/data
fi
if [ "$(stat -c %u /consul/config)" != "$(id -u consul)" ]; then
chown consul:consul /consul/config
fi

Thank you! Apologies if this has already been addressed somewhere else.

Server nodes will be marked failed instead of left if SIGTERM sent to container

Just tried a docker stop containerid on a server node (which sends a SIGTERM by default). Then checked the consul members output on a client node and found the stopped server was marked failed. I can confirm that a docker logs containerid showed zero signs of a graceful leave being processed. The container exited within a split second, no delays.

Is it possible that gosu or dumb-init are just performing SIGKILL's on the consul process...? This is obviously not ideal :)

Currently using a variation of the official 0.X (0.7) image provided here, just copy pasted into our own Dockerfile to allow adding an extra orchestration utility into the build process. Also running with 2 distinct Dockerfile phases, one to build a generic consul image, then I build a consul-server and consul-agent image respectively using the parent with FROM.

Parent Dockerfile:

# Image Source (with modifications):
# https://github.com/hashicorp/docker-consul/blob/470868df3885ad93f45a2c63c648bf119a544fa4/0.X/Dockerfile

FROM alpine:3.4

MAINTAINER James Phillips <[email protected]> (@slackpad)

# This is the release of Consul to pull in.
ENV CONSUL_VERSION=0.7.0

# This is the release of https://github.com/hashicorp/docker-base to pull in order
# to provide HashiCorp-built versions of basic utilities like dumb-init and gosu.
ENV DOCKER_BASE_VERSION=0.0.4

# Used for discovering other nodes on ECS
ENV ECS_DISCOVERER_VERSION 0.3.3

# Create a consul user and group first so the IDs get set the same way, even as
# the rest of this may change over time.
RUN addgroup consul && \
    adduser -S -G consul consul

# Set up certificates, our base tools, and Consul.
# curl remains installed for usage by things like health checks.
RUN apk add --no-cache bash ca-certificates curl gnupg libcap openssl unzip && \
    gpg --recv-keys 91A6E7F85D05C65630BEF18951852D87348FFC4C && \
    mkdir -p /tmp/build && \
    cd /tmp/build && \
    wget https://releases.hashicorp.com/docker-base/${DOCKER_BASE_VERSION}/docker-base_${DOCKER_BASE_VERSION}_linux_amd64.zip && \
    wget https://releases.hashicorp.com/docker-base/${DOCKER_BASE_VERSION}/docker-base_${DOCKER_BASE_VERSION}_SHA256SUMS && \
    wget https://releases.hashicorp.com/docker-base/${DOCKER_BASE_VERSION}/docker-base_${DOCKER_BASE_VERSION}_SHA256SUMS.sig && \
    gpg --batch --verify docker-base_${DOCKER_BASE_VERSION}_SHA256SUMS.sig docker-base_${DOCKER_BASE_VERSION}_SHA256SUMS && \
    grep ${DOCKER_BASE_VERSION}_linux_amd64.zip docker-base_${DOCKER_BASE_VERSION}_SHA256SUMS | sha256sum -c && \
    unzip docker-base_${DOCKER_BASE_VERSION}_linux_amd64.zip && \
    cp bin/gosu bin/dumb-init /bin && \
    wget https://releases.hashicorp.com/consul/${CONSUL_VERSION}/consul_${CONSUL_VERSION}_linux_amd64.zip && \
    wget https://releases.hashicorp.com/consul/${CONSUL_VERSION}/consul_${CONSUL_VERSION}_SHA256SUMS && \
    wget https://releases.hashicorp.com/consul/${CONSUL_VERSION}/consul_${CONSUL_VERSION}_SHA256SUMS.sig && \
    gpg --batch --verify consul_${CONSUL_VERSION}_SHA256SUMS.sig consul_${CONSUL_VERSION}_SHA256SUMS && \
    grep consul_${CONSUL_VERSION}_linux_amd64.zip consul_${CONSUL_VERSION}_SHA256SUMS | sha256sum -c && \
    unzip -d /bin consul_${CONSUL_VERSION}_linux_amd64.zip && \
    curl -L -O https://github.com/CpuID/ecs-discoverer/releases/download/$ECS_DISCOVERER_VERSION/ecs-discoverer-$ECS_DISCOVERER_VERSION-linux_musl_amd64.zip && \
    unzip ./ecs-discoverer-$ECS_DISCOVERER_VERSION-linux_musl_amd64.zip && \
    mv ./ecs-discoverer-$ECS_DISCOVERER_VERSION-linux_musl_amd64 /ecs-discoverer && \
    chmod +x /ecs-discoverer && \
    rm -f ./ecs-discoverer-$ECS_DISCOVERER_VERSION-linux_amd64.zip && \
    cd /tmp && \
    rm -rf /tmp/build && \
    apk del gnupg openssl unzip && \
    rm -rf /root/.gnupg

# The /consul/data dir is used by Consul to store state. The agent will be started
# with /consul/config as the configuration directory so you can add additional
# config files in that location.
RUN mkdir -p /consul/data && \
    mkdir -p /consul/config && \
    chown -R consul:consul /consul

# Expose the consul data directory as a volume since there's mutable state in there.
VOLUME /consul/data

# No ENTRYPOINT or CMD specified, designed to be used as a parent image for consul-server + consul-agent images.

Server child Dockerfile:

# Logic adjusted to allow Consul servers to detect eachother by interrogating
# Amazon ECS APIs for other existing servers to join.

FROM private.registry.host.here:5000/infrastructure-consul:0.7.0d

COPY ./docker-entrypoint.sh /docker-entrypoint.sh
COPY ./config-server.json /consul/config/server.json
COPY ./config-server-encrypt.json /consul/config/encrypt.json

RUN chmod +x /docker-entrypoint.sh

EXPOSE 8300 8303 8303/udp 8304 8304/udp 8401 8501

ENTRYPOINT ["/docker-entrypoint.sh"]

# Server mode specified in JSON config
CMD [ "agent" ]

docker-entrypoint.sh:

#!/bin/dumb-init /bin/bash
set -e

# Note above that we run dumb-init as PID 1 in order to reap zombie processes
# as well as forward signals to all processes in its session. Normally, sh
# wouldn't do either of these functions so we'd leak zombies as well as do
# unclean termination of all our sub-processes.

# You can set CONSUL_BIND_INTERFACE to the name of the interface you'd like to
# bind to and this will look up the IP and pass the proper -bind= option along
# to Consul.
CONSUL_BIND=
if [ -n "$CONSUL_BIND_INTERFACE" ]; then
  CONSUL_BIND_ADDRESS=$(ip -o -4 addr list $CONSUL_BIND_INTERFACE | head -n1 | awk '{print $4}' | cut -d/ -f1)
  if [ -z "$CONSUL_BIND_ADDRESS" ]; then
    echo "$(date) : Could not find IP for interface '$CONSUL_BIND_INTERFACE', exiting"
    exit 1
  fi

  CONSUL_BIND="-bind=$CONSUL_BIND_ADDRESS"
  echo "==> $(date) : Found address '$CONSUL_BIND_ADDRESS' for interface '$CONSUL_BIND_INTERFACE', setting bind option..."
fi

# You can set CONSUL_CLIENT_INTERFACE to the name of the interface you'd like to
# bind client intefaces (HTTP, DNS, and RPC) to and this will look up the IP and
# pass the proper -client= option along to Consul.
CONSUL_CLIENT=
if [ -n "$CONSUL_CLIENT_INTERFACE" ]; then
  CONSUL_CLIENT_ADDRESS=$(ip -o -4 addr list $CONSUL_CLIENT_INTERFACE | head -n1 | awk '{print $4}' | cut -d/ -f1)
  if [ -z "$CONSUL_CLIENT_ADDRESS" ]; then
    echo "$(date) : Could not find IP for interface '$CONSUL_CLIENT_INTERFACE', exiting"
    exit 1
  fi

  CONSUL_CLIENT="-client=$CONSUL_CLIENT_ADDRESS"
  echo "==> $(date) : Found address '$CONSUL_CLIENT_ADDRESS' for interface '$CONSUL_CLIENT_INTERFACE', setting client option..."
fi

# CONSUL_DATA_DIR is exposed as a volume for possible persistent storage. The
# CONSUL_CONFIG_DIR isn't exposed as a volume but you can compose additional
# config files in there if you use this image as a base, or use CONSUL_LOCAL_CONFIG
# below.
CONSUL_DATA_DIR=/consul/data
CONSUL_CONFIG_DIR=/consul/config

# You can also set the CONSUL_LOCAL_CONFIG environemnt variable to pass some
# Consul configuration JSON without having to bind any volumes.
if [ -n "$CONSUL_LOCAL_CONFIG" ]; then
  echo "$CONSUL_LOCAL_CONFIG" > "$CONSUL_CONFIG_DIR/local.json"
fi

# END upstream docker-entrypoint.sh
#########################################
# START custom docker-entrypoint.sh

# Required for ecs-discoverer
if [ -z "$ECS_SERVICE_NAME" ]; then
  echo "Error - the ECS_SERVICE_NAME environment variable is not defined. Required to discover other nodes."
  exit 1
fi
echo "==> $(date) : ECS Service Name - ${ECS_SERVICE_NAME}"

# Define the datacenter via env var
if [ -z "$CONSUL_DATACENTER" ]; then
  echo "Error - the CONSUL_DATACENTER environment variable is not defined. Required to define the Consul Datacenter for this node."
  exit 1
fi
echo "==> $(date) : Consul Datacenter - ${CONSUL_DATACENTER}"
CONSUL_DC="-dc ${CONSUL_DATACENTER}"

# Required for ecs-discoverer and WAN gossip
if [ -z "$OTHER_ECS_CLUSTERS" ]; then
  echo "Warning - the OTHER_ECS_CLUSTERS environment variable is not defined. Required to discover other servers for WAN gossip (separate Datacenter defs)."
  echo "No WAN gossip will occur from this cluster on startup."
else
  echo "$(date) : Other ECS Clusters - ${OTHER_ECS_CLUSTERS}"
fi

old_ifs="$IFS"
IFS=","

# LAN Gossip
CONSUL_JOIN_PARAM=
echo "==> $(date) : Detecting other nodes to join..."
consul_server_join_ips=$(/ecs-discoverer -s "${ECS_SERVICE_NAME}")
if [ $? -ne 0 ]; then
  echo "$(date) : No other nodes found to join, proceeding anyway (hopefully another node will join this one instead). Error:"
  printf "${consul_server_join_ips}"
  echo ""
else
  echo "==> $(date) : Nodes found to join - ${consul_server_join_ips}"
  for i in $consul_server_join_ips; do
    # Custom port of 8303 allows for communication to Consul servers on alternate Serf LAN port.
    CONSUL_JOIN_PARAM="${CONSUL_JOIN_PARAM} -retry-join=${i}:8303"
  done
fi

# WAN Gossip (if applicable)
CONSUL_JOIN_WAN_PARAM=
if [ ! -z "$OTHER_ECS_CLUSTERS" ]; then
  # IFS splits this by comma
  for j in $OTHER_ECS_CLUSTERS; do
    consul_server_join_wan_ips=$(/ecs-discoverer -c "${j}" -s "${ECS_SERVICE_NAME}")
    if [ $? -ne 0 ]; then
      echo "$(date) : No other WAN nodes found to join for Cluster '${j}', proceeding anyway (another node or cluster can attempt a join instead later). Error:"
      printf "${consul_server_join_wan_ips}"
      echo ""
    else
      echo "==> $(date) : WAN nodes (for Cluster '${j}') found to join - ${consul_server_join_wan_ips}"
      for i in $consul_server_join_wan_ips; do
        # Custom port of 8304 allows for communication to Consul servers on alternate Serf WAN port.
        CONSUL_JOIN_WAN_PARAM="${CONSUL_JOIN_WAN_PARAM} -retry-join-wan=${i}:8304"
      done
    fi
  done
fi

IFS="$old_ifs"

# Get the private IP of the EC2 instance to use as an advertise address.
get_private_ip=$(curl -s http://169.254.169.254/2014-11-05/meta-data/local-ipv4)
get_private_ip_exit_code=$?
if [ $get_private_ip_exit_code -ne 0 ]; then
  echo "Non-zero exit code (${get_private_ip_exit_code}) received attempting to fetch EC2 Private IP from Metadata Service"
  printf "$get_private_ip"
  exit 1
fi
# TODOLATER: use a public IP for WAN advertisement? we will need a DNS mechanism of sorts to allow other region
# nodes to find nodes in the current region for this... multi-region cluster discovery of sorts.
# For now we are using a single region, and as such have internal connectivity between all nodes. Not required yet.
CONSUL_ADVERTISE="-advertise=${get_private_ip} -advertise-wan=${get_private_ip}"

# -node not specified for servers, we rely on the container ID. agents get an IP based node specified.

# TODO: remove once reviewed
#trap 'echo "SIGTERM received, initiate a leave then a graceful shutdown."; /bin/consul leave -rpc-addr=127.0.0.1:8401; sleep 3; kill -TERM $PID; sleep 3; ps aux | grep consul' SIGTERM

# END custom docker-entrypoint.sh
#########################################
# START upstream docker-entrypoint.sh

# If the user is trying to run Consul directly with some arguments, then
# pass them to Consul.
if [ "${1:0:1}" = '-' ]; then
    set -- consul "$@"
fi

# Look for Consul subcommands.
if [ "$1" = 'agent' ]; then
    shift
    ###
    # Modified for internal usage, extra variables
    set -- consul agent \
        -data-dir="$CONSUL_DATA_DIR" \
        -config-dir="$CONSUL_CONFIG_DIR" \
        $CONSUL_BIND \
        $CONSUL_CLIENT \
        $CONSUL_ADVERTISE \
        $CONSUL_DC \
        $CONSUL_JOIN_PARAM \
        $CONSUL_JOIN_WAN_PARAM
        "$@"
elif [ "$1" = 'version' ]; then
    # This needs a special case because there's no help output.
    set -- consul "$@"
elif consul --help "$1" 2>&1 | grep -q "consul $1"; then
    # We can't use the return code to check for the existence of a subcommand, so
    # we have to use grep to look for a pattern in the help output.
    set -- consul "$@"
fi

# If we are running Consul, make sure it executes as the proper user.
if [ "$1" = 'consul' ]; then
    # If the data or config dirs are bind mounted then chown them.
    # Note: This checks for root ownership as that's the most common case.
    if [ "$(stat -c %u /consul/data)" = '0' ]; then
        chown consul:consul /consul/data
    fi
    if [ "$(stat -c %u /consul/config)" = '0' ]; then
        chown consul:consul /consul/config
    fi

    # If requested, set the capability to bind to privileged ports before
    # we drop to the non-root user. Note that this doesn't work with all
    # storage drivers (it won't work with AUFS).
    if [ ! -z ${CONSUL_ALLOW_PRIVILEGED_PORTS+x} ]; then
        setcap "cap_net_bind_service=+ep" /bin/consul
    fi

    set -- gosu consul "$@"
fi

exec "$@"

config-server.json:

{
  "bootstrap_expect": 3,
  "client_addr": "0.0.0.0",
  "ui": true,
  "server": true,
  "skip_leave_on_interrupt": false,
  "leave_on_terminate": true,
  "retry_interval": "15s",
  "retry_interval_wan": "15s",
  "translate_wan_addrs": false,
  "dns_config": {
    "allow_stale": false,
    "only_passing": true
  },
  "ports": {
    "serf_lan": 8303,
    "serf_wan": 8304,
    "rpc": 8401,
    "http": 8501,
    "dns": -1
  }
}

The `consul` user does not have write access to /consul/data in the offical docker image.

hashicorp/consul#2025

@jhmartin
the z or Z options do not work for me

Ubuntu 16.04 , no SELinux,

$ docker version
Client:
Version: 1.11.2
API version: 1.23
Go version: go1.5.4
Git commit: b9f10c9
Built: Wed Jun 1 22:00:43 2016
OS/Arch: linux/amd64

Server:
Version: 1.11.2
API version: 1.23
Go version: go1.5.4
Git commit: b9f10c9
Built: Wed Jun 1 22:00:43 2016
OS/Arch: linux/amd64

docker logs:

$ docker logs consul
WARNING: ca_cert.pem does not contain exactly one certificate or CRL: skipping
WARNING: ca-certificates.crt does not contain exactly one certificate or CRL: skipping
WARNING: ca-cert-consulca.pem does not contain exactly one certificate or CRL: skipping
==> WARNING: Expect Mode enabled, expecting 3 servers
==> Starting Consul agent...
==> Error starting agent: Failed to configure keyring: mkdir /data/serf: permission denied

Allow users to bind and interface to set CONSUL_HTTP_ADDR via entrypoint script

Because of https://github.com/aws/amazon-ecs-agent/issues/3, I'm having a very hard time passing in an "arbitrary" ENV variable value via the ECS Task Definition. My ECS hosts do have an ENV variable value that is the IP of the eth0 but like I said, super hard if not impossible to pass that in via task definition.

I'm proposing to enhance the entrypoint script to allow users to bind CONSUL_HTTP_ADDR to an interface, such as eth0.

Cannot launch official consul container on Mac (El Capitan)

I got the following error on my MAC after build the container successfully:

docker build -t test_consul . (The build is successful.)
docker run test_consul

Error response from daemon: Container command 'docker-entrypoint.sh' not found or does not exist.

I am running docker-machine version 0.4.1 (e2c88d6).

I tried to change the entry point pathname to /usr/bin, not working either.

Failed to take snapshot: failed to close snapshot

Hi !
With the latest image of consul, the sync after a snapshot failed.

Image: consul:latest or sha256:dff07cab6abd4a76549acba60b429dbacc1b5007bb6d76d0060add3ac354e4b5

Error:

traefik_consul.1.51u6mwjpoavl@docker-1    |     2017/11/17 16:43:08 [ERR] raft: Failed to take snapshot: failed to close snapshot: sync /consul/data/raft/snapshots: invalid argument
traefik_consul.1.51u6mwjpoavl@docker-1    |     2017/11/17 16:43:17 [INFO] consul.fsm: snapshot created in 23.901µs
traefik_consul.1.51u6mwjpoavl@docker-1    |     2017/11/17 16:43:17 [INFO] raft: Starting snapshot up to 444144
traefik_consul.1.51u6mwjpoavl@docker-1    |     2017/11/17 16:43:17 [INFO] snapshot: Creating new snapshot at /consul/data/raft/snapshots/4-444144-1510936997578.tmp
traefik_consul.1.51u6mwjpoavl@docker-1    |     2017/11/17 16:43:17 [ERR] snapshot: Failed syncing parent directory /consul/data/raft/snapshots, error: sync /consul/data/raft/snapshots: invalid argument

After checking sync on busybox, we need to add -f for syncing the directory/file.

/ # sync --help
BusyBox v1.26.2 (2017-10-04 13:37:41 GMT) multi-call binary.

Usage: sync [-df] [FILE]...

Write all buffered blocks (in FILEs) to disk
	-d	Avoid syncing metadata
	-f	Sync filesystems underlying FILEs

Example:

/ # sync /consul/data/raft/snapshots
sync: /consul/data/raft/snapshots: Invalid argument

/ # sync -f /consul/data/raft/snapshots
/ #

[ERR] memberlist: Failed to send ping: write udp 10.255.0.10:8302->10.254.0.7:8302: sendto: operation not permitted

HI,

I'm getting some errors in the seed and the nodes that I don't yet fully understand.

I'm using this docker stack / docker-compose.yml and spinning up the service(s) with:

$ docker stack deploy -c consul.yml consul

Seed logs:

==> Starting Consul agent...
==> WARNING: Expect Mode enabled, expecting 3 servers
==> Consul agent running!
           Version: 'v0.8.3'
           Node ID: 'af95ca51-bce1-b19b-e3c5-b8bf57df02f1'
         Node name: 'seed'
        Datacenter: 'dc1'
            Server: true (bootstrap: false)
       Client Addr: 127.0.0.1 (HTTP: 8500, HTTPS: -1, DNS: 8600)
      Cluster Addr: 10.254.0.7 (LAN: 8301, WAN: 8302)
    Gossip encrypt: false, RPC-TLS: false, TLS-Incoming: false
             Atlas: <disabled>

==> Log data will now stream in as it occurs:

    2017/06/07 06:41:28 [INFO] raft: Initial configuration (index=0): []
    2017/06/07 06:41:28 [INFO] raft: Node at 10.254.0.7:8300 [Follower] entering Follower state (Leader: "")
    2017/06/07 06:41:28 [INFO] serf: EventMemberJoin: seed 10.254.0.7
    2017/06/07 06:41:28 [INFO] serf: EventMemberJoin: seed.dc1 10.254.0.7
    2017/06/07 06:41:28 [INFO] consul: Adding LAN server seed (Addr: tcp/10.254.0.7:8300) (DC: dc1)
    2017/06/07 06:41:28 [INFO] consul: Handled member-join event for server "seed.dc1" in area "wan"
    2017/06/07 06:41:35 [ERR] agent: failed to sync remote state: No cluster leader
    2017/06/07 06:41:36 [WARN] raft: no known peers, aborting election
    2017/06/07 06:41:57 [INFO] serf: EventMemberJoin: 6d158c5b5562 10.255.0.13
    2017/06/07 06:41:57 [INFO] consul: Adding LAN server 6d158c5b5562 (Addr: tcp/10.255.0.13:8300) (DC: dc1)
    2017/06/07 06:41:57 [INFO] serf: EventMemberJoin: 6d158c5b5562.dc1 10.255.0.13
    2017/06/07 06:41:57 [INFO] consul: Handled member-join event for server "6d158c5b5562.dc1" in area "wan"
    2017/06/07 06:41:57 [INFO] serf: EventMemberJoin: f88dacfafd73 10.255.0.16
    2017/06/07 06:41:57 [INFO] consul: Adding LAN server f88dacfafd73 (Addr: tcp/10.255.0.16:8300) (DC: dc1)
    2017/06/07 06:41:58 [INFO] serf: EventMemberJoin: 1ae40b55c20b 10.255.0.10
    2017/06/07 06:41:59 [ERR] agent: failed to sync remote state: No cluster leader
    2017/06/07 06:42:02 [ERR] agent: coordinate update error: No cluster leader
==> Failed to check for updates: Get https://checkpoint-api.hashicorp.com/v1/check/consul?arch=amd64&os=linux&signature=b7827419-5378-9b71-0bcc-67ea0039da7d&version=0.8.3: dial tcp: lookup checkpoint-api.hashicorp.com on 127.0.0.11:53: read udp 127.0.0.1:58949->127.0.0.11:53: i/o timeout
    2017/06/07 06:42:07 [ERR] consul: Failed to confirm peer status for 6d158c5b5562: rpc error: failed to get conn: dial tcp 10.254.0.7:0->10.255.0.13:8300: i/o timeout. Retrying in 1s...
    2017/06/07 06:42:09 [INFO] memberlist: Suspect 1ae40b55c20b has failed, no acks received
    2017/06/07 06:42:09 [ERR] memberlist: Push/Pull with f88dacfafd73 failed: dial tcp 10.255.0.16:8301: i/o timeout
    2017/06/07 06:42:14 [INFO] memberlist: Marking 1ae40b55c20b as failed, suspect timeout reached (0 peer confirmations)
    2017/06/07 06:42:14 [INFO] serf: EventMemberFailed: 1ae40b55c20b 10.255.0.10
    2017/06/07 06:42:18 [ERR] consul: Failed to confirm peer status for 6d158c5b5562: rpc error: failed to get conn: dial tcp 10.254.0.7:0->10.255.0.13:8300: i/o timeout. Retrying in 2s...
    2017/06/07 06:42:19 [INFO] memberlist: Suspect f88dacfafd73 has failed, no acks received
    2017/06/07 06:42:24 [INFO] memberlist: Marking f88dacfafd73 as failed, suspect timeout reached (0 peer confirmations)
    2017/06/07 06:42:24 [INFO] serf: EventMemberFailed: f88dacfafd73 10.255.0.16
    2017/06/07 06:42:27 [INFO] serf: EventMemberJoin: f88dacfafd73.dc1 10.255.0.16
    2017/06/07 06:42:27 [INFO] consul: Handled member-join event for server "f88dacfafd73.dc1" in area "wan"
    2017/06/07 06:42:28 [ERR] agent: coordinate update error: No cluster leader
    2017/06/07 06:42:28 [INFO] serf: attempting reconnect to f88dacfafd73 10.255.0.16:8301
    2017/06/07 06:42:30 [INFO] memberlist: Suspect 6d158c5b5562 has failed, no acks received
    2017/06/07 06:42:30 [ERR] consul: Failed to confirm peer status for 6d158c5b5562: rpc error: failed to get conn: dial tcp 10.254.0.7:0->10.255.0.13:8300: i/o timeout. Retrying in 4s...
    2017/06/07 06:42:32 [ERR] agent: failed to sync remote state: No cluster leader
    2017/06/07 06:42:35 [INFO] memberlist: Marking 6d158c5b5562 as failed, suspect timeout reached (0 peer confirmations)
    2017/06/07 06:42:35 [INFO] serf: EventMemberFailed: 6d158c5b5562 10.255.0.13
    2017/06/07 06:42:36 [INFO] memberlist: Suspect 6d158c5b5562.dc1 has failed, no acks received
    2017/06/07 06:42:41 [INFO] memberlist: Suspect 6d158c5b5562 has failed, no acks received
    2017/06/07 06:42:44 [ERR] consul: Failed to confirm peer status for 6d158c5b5562: rpc error: failed to get conn: dial tcp 10.254.0.7:0->10.255.0.13:8300: i/o timeout. Retrying in 8s...
    2017/06/07 06:42:56 [ERR] agent: coordinate update error: No cluster leader
    2017/06/07 06:42:57 [INFO] serf: EventMemberJoin: f88dacfafd73 10.255.0.16
    2017/06/07 06:42:58 [INFO] serf: EventMemberJoin: 1ae40b55c20b.dc1 10.255.0.10
    2017/06/07 06:42:58 [INFO] consul: Handled member-join event for server "1ae40b55c20b.dc1" in area "wan"
    2017/06/07 06:43:01 [ERR] agent: failed to sync remote state: No cluster leader
    2017/06/07 06:43:02 [ERR] consul: Failed to confirm peer status for 6d158c5b5562: rpc error: failed to get conn: dial tcp 10.254.0.7:0->10.255.0.13:8300: i/o timeout. Retrying in 16s...
    2017/06/07 06:43:03 [INFO] serf: EventMemberJoin: 1ae40b55c20b 10.255.0.10
    2017/06/07 06:43:06 [INFO] memberlist: Marking 6d158c5b5562.dc1 as failed, suspect timeout reached (0 peer confirmations)
    2017/06/07 06:43:06 [INFO] serf: EventMemberFailed: 6d158c5b5562.dc1 10.255.0.13
    2017/06/07 06:43:06 [INFO] consul: Handled member-failed event for server "6d158c5b5562.dc1" in area "wan"
    2017/06/07 06:43:09 [INFO] memberlist: Suspect 6d158c5b5562.dc1 has failed, no acks received
    2017/06/07 06:43:10 [INFO] memberlist: Suspect f88dacfafd73 has failed, no acks received
    2017/06/07 06:43:13 [INFO] serf: EventMemberJoin: 6d158c5b5562 10.255.0.13
    2017/06/07 06:43:15 [INFO] memberlist: Marking f88dacfafd73 as failed, suspect timeout reached (0 peer confirmations)
    2017/06/07 06:43:15 [INFO] serf: EventMemberFailed: f88dacfafd73 10.255.0.16
    2017/06/07 06:43:19 [ERR] memberlist: Push/Pull with f88dacfafd73 failed: dial tcp 10.255.0.16:8301: i/o timeout
    2017/06/07 06:43:20 [INFO] memberlist: Suspect f88dacfafd73 has failed, no acks received
    2017/06/07 06:43:21 [ERR] agent: coordinate update error: No cluster leader
    2017/06/07 06:43:26 [ERR] agent: failed to sync remote state: No cluster leader
    2017/06/07 06:43:28 [ERR] consul: Failed to confirm peer status for 6d158c5b5562: rpc error: failed to get conn: dial tcp 10.254.0.7:0->10.255.0.13:8300: i/o timeout. Retrying in 32s...
    2017/06/07 06:43:31 [INFO] memberlist: Suspect 1ae40b55c20b has failed, no acks received
    2017/06/07 06:43:32 [ERR] memberlist: Push/Pull with 1ae40b55c20b.dc1 failed: dial tcp 10.255.0.10:8302: i/o timeout
    2017/06/07 06:43:36 [INFO] memberlist: Marking 1ae40b55c20b as failed, suspect timeout reached (0 peer confirmations)
    2017/06/07 06:43:36 [INFO] serf: EventMemberFailed: 1ae40b55c20b 10.255.0.10
    2017/06/07 06:43:38 [INFO] serf: attempting reconnect to 1ae40b55c20b 10.255.0.10:8301
    2017/06/07 06:43:41 [INFO] memberlist: Suspect 6d158c5b5562 has failed, no acks received
    2017/06/07 06:43:42 [INFO] memberlist: Suspect f88dacfafd73.dc1 has failed, no acks received
    2017/06/07 06:43:46 [INFO] memberlist: Marking 6d158c5b5562 as failed, suspect timeout reached (0 peer confirmations)
    2017/06/07 06:43:46 [INFO] serf: EventMemberFailed: 6d158c5b5562 10.255.0.13
    2017/06/07 06:43:47 [ERR] agent: coordinate update error: No cluster leader
    2017/06/07 06:43:53 [INFO] memberlist: Suspect 6d158c5b5562 has failed, no acks received
    2017/06/07 06:44:02 [ERR] agent: failed to sync remote state: No cluster leader
    2017/06/07 06:44:08 [ERR] consul: Failed to confirm peer status for f88dacfafd73: rpc error: failed to get conn: rpc error: lead thread didn't get connection. Retrying in 1s...
    2017/06/07 06:44:12 [INFO] memberlist: Marking f88dacfafd73.dc1 as failed, suspect timeout reached (0 peer confirmations)
    2017/06/07 06:44:12 [INFO] serf: EventMemberFailed: f88dacfafd73.dc1 10.255.0.16
    2017/06/07 06:44:12 [INFO] consul: Handled member-failed event for server "f88dacfafd73.dc1" in area "wan"
    2017/06/07 06:44:13 [INFO] serf: EventMemberJoin: 6d158c5b5562 10.255.0.13
    2017/06/07 06:44:15 [INFO] memberlist: Suspect f88dacfafd73.dc1 has failed, no acks received
    2017/06/07 06:44:18 [INFO] serf: attempting reconnect to f88dacfafd73 10.255.0.16:8301
    2017/06/07 06:44:19 [ERR] consul: Failed to confirm peer status for f88dacfafd73: rpc error: failed to get conn: dial tcp 10.254.0.7:0->10.255.0.16:8300: i/o timeout. Retrying in 2s...
    2017/06/07 06:44:22 [ERR] agent: coordinate update error: No cluster leader
    2017/06/07 06:44:24 [INFO] memberlist: Suspect 6d158c5b5562 has failed, no acks received
    2017/06/07 06:44:28 [INFO] serf: attempting reconnect to 6d158c5b5562.dc1 10.255.0.13:8302
    2017/06/07 06:44:29 [INFO] memberlist: Marking 6d158c5b5562 as failed, suspect timeout reached (0 peer confirmations)
    2017/06/07 06:44:29 [INFO] serf: EventMemberFailed: 6d158c5b5562 10.255.0.13
    2017/06/07 06:44:29 [ERR] memberlist: Push/Pull with 6d158c5b5562 failed: dial tcp 10.255.0.13:8301: i/o timeout
    2017/06/07 06:44:31 [ERR] consul: Failed to confirm peer status for f88dacfafd73: rpc error: failed to get conn: dial tcp 10.254.0.7:0->10.255.0.16:8300: i/o timeout. Retrying in 4s...
    2017/06/07 06:44:34 [INFO] memberlist: Suspect 6d158c5b5562 has failed, no acks received
    2017/06/07 06:44:37 [ERR] agent: failed to sync remote state: No cluster leader
    2017/06/07 06:44:45 [ERR] consul: Failed to confirm peer status for f88dacfafd73: rpc error: failed to get conn: dial tcp 10.254.0.7:0->10.255.0.16:8300: i/o timeout. Retrying in 8s...
    2017/06/07 06:44:48 [INFO] memberlist: Suspect 1ae40b55c20b.dc1 has failed, no acks received
    2017/06/07 06:44:51 [ERR] agent: coordinate update error: No cluster leader
    2017/06/07 06:44:53 [INFO] serf: EventMemberJoin: 1ae40b55c20b 10.255.0.10
    2017/06/07 06:44:53 [INFO] serf: EventMemberJoin: f88dacfafd73 10.255.0.16
    2017/06/07 06:44:58 [INFO] serf: attempting reconnect to 6d158c5b5562 10.255.0.13:8301
    2017/06/07 06:45:01 [INFO] serf: EventMemberJoin: 6d158c5b5562.dc1 10.255.0.13
    2017/06/07 06:45:01 [INFO] consul: Handled member-join event for server "6d158c5b5562.dc1" in area "wan"
    2017/06/07 06:45:02 [ERR] memberlist: Push/Pull with 1ae40b55c20b.dc1 failed: dial tcp 10.255.0.10:8302: i/o timeout
    2017/06/07 06:45:03 [ERR] consul: Failed to confirm peer status for f88dacfafd73: rpc error: failed to get conn: dial tcp 10.254.0.7:0->10.255.0.16:8300: i/o timeout. Retrying in 16s...
    2017/06/07 06:45:04 [INFO] memberlist: Suspect 1ae40b55c20b has failed, no acks received
    2017/06/07 06:45:09 [INFO] memberlist: Marking 1ae40b55c20b as failed, suspect timeout reached (0 peer confirmations)
    2017/06/07 06:45:09 [INFO] serf: EventMemberFailed: 1ae40b55c20b 10.255.0.10
    2017/06/07 06:45:09 [ERR] memberlist: Push/Pull with 1ae40b55c20b failed: dial tcp 10.255.0.10:8301: i/o timeout
    2017/06/07 06:45:13 [INFO] serf: EventMemberJoin: 6d158c5b5562 10.255.0.13
    2017/06/07 06:45:13 [ERR] agent: failed to sync remote state: No cluster leader
    2017/06/07 06:45:14 [INFO] memberlist: Suspect f88dacfafd73 has failed, no acks received
    2017/06/07 06:45:18 [INFO] memberlist: Marking 1ae40b55c20b.dc1 as failed, suspect timeout reached (0 peer confirmations)
    2017/06/07 06:45:18 [INFO] serf: EventMemberFailed: 1ae40b55c20b.dc1 10.255.0.10
    2017/06/07 06:45:18 [INFO] consul: Handled member-failed event for server "1ae40b55c20b.dc1" in area "wan"
    2017/06/07 06:45:19 [INFO] memberlist: Marking f88dacfafd73 as failed, suspect timeout reached (0 peer confirmations)
    2017/06/07 06:45:19 [INFO] serf: EventMemberFailed: f88dacfafd73 10.255.0.16
    2017/06/07 06:45:21 [INFO] memberlist: Suspect 1ae40b55c20b.dc1 has failed, no acks received
    2017/06/07 06:45:25 [INFO] memberlist: Suspect 6d158c5b5562 has failed, no acks received
    2017/06/07 06:45:27 [ERR] agent: coordinate update error: No cluster leader
    2017/06/07 06:45:28 [INFO] serf: attempting reconnect to f88dacfafd73.dc1 10.255.0.16:8302
    2017/06/07 06:45:29 [ERR] consul: Failed to confirm peer status for f88dacfafd73: rpc error: failed to get conn: dial tcp 10.254.0.7:0->10.255.0.16:8300: i/o timeout. Retrying in 32s...
    2017/06/07 06:45:30 [INFO] memberlist: Marking 6d158c5b5562 as failed, suspect timeout reached (0 peer confirmations)
    2017/06/07 06:45:30 [INFO] serf: EventMemberFailed: 6d158c5b5562 10.255.0.13
    2017/06/07 06:45:36 [INFO] memberlist: Suspect 6d158c5b5562 has failed, no acks received
    2017/06/07 06:45:37 [ERR] agent: failed to sync remote state: No cluster leader
    2017/06/07 06:45:38 [INFO] serf: attempting reconnect to 6d158c5b5562 10.255.0.13:8301
    2017/06/07 06:45:56 [INFO] memberlist: Suspect 6d158c5b5562.dc1 has failed, no acks received
    2017/06/07 06:46:01 [ERR] agent: coordinate update error: No cluster leader
    2017/06/07 06:46:01 [INFO] consul: Found expected number of peers, attempting bootstrap: 10.254.0.7:8300,10.255.0.13:8300,10.255.0.16:8300
    2017/06/07 06:46:01 [INFO] consul: Adding LAN server 1ae40b55c20b (Addr: tcp/10.255.0.10:8300) (DC: dc1)
    2017/06/07 06:46:01 [INFO] consul: Removing LAN server 1ae40b55c20b (Addr: tcp/10.255.0.10:8300) (DC: dc1)
    2017/06/07 06:46:01 [INFO] consul: Removing LAN server f88dacfafd73 (Addr: tcp/10.255.0.16:8300) (DC: dc1)
    2017/06/07 06:46:01 [INFO] consul: Removing LAN server 6d158c5b5562 (Addr: tcp/10.255.0.13:8300) (DC: dc1)
    2017/06/07 06:46:01 [INFO] consul: Adding LAN server f88dacfafd73 (Addr: tcp/10.255.0.16:8300) (DC: dc1)
    2017/06/07 06:46:01 [INFO] consul: Adding LAN server 1ae40b55c20b (Addr: tcp/10.255.0.10:8300) (DC: dc1)
    2017/06/07 06:46:01 [INFO] consul: Adding LAN server 6d158c5b5562 (Addr: tcp/10.255.0.13:8300) (DC: dc1)
    2017/06/07 06:46:01 [INFO] consul: Removing LAN server f88dacfafd73 (Addr: tcp/10.255.0.16:8300) (DC: dc1)
    2017/06/07 06:46:01 [INFO] consul: Removing LAN server 1ae40b55c20b (Addr: tcp/10.255.0.10:8300) (DC: dc1)
    2017/06/07 06:46:01 [INFO] consul: Removing LAN server 6d158c5b5562 (Addr: tcp/10.255.0.13:8300) (DC: dc1)
    2017/06/07 06:46:01 [INFO] consul: Adding LAN server 6d158c5b5562 (Addr: tcp/10.255.0.13:8300) (DC: dc1)
    2017/06/07 06:46:01 [INFO] consul: Removing LAN server 6d158c5b5562 (Addr: tcp/10.255.0.13:8300) (DC: dc1)
    2017/06/07 06:46:01 [INFO] consul: Adding LAN server 1ae40b55c20b (Addr: tcp/10.255.0.10:8300) (DC: dc1)
    2017/06/07 06:46:01 [INFO] consul: Adding LAN server f88dacfafd73 (Addr: tcp/10.255.0.16:8300) (DC: dc1)
    2017/06/07 06:46:01 [INFO] consul: Removing LAN server 1ae40b55c20b (Addr: tcp/10.255.0.10:8300) (DC: dc1)
    2017/06/07 06:46:01 [INFO] consul: Adding LAN server 6d158c5b5562 (Addr: tcp/10.255.0.13:8300) (DC: dc1)
    2017/06/07 06:46:01 [INFO] consul: Removing LAN server f88dacfafd73 (Addr: tcp/10.255.0.16:8300) (DC: dc1)
    2017/06/07 06:46:01 [INFO] consul: Removing LAN server 6d158c5b5562 (Addr: tcp/10.255.0.13:8300) (DC: dc1)
    2017/06/07 06:46:04 [WARN] raft: Heartbeat timeout from "" reached, starting election
    2017/06/07 06:46:04 [INFO] raft: Node at 10.254.0.7:8300 [Candidate] entering Candidate state in term 2
    2017/06/07 06:46:10 [WARN] raft: Election timeout reached, restarting election
    2017/06/07 06:46:10 [INFO] raft: Node at 10.254.0.7:8300 [Candidate] entering Candidate state in term 3
    2017/06/07 06:46:13 [INFO] serf: EventMemberJoin: 6d158c5b5562 10.255.0.13
    2017/06/07 06:46:13 [INFO] consul: Adding LAN server 6d158c5b5562 (Addr: tcp/10.255.0.13:8300) (DC: dc1)
    2017/06/07 06:46:14 [ERR] raft: Failed to make RequestVote RPC to {Voter 10.255.0.16:8300 10.255.0.16:8300}: dial tcp 10.254.0.7:0->10.255.0.16:8300: i/o timeout
    2017/06/07 06:46:14 [ERR] raft: Failed to make RequestVote RPC to {Voter 10.255.0.13:8300 10.255.0.13:8300}: dial tcp 10.254.0.7:0->10.255.0.13:8300: i/o timeout
    2017/06/07 06:46:14 [ERR] agent: failed to sync remote state: No cluster leader
    2017/06/07 06:46:18 [INFO] serf: attempting reconnect to f88dacfafd73 10.255.0.16:8301
    2017/06/07 06:46:19 [WARN] raft: Election timeout reached, restarting election
    2017/06/07 06:46:19 [INFO] raft: Node at 10.254.0.7:8300 [Candidate] entering Candidate state in term 4
    2017/06/07 06:46:20 [ERR] raft: Failed to make RequestVote RPC to {Voter 10.255.0.16:8300 10.255.0.16:8300}: dial tcp 10.254.0.7:0->10.255.0.16:8300: i/o timeout
    2017/06/07 06:46:20 [ERR] raft: Failed to make RequestVote RPC to {Voter 10.255.0.13:8300 10.255.0.13:8300}: dial tcp 10.254.0.7:0->10.255.0.13:8300: i/o timeout
    2017/06/07 06:46:25 [INFO] memberlist: Suspect 6d158c5b5562 has failed, no acks received
    2017/06/07 06:46:26 [INFO] memberlist: Marking 6d158c5b5562.dc1 as failed, suspect timeout reached (0 peer confirmations)
    2017/06/07 06:46:26 [INFO] serf: EventMemberFailed: 6d158c5b5562.dc1 10.255.0.13
    2017/06/07 06:46:26 [INFO] consul: Handled member-failed event for server "6d158c5b5562.dc1" in area "wan"
==> Caught signal: terminated
    2017/06/07 06:46:28 [INFO] agent: requesting shutdown
    2017/06/07 06:46:28 [INFO] consul: shutting down server
    2017/06/07 06:46:28 [WARN] serf: Shutdown without a Leave
    2017/06/07 06:46:28 [ERR] dns: error starting tcp server: accept tcp 127.0.0.1:8600: use of closed network connection
    2017/06/07 06:46:28 [WARN] serf: Shutdown without a Leave
    2017/06/07 06:46:29 [ERR] raft: Failed to make RequestVote RPC to {Voter 10.255.0.16:8300 10.255.0.16:8300}: dial tcp 10.254.0.7:0->10.255.0.16:8300: i/o timeout
    2017/06/07 06:46:29 [ERR] raft: Failed to make RequestVote RPC to {Voter 10.255.0.13:8300 10.255.0.13:8300}: dial tcp 10.254.0.7:0->10.255.0.13:8300: i/o timeout
    2017/06/07 06:46:29 [INFO] agent: shutdown complete

Sample node logs:

    2017/06/07 06:54:49 [ERR] memberlist: Failed to send ping: write udp 10.255.0.10:8301->10.254.0.7:8301: sendto: operation not permitted
    2017/06/07 06:54:49 [ERR] memberlist: Push/Pull with seed.dc1 failed: dial tcp 10.254.0.7:8302: getsockopt: no route to host
    2017/06/07 06:54:51 [ERR] memberlist: Failed to send ping: write udp 10.255.0.10:8301->10.255.0.13:8301: sendto: operation not permitted
    2017/06/07 06:54:52 [ERR] memberlist: Failed to send ping: write udp 10.255.0.10:8301->10.254.0.7:8301: sendto: operation not permitted
    2017/06/07 06:54:52 [ERR] memberlist: Push/Pull with seed failed: dial tcp 10.254.0.7:8301: getsockopt: no route to host

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.