Giter Site home page Giter Site logo

duke-lv / deploy-consul-on-kubernetes Goto Github PK

View Code? Open in Web Editor NEW
27.0 3.0 15.0 141 KB

runing a three nodes consul cluster on kubernetes,fork from consul-on-kubernetes of kelseyhightower; i change sth to adapter to my test environment.

License: Apache License 2.0

Shell 100.00%

deploy-consul-on-kubernetes's Introduction

在kubernetes上部署consul 集群

本教程将帮助你在kubernetes上部署一个拥有3个节点的consul集群 备注:consul教程见Consul

预览

预备知识

本教程利用了Kubernetes 1.7.0和更高版本的特性.

下客户端必须安装在本教程所使用的机器上:

使用

Clone this repo:

git clone https://github.com/nicklv/consul_cluster_on_kubernetes.git

进入到 consul-on-kubernetes 目录:

cd consul-on-kubernetes

生成 TLS 证书

Consul集群中成员之间的RPC通信使用TLS进行加密。通过一下命令初始化CA证书:

cfssl gencert -initca ca/ca-csr.json | cfssljson -bare ca

使用以下命令创建 TLS 证书 和 私有密钥:

cfssl gencert \
  -ca=ca.pem \
  -ca-key=ca-key.pem \
  -config=ca/ca-config.json \
  -profile=default \
  ca/consul-csr.json | cfssljson -bare consul

执行完以上命令之后,在当前目录你应该看到以下证书文件:

ca-key.pem
ca.pem
consul-key.pem
consul.pem

生成 Consul Gossip 加密密钥

Gossip communication Consul 集群成员之间使用共享的加密密钥进行加密通信。使用以下命令生成加密密钥:

GOSSIP_ENCRYPTION_KEY=$(consul keygen)

使用以下命令生成Consul 集群使用的Secret 和 Configmap

Consul集群将通过CLI标志、TLS证书和配置文件来完成配置,这些内容通过Kubernetes的configmaps和secrets来存储。

将 gossip 加密密钥 和 TLS 证书 存储在kubernetes的 Secret中:

kubectl create secret generic consul \
  --from-literal="gossip-encryption-key=${GOSSIP_ENCRYPTION_KEY}" \
  --from-file=ca.pem \
  --from-file=consul.pem \
  --from-file=consul-key.pem

将Consul集群使用的配置文件存储在kubernetes的 ConfigMap中:

kubectl create configmap consul --from-file=configs/server.json

在创建Consul service和statefulSet之前,先创建Consul集群使用到的pv(persistVolume)持久化存储卷和pvc(persistVolumeClaim)持久化存储卷声明

kubectl create -f pvc/pvc.yaml 

通过kubectl 命令创建Consul 集群服务

通过创建一个service来暴露Consul 集群功能:

kubectl create -f services/consul.yaml

创建 Consul 集群 StatefulSet

通过部署StatefulSet(有状态副本集)来启动Consul集群的3个实例节点:

kubectl create -f statefulsets/consul.yaml

所有的Consul节点都会被创建,在进入下一步操作之前,先验证所有的节点都处在正常的 Running 状态,执行以下命令可以看到如下结果:

kubectl get pods
NAME       READY     STATUS    RESTARTS   AGE
consul-0   1/1       Running   0          50s
consul-1   1/1       Running   0          29s
consul-2   1/1       Running   0          15s

核实节点状态

在这个环节,Consule 集群的所有节点都已经启动。为了确定每个节点都已经正常的工作,可以通过查看日志的命令,查看每个集群的成员状态。

kubectl logs consul-0

(这一步可不执行)可以使用Consul CLI检查consul集群的健康状态,可以在新的命令窗口使用port-forward模式来通过IP访问,在本教程的中我们的测试环境申请了域名,通过花生壳支持外网访问。

kubectl port-forward consul-0 8400:8400
Forwarding from 127.0.0.1:8400 -> 8400
Forwarding from [::1]:8400 -> 8400

(如果不执行上一步命令,可以通过以下命令进入到consul-0 pod,kubectl exec -it consul-0 /bin/sh) 执行consul members命令查看consul集群所有节点的状态。

consul members
Node      Address           Status  Type    Build  Protocol  DC
consul-0  10.244.2.197:8301  alive   server  0.9.1  2        dc1
consul-1  10.244.1.231:8301  alive   server  0.9.1  2        dc1
consul-2  10.244.2.198:8301  alive   server  0.9.1  2        dc1

访问consul 集群Web UI

目前Consul集群的UI站点不支持权限验证,所以为了安全起见不建议在生产环境把Web UI访问暴露到外网。访问Web-UI有两种访问方式:

  1. 通过port-forward把consul-0节点的8500端口暴露出来,这种方式类似nodeport模式,所有访问consul-0节点所在服务器8500端口的流量都被转发到consul-0。弊端,节点重启之后所在服务器节点发生变化的话访问IP也跟着变化。建议在实验环境使用这种模式。
kubectl port-forward consul-0 8500:8500
执行以上命令之后,通过URL地址 http://127.0.0.1:8500访问consul集群web管理界面。
  1. 通过Ingress暴露外网访问,我们配置了一个consul.xxxx.cn的域名,暴露到了外网。这种模式,外网可以无限制进行consul集群web管理界面访问,所以建议在开发或者测试环境使用,不要在生产环境使用该模式。

Image of Consul UI

清理

运行cleanup脚本,会自动清除掉所有正在运行的consul相关的kubernetes资源。备注:根据实际情况调整下 cleanup脚本。

bash cleanup

deploy-consul-on-kubernetes's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

deploy-consul-on-kubernetes's Issues

通讯证书过期

证书过期了怎么更新证书啊。
2020/11/26 04:32:46 [ERR] raft: Failed to make RequestVote RPC to {Voter ed14b1ad-5080-29da-c4c8-11030bf2ca60 10.254.11.25:8300}: x509: certificate has expired or is not yet valid

不支持新镜像1.9版本

使用该方案部署,并修改image为1.9.4版本就会导致没有leader,请问怎么解决?报错如下:
[ERR] agent: failed to sync remote state: No cluster leader
2021/03/31 08:30:11 [WARN] raft: Unable to get address for server id 172.17.0.12:8300, using fallback address 172.17.0.12:8300: Could not find address for server id 172.17.0.12:8300 2021/03/31 08:30:11 [WARN] raft: Unable to get address for server id 172.17.0.11:8300, using fallback address 172.17.0.11:8300: Could not find address for server id 172.17.0.11:8300 2021/03/31 08:30:12 [INFO] consul: New leader elected: consul-1 2021/03/31 08:30:17 [INFO] consul: New leader elected: consul-1 2021/03/31 08:30:18 [ERR] agent: failed to sync remote state: No cluster leader 2021/03/31 08:30:19 [WARN] raft: Election timeout reached, restarting election 2021/03/31 08:30:19 [INFO] raft: Node at 172.17.0.9:8300 [Candidate] entering Candidate state in term 224 2021/03/31 08:30:19 [WARN] raft: Unable to get address for server id 172.17.0.12:8300, using fallback address 172.17.0.12:8300: Could not find address for server id 172.17.0.12:8300 2021/03/31 08:30:19 [WARN] raft: Unable to get address for server id 172.17.0.11:8300, using fallback address 172.17.0.11:8300: Could not find address for server id 172.17.0.11:8300 2021/03/31 08:30:19 [INFO] raft: Node at 172.17.0.9:8300 [Follower] entering Follower state (Leader: "") 2021/03/31 08:30:23 [WARN] agent: Check "service:demo-service:192.168.22.104:8765" HTTP request failed: Get http://192.168.22.104:8765/actuator/health: dial tcp 192.168.22.104:8765: connect: connection refused 2021/03/31 08:30:25 [ERR] agent: failed to sync remote state: No cluster leader 2021/03/31 08:30:25 [INFO] consul: New leader elected: consul-1 2021/03/31 08:30:25 [WARN] raft: Heartbeat timeout from "" reached, starting election 2021/03/31 08:30:25 [INFO] raft: Node at 172.17.0.9:8300 [Candidate] entering Candidate state in term 226 2021/03/31 08:30:25 [WARN] raft: Unable to get address for server id 172.17.0.12:8300, using fallback address 172.17.0.12:8300: Could not find address for server id 172.17.0.12:8300 2021/03/31 08:30:25 [WARN] raft: Unable to get address for server id 172.17.0.11:8300, using fallback address 172.17.0.11:8300: Could not find address for server id 172.17.0.11:8300 2021/03/31 08:30:33 [WARN] raft: Election timeout reached, restarting election 2021/03/31 08:30:33 [INFO] raft: Node at 172.17.0.9:8300 [Candidate] entering Candidate state in term 227 2021/03/31 08:30:33 [WARN] raft: Unable to get address for server id 172.17.0.12:8300, using fallback address 172.17.0.12:8300: Could not find address for server id 172.17.0.12:8300 2021/03/31 08:30:33 [WARN] raft: Unable to get address for server id 172.17.0.11:8300, using fallback address 172.17.0.11:8300: Could not find address for server id 172.17.0.11:8300 2021/03/31 08:30:33 [ERR] agent: failed to sync remote state: No cluster leader 2021/03/31 08:30:35 [INFO] consul: New leader elected: consul-2
image

进行pvc挂载报错权限问题

备注:pv和pvc都有读写权限。
以下是报错信息
==> Starting Consul agent...
BootstrapExpect is set to 1; this is the same as Bootstrap mode.
bootstrap = true: do not enable unless necessary
==> Error starting agent: Failed to setup node ID: open /data/consul/node-id: permission denied

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.