golimix/nacos-k8s

This project contains a Nacos Docker image meant to facilitate the deployment of Nacos on Kubernetes using StatefulSets.

Kubernetes Nacos

This project contains a Nacos Docker image meant to facilitate the deployment of Nacos on Kubernetes via StatefulSets.

中文文档

Tips

If you are using Nacos version 1.1.4 or lower,, please refer to this Tag

It is recommended to deploy Nacos in Kubernetes using Nacos Operator.

Quick Start

• Clone Project

git clone https://github.com/nacos-group/nacos-k8s.git

• Simple Start

If you want to start Nacos without NFS, but emptyDirs will possibly result in a loss of data. as follows:

cd nacos-k8s
chmod +x quick-startup.sh
./quick-startup.sh

• Testing

  • Service registration

  curl -X PUT 'http://cluster-ip:8848/nacos/v1/ns/instance?serviceName=nacos.naming.serviceName&ip=20.18.7.10&port=8080'

  • Service discovery

  curl -X GET 'http://cluster-ip:8848/nacos/v1/ns/instances?serviceName=nacos.naming.serviceName'

  • Publish config

  curl -X POST "http://cluster-ip:8848/nacos/v1/cs/configs?dataId=nacos.cfg.dataId&group=test&content=helloWorld"

  • Get config

  curl -X GET "http://cluster-ip:8848/nacos/v1/cs/configs?dataId=nacos.cfg.dataId&group=test"

Advanced

Tips

If you use a custom database, please initialize the database script yourself first. https://github.com/alibaba/nacos/blob/develop/distribution/conf/nacos-mysql.sql

In advanced use, the cluster is automatically scaled and data is persisted, but PersistentVolumeClaims must be deployed. In this example, NFS is used.

Deploy NFS

• Create Role

kubectl create -f deploy/nfs/rbac.yaml

If your K8S namespace is not default, execute the following script before creating RBAC

# Set the subject of the RBAC objects to the current namespace where the provisioner is being deployed
$ NS=$(kubectl config get-contexts|grep -e "^\*" |awk '{print $5}')
$ NAMESPACE=${NS:-default}
$ sed -i '' "s/namespace:.*/namespace: $NAMESPACE/g" ./deploy/nfs/rbac.yaml

• Create ServiceAccount And deploy NFS-Client Provisioner

kubectl create -f deploy/nfs/deployment.yaml

• Create NFS StorageClass

kubectl create -f deploy/nfs/class.yaml

• Verify that NFS is working

kubectl get pod -l app=nfs-client-provisioner

Deploy database

• Deploy mysql

cd nacos-k8s

kubectl create -f deploy/mysql/mysql-nfs.yaml

• Verify that Database is working

kubectl get pod 
NAME                         READY   STATUS    RESTARTS   AGE
mysql-gf2vd                        1/1     Running   0          111m

Deploy Nacos

• Modify deploy/nacos/nacos-pvc-nfs.yaml

data:
  mysql.db.name: "db name"
  mysql.port: " db port"
  mysql.user: " db username"
  mysql.password: " db password"

• Create Nacos

kubectl create -f nacos-k8s/deploy/nacos/nacos-pvc-nfs.yaml

• Verify that Nacos is working

kubectl get pod -l app=nacos


NAME      READY   STATUS    RESTARTS   AGE
nacos-0   1/1     Running   0          19h
nacos-1   1/1     Running   0          19h
nacos-2   1/1     Running   0          19h

Scale Testing

• Use kubectl exec to get the cluster config of the Pods in the nacos StatefulSet.

for i in 0 1; do echo nacos-$i; kubectl exec nacos-$i cat conf/cluster.conf; done

The StatefulSet controller provides each Pod with a unique hostname based on its ordinal index. The hostnames take the form of <statefulset name>-<ordinal index>. Because the replicas field of the nacos StatefulSet is set to 2, In the cluster file only two nacos address

• Use kubectl to scale StatefulSets

kubectl scale sts nacos --replicas=3

• Use kubectl exec to get the cluster config of the Pods in the nacos StatefulSet after scale StatefulSets

for i in 0 1 2; do echo nacos-$i; kubectl exec nacos-$i cat conf/cluster.conf; done

• Use kubectl exec to get the state of the Pods in the nacos StatefulSet after scale StatefulSets

for i in 0 1 2; do echo nacos-$i; kubectl exec nacos-$i curl GET "http://localhost:8848/nacos/v1/ns/raft/state"; done

You can find that the new node has joined the cluster

Prerequisites

• Kubernetes Node configuration(for reference only)

Hostname	Configuration
k8s-master	CentOS Linux release 7.4.1708 (Core) Single-core processor Mem 4G Cloud disk 40G
node01	CentOS Linux release 7.4.1708 (Core) Single-core processor Mem 4G Cloud disk 40G
node02	CentOS Linux release 7.4.1708 (Core) Single-core processor Mem 4G Cloud disk 40G

• Kubernetes version：1.12.2+
• NFS version：4.1+

Limitations

• Persistent Volumes must be used. emptyDirs will possibly result in a loss of data

Project directory

Directory Name	Description
plugin	Help Nacos cluster achieve automatic scaling in K8s
deploy	Deploy the required files

Configuration properties

• nacos-pvc-nfs.yaml or nacos-quick-start.yaml

Name	Required	Description
mysql.db.name	Y	database name
mysql.port	N	database port
mysql.user	Y	database username
mysql.password	Y	database password
NACOS_REPLICAS	Y	The number of clusters must be consistent with the value of the replicas attribute
NACOS_SERVER_PORT	N	Nacos port,default:8848 for Peer-finder plugin
NACOS_APPLICATION_PORT	N	Nacos port， default:8848
PREFER_HOST_MODE	Y	Enable Nacos cluster node domain name support

• nfs deployment.yaml

Name	Required	Description
NFS_SERVER	Y	NFS server address
NFS_PATH	Y	NFS server shared directory
server	Y	NFS server address
path	Y	NFS server shared directory

• mysql yaml

Name	Required	Description
MYSQL_ROOT_PASSWORD	N	Root password
MYSQL_DATABASE	Y	Database Name
MYSQL_USER	Y	Database Username
MYSQL_PASSWORD	Y	Database Password
Nfs:server	Y	NFS server address
Nfs:path	Y	NFS server shared path