winnerineast/coreos-kubernetes-generator

CoreOS Kubernetes Ignition generator

coreos-kubernetes-generator

This document provides instructions on how to install and use the Python Kubernetes Generator Script. the script helps automate a Kubernetes deployment on CoreOS. See the deployment notes for additional details.

This repository includes the Python Kubernetes Generator Script, as well as pre configured samples.

1. The script helps automate a Kubernetes Multi-Master deployment on CoreOS.
2. The script generates a CoreOS Ignition file that can be used, either as part of a 3 node Master cluster or just as a worker node.
3. The generated Ignition file includes all required properties for, Etcd, Flannel(with CNI), Kubelet using (Rkt), etc.. all protocols are configured to use SSL.

NEW! version 0.8 is now available to run as a Docker image (elik1001/coreos-kubernetes-generator)
To use the Docker image just run the below.
Note: Output for use will be saved in the configs directory. all settings selected doing run time, will be stored in a list of directory's like ssl, ssh keys, etc...

docker run \
-e PYTHONUNBUFFERED=0 \
-v $(pwd)/configs:/kub-generator/configs:rw,shared \
-v $(pwd)/keys:/kub-generator/keys:rw,shared \
-v $(pwd)/bin:/kub-generator/bin:rw,shared \
-v $(pwd)/ssl:/kub-generator/ssl:rw,shared \
-v $(pwd)/work:/kub-generator/work:rw,shared \
-v $(pwd)/tmp:/kub-generator/tmp:rw,shared \
--rm -it elik1001/coreos-kubernetes-generator:0.8.5

# Behind a proxy, use the below.
docker run \
-e PYTHONUNBUFFERED=0 \
--env HTTP_PROXY=$http_proxy \
--env HTTPS_PROXY=$http_proxy \
--env NO_PROXY=$no_proxy \
--env http_proxy=$http_proxy \
--env https_proxy=$http_proxy \
--env no_proxy=$no_proxy \
-v $(pwd)/configs:/kub-generator/configs:rw,shared \
-v $(pwd)/keys:/kub-generator/keys:rw,shared \
-v $(pwd)/bin:/kub-generator/bin:rw,shared \
-v $(pwd)/ssl:/kub-generator/ssl:rw,shared \
-v $(pwd)/work:/kub-generator/work:rw,shared \
-v $(pwd)/tmp:/kub-generator/tmp:rw,shared \
--rm -it elik1001/coreos-kubernetes-generator:0.8.5

The script uses a wizard-like approach with a minimum set of questions.
The script then generates a workable CoreOS configuration file(s) in 3 formats.

1. YAML The script uses the yaml file to generate the ignition file (.ign).
2. IGN The ignition file is used by the CoreOS live CD to create / install the OS with all required configurations.
3. ISO The script then creates / generates an ISO containing the ignition file, so you can just mount the ISO in the CoreOS live CD to install.

Note: Version (0.7+) will only work with Kubernetes version 1.13.x+

You run the script for each Node Master or Worker.
The first time you run the script it generates the SSH keys, SSL CA and SSL keys, you select / modify the options like dns, domain, proxy, etc..
You re-run the script for each master or worker node, it will remember/keep all your previous settings.
For each run/Node it will generated an output ignition configuration file in 3 formats (YAML, IGN, ISO) for you to use.

With this configuration you can hopefully run / configure a new Kubernetes in a matter of minutes.

Getting Started

Please read the change log before you begin.

Installation

Docker Image Usage

To use the Docker image just run the below.
Note: Output for use will be saved in the configs directory. all settings selected doing run time, will be stored in a list of directory's like ssl, ssh keys, etc...

docker run \
-e PYTHONUNBUFFERED=0 \
--env HTTP_PROXY=$http_proxy --env HTTPS_PROXY=$http_proxy --env NO_PROXY=$no_proxy \
--env http_proxy=$http_proxy --env https_proxy=$http_proxy --env no_proxy=$no_proxy \
-v $(pwd)/configs:/kub-generator/configs:rw,shared \
-v $(pwd)/keys:/kub-generator/keys:rw,shared \
-v $(pwd)/bin:/kub-generator/bin:rw,shared \
-v $(pwd)/ssl:/kub-generator/ssl:rw,shared \
-v $(pwd)/work:/kub-generator/work:rw,shared \
-v $(pwd)/tmp:/kub-generator/tmp:rw,shared \
--rm -it elik1001/coreos-kubernetes-generator:0.8.5

You can also build / create your own Docker image, by running the below.

# Build image
docker build --no-cache \
-t coreos-kubernetes-generator:0.8.5 app

# If behind a proxy
docker build --no-cache --build-arg HTTP_PROXY=$http_proxy \
--build-arg HTTPS_PROXY=$http_proxy --build-arg NO_PROXY=$no_proxy \
--build-arg http_proxy=$http_proxy --build-arg https_proxy=$http_proxy \
--build-arg no_proxy=$no_proxy -t coreos-kubernetes-generator:0.8.5 app

Standalone Github Application

Dependencies / Prerequisites

The following libraries are required: Note: The script will (try to) download and install the required libraries (if needed).

sys,
os,
pip,
re,   
ast,   
pwd,   
uuid,   
crypt,   
shutil,   
pprint,
getpass,   
requests,   
pycryptodome,
platform,
subprocess,   
inquirer,
from contextlib import contextmanager
Crypto.PublicKey.RSA

Note: You can manually install libraries by running pip install [library]
At the current time, openssl is required, as its used as part of the certificate creation.
(this requirement will be removed once certificate creation is fully done in Python).

Usage examples

To use the application just run the ./generate_template.py.

Example You can leave most of the default values by just hitting enter - (Just replace the IP's, Hostnames if needed..).

The default network settings are below

Default cluster IP Address (everything can be overwritten/updated if liked)
Name	IP Addrss
coreos1	172.20.0.11/20
coreos2	172.20.0.12/20
coreos3	172.20.0.13/20
worker1	172.20.0.51/20
worker2	172.20.0.52/20
worker3	172.20.0.53/20
Default Gateway	172.20.0.1
Cluster Network Ranges
Name	Range
Pod CIDR	172.20.0.0/20
Cluster CIDR	10.20.0.0/21
DNS Configuration
Name	IP Address
Google DNS1	8.8.8.8
Google DNS2	8.8.4.4
Cluster DNS (requires kube-dns)	10.3.0.10
Default Domain Name	example.com

Tip: Re-run the script for every Master, worker, etc.. each run will generate a new configuration file that you can use.

./generate_template.py
[..] snip
[?] What kubernetes system type are you building ?: Master
 > Master
   Worker

[..] snip
Ignition Template created successfully.
-----------------------------------------------------------
  SSH private key is             : keys/id_rsa
  SSH public key is              : keys/id_rsa.pub
  SSL certificates are in        : ./ssl/
  Build Template file is         : configs/coreos1_template.yaml 
  Ignition Template file is      : configs/coreos1_template.ign 
  Ignition Template ISO file is  : configs/coreos1_template.iso

Details

The script generates the below list of files and Directories.

1. ssl: Contains all SSL certificates used in the configuration including the CA certificate
2. manifests: Contains all the manifests used to generate the Ignition
3. keys: Contains the SSH Private and Public keys in the configuration
4. configs: Contains the final configuration files, like. Ignition (mastr[n].ign), ISO with Ignition data on it(master[n].iso) and the YAML config(master[n].yaml).
5. tmp: Contains all temporary configuration files.
6. src: Contains the default Kubernetes properties, these files get updated at run time, based on your selections set as the default properties(for the next run).
7. template: Contains all templates or ready sample templates used in the configuration.

Additional Details

You can use the ign (or ISO with ign file) in a verity of ways. If you are using bear-metal or virtual box, an example is explained below.

First, configure VirtualBox Networking. for more details you can follow this document.

Boot from a CoreOS ISO/CD/USB which already contains the coreos-install script or use any bootable CD, then download the CoreOS coreos-install script from here.

Next, use the generated Ignition file, you can use the IGN file, or use the ISO which contains the IGN file, the process is the same (like the below).

Now, run the coreos-install with the below parameters. Note: the coreos-install script comes pre-installed on CoreOS.

mount /dev/[sr0] /media
coreos-install -d /dev/[sda] -C [stable|alpha] -i /media/master1_template.ign

Replace sr0 with your cdrom. Replace sda with your disk. Use alpha or stable channel.

Once completed just reboot the server (or virtual) and just login with the user/password you selected. Note: If using VirtualBox you login by doing ssh user@localhost -p 2011

For RBAC to work properly you will need to run the below one of the Master(s) once Kubernetes is fully up, you can verify with kubectl get all --all-namepsaces -o wide.

Add/create the below role binding (this will address admin access errors)

kubectl create clusterrolebinding cluster-admin-binding \
--clusterrole=cluster-admin \
--namespace=kube-system \
--user=admin

For the ERROR below, just run the below rolebinding.

# ERROR
W1221 15:36:11.451721       1 authentication.go:262] Unable to get configmap/extension-apiserver-authentication in kube-system.  Usually fixed by 'kubectl create rolebinding -n kube-system ROLE_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA'
configmaps "extension-apiserver-authentication" is forbidden: User "admin" cannot get resource "configmaps" in API group "" in the namespace "kube-system"

# Create rolebinding
kubectl create rolebinding -n kube-system extension-api-server-role --role=extension-apiserver-authentication-reader --serviceaccount=kube-system:admin

Add the below to the template/master_templ.txt to automaticly download and create the kubectl file.

    - path: /opt/bin/kubectl
      filesystem: root
      mode: 511 # 0555
      contents:
        remote:
          url: http://storage.googleapis.com/kubernetes-release/release/v1.13.1/bin/linux/amd64/kubectl
          verification:
            hash:
              function: sha512
              sum: d991aa36f239b4c5262077b9fa2eeb1c4931c01c5223748ed5167838b9886b8d53cfff36ebe1344db5e7c1962af90faa0902d9b0a73174c3defa1029b6a04841

Note: If you are behind a proxy the above code might/will not work, so copy the kubectl to /opt/bin manually.

To Do's

1. Complete Documentation
2. Code Optimization
3. Convert SSL subprocess to pure Python code.
4. Convert configuration files (in src directory) to a DB layer.
5. Add include matchbox, PXE, DHCP, as an option for full automation.
6. Add a Web-UI interface to manipulate properties.

Known Issues/Troubleshooting

License

This project is licensed under the MIT License - see the LICENSE file for details