teracyhq-incubator/teracy-dev-k8s

teracy-dev-k8s extension

teracy-dev-k8s's Issues

should add docs how to backup/restore k8s cluster

https://github.com/heptio/ark is an option to consider

should update to the new location sync format

target: teracy-dev v0.6.0-a5

$ vagrant status
[TeracyDev::Location::GitSynch][WARN]: Deprecated string value at location.git of location: {"git"=>"https://github.com/kubernetes-incubator/kubespray.git", "branch"=>"v2.7.0", "lookup_path"=>"/Users/hoatle/k8s-dev/lib/../extensions", "path"=>"/Users/hoatle/k8s-dev/lib/../extensions/kubespray"}, please use location.git.remote.origin instead
/Users/hoatle/k8s-dev/lib/teracy-dev/location/git_synch.rb:26:in `sync'

[TeracyDev::Location::GitSynch][WARN]: ["branch"] of location setting has been deprecated at location: {"git"=>"https://github.com/kubernetes-incubator/kubespray.git", "branch"=>"v2.7.0", "lookup_path"=>"/Users/hoatle/k8s-dev/lib/../extensions", "path"=>"/Users/hoatle/k8s-dev/lib/../extensions/kubespray"}, please use location['git'][<branch|tag|ref|dir>] instead
/Users/hoatle/k8s-dev/lib/teracy-dev/location/git_synch.rb:38:in `sync'

should support to use custom playbook

so that we can configure and use, the playbook can be within the kubespray directory or relative to the Vagrantfile.

should update to target teracy-dev v0.6.0-a3

should update:

• manifest.yaml
• rename config_default.yaml to config.yaml
• remove config_override.yaml from the .gitignore file

should add docs how to deploy HA, scalable Wordpress

using:

• rook storage service
• mysql-operator
• wordpress helm chart
• cert-manager
• nginx-ingress

should upgrade kubespray to v2.7.0

• ubuntu to 18.04
• kubernetes to 1.11.3 (by default on kuberspray so we don't have to set it explicitly)

should add docs how to get SSO work with Dex

https://github.com/coreos/dex

so that we can manage users, it's expected that this SSO service will be used for both k8s and other services on top of k8s (think about service mesh integration, eg: https://istio.io/)

should upgrade docs to use rook 1.0 instead of 0.9

rook 1.0.0 was released and we should use it instead of the current 0.9 version.

should add docs how to deploy HA, scalable Ghost

using:

• rook storage service
• mysql-operator
• wordpress helm chart
• cert-manager
• nginx-ingress

should add docs how to get localtunner work

so that we can access the provided k8s services worldwide to the local k8s cluster, this is really useful for webhook and public access.

ngrok is another choice, we prefer localtunnel because it has no limitation for custom subdomain?

https://localtunnel.github.io/www/

should set min cpus is 2

Requirement of kubeadm:

• 2 GB or more of RAM per machine (any less will leave little room for your apps).
• 2 CPUs or more.

actual: (current config)

vm_memory: 2048
vm_cpus: 1

should install require modules when ansible was installed before

related: teracyhq-incubator/teracy-dev-entry-k8s#14

when ansible was installed, vagrant will auto-detect ansible, and k8s will not install requirement!
So must be a provisioner run before k8s set up
pip install -r /path/kubespray/requirement.txt

should update manifest to target v0.6.0-a4

v0.6.0-a4 - the latest release at present - should be used

should add docs how to create HA system with live backup/restore for the wordpress deployment

This should work on local deployment and applicable for cloud deployment

should remove the swap provisioner to improve performance

actual:

provisioners:
      - _id: "k8s-0"
        name: "teracy-dev-k8s-swapoff"
        type: shell
        enabled: true
        inline: swapoff -a

current: Currently kubespray uses kubeadm by default, during the process of disabling the swap disabled.

should release v0.1.0

target: teracy-dev >= v0.6.0-a1, <= v0.7.0

should set vm_memory and vm_cpus dynamically basing on the num_instances

should calculate the recommended configuration memory and cpus so that the cluster should work normally and set it dynamically if not overridden by users.

should release v0.3.0

it's ready to be shipped

should support multiple nodes for ansible guest mode

Currently, only 1 node is supported, we should find a way to support multiple nodes.

Multiple nodes are currently being supported by ansible host mode only.

should have docs how to set up the docker registry with SSO support

SSO setup is instructed here: https://github.com/teracyhq-incubator/teracy-dev-k8s/blob/develop/docs/sso.md

Make sure that users can access the registry with the logged in account with their assigned permissions.

should update to use kubespray v2.8

https://github.com/kubernetes-sigs/kubespray/releases/tag/v2.8.0 is released

the remote git url is changed, so we need to update the location's git remote origin url.

should check client lib version (kubectl, helm) compatibility with server version

Client lib version should be the same version as server

should support LAN accessible to the master node

Currently, it's only private network supported.

We'd like to add support for public network so that we can access the cluster within the LAN

should add teracy-dev-certs >= 0.4.0 dependency and update cert-manager docs

• update dependency
• configure:
  • disable cert generation
  • enable pkcs1 generation (so that we don't have to generate it manually https://github.com/teracyhq-incubator/teracy-dev-k8s/blob/develop/docs/cert-manager.md#use-the-generated-key-pair-from-teracy-dev-certs-to-create-a-ca-issuer)
• update docs/cert-manager to remove the manual step of generating pkcs1 format

should sync supported_oses with kubespray and check to make sure it works

see: https://github.com/kubernetes-incubator/kubespray/blob/master/Vagrantfile#L15

Fedora is not yet on this project config

should add docs how to use docker client from the host to connect to docker daemon from the guest machine

so that we can use docker client from the host

I'm not sure if we need to support connecting to all nodes or just master node.

should have docs how to set up monitoring

so that we can monitor k8s and all the running services

should release v0.2.0

v0.2.0 is ready to be shipped

k8s cluster should work when $ vagrant reload or $ vagrant halt && vagrant up

Basically, when the VM nodes are up and running, the k8s cluster should work.

Expect after:

$ vagrant reload

or

$ vagrant halt
$ vagrant up

then

$ kubectl cluster-info

should connect to the k8s cluster successfully.

Actual:

k8s-dev hoatle$ kubectl cluster-info
Kubernetes master is running at https://172.17.8.101:6443

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
The connection to the server 172.17.8.101:6443 was refused - did you specify the right host or port?

should have docs how to set up the dashboard with SSO support

SSO setup is instructed here: #11 (comment)

Make sure that after login, users can access the dashboard with that logged in account.

should update sso docs to use ca-cluster-issuer from cert-manager

related: #51

should add guide how to use cert-manager along with self-signed root CA cert from teracy-dev-certs

on local k8s cluster, use the self-signed root CA cert (generated from teracy-dev-certs) as the cert issuer: https://cert-manager.readthedocs.io/en/latest/tutorials/ca/creating-ca-issuer.html

should update SSO docs by using teracy-dev-certs

we should use the generated certs from teracy-dev-certs for local dev so that we can add the CA cert to be trusted into the CA store and we can use the own CA signed cert for different domains

should auto update ~/.kube/config file with the generated admin.conf file

so that users don't have to manage this manually.

should add docs how to manage different type of data sources

different types of data sources:

• data store with ceph (rook operator)
• database
• block storage
• etc

how to deploy, live backup, backup verification and restore with best practices for a HA system by using helm chart; zero downtime upgrade/downgrade/backup/restore

• mysql
• postgresql
• mongodb
• minio

etc

should not ask users to fill in the sudo password with ansible host mode

With ansible host mode, ansible asks for the sudo password.
We should auto bypass this (maybe by auto-filling the sudo password)

    k8s-01: Running ansible-playbook...
PYTHONUNBUFFERED=1 ANSIBLE_FORCE_COLOR=true ANSIBLE_CONFIG='extensions/kubespray/ansible.cfg' ANSIBLE_HOST_KEY_CHECKING=false ANSIBLE_SSH_ARGS='-o UserKnownHostsFile=/dev/null -o IdentitiesOnly=yes -o ControlMaster=auto -o ControlPersist=60s' ansible-playbook --connection=ssh --timeout=30 --limit="all" --inventory-file=/Users/hoatle/k8s-dev-test/.vagrant/provisioners/ansible/inventory --become -v --forks=1 --flush-cache --ask-become-pass extensions/kubespray/cluster.yml
SUDO password:

Processors::Settings.sync_kubespray should be fully implemented

See: https://github.com/teracyhq-incubator/teracy-dev-k8s/blob/develop/lib/teracy-dev-k8s/processors/settings.rb#L56

should add docs how to install, upgrade, downgrade: kubectl, helm on macOS, Linux (Ubuntu) and Windows with autocompletion

We're going to use kubectl, helm on the host machine so we need docs how to install them.

Assume:

• homebrew available on macOS
• chocolatey available on Windows
• Ubuntu is preferred Linux distro

this doc should be under docs directory with markdown format

should move SUPPORTED_OS to config.yaml file instead

so that users can override, add more OS when required

Currently, SUPPORTED_OS is hard-coded.

should upgrade rook to 1.0 with rook-storage-service docs to make it work with the current develop branch

to the latest version (1.0) and make sure everything works.

Currently, I could not use the storageclass for a pod to run

  Warning  FailedMount       6s (x4 over 6m55s)  kubelet, k8s-01    Unable to mount volumes for pod "nginx-deployment-5dd94c99d-n7tpd_nginx-example(135d0da5-2193-11e9-8b79-08002702ccc4)": timeout expired waiting for volumes to attach or mount for pod "nginx-example"/"nginx-deployment-5dd94c99d-n7tpd". list of unmounted volumes=[nginx-logs]. list of unattached volumes=[nginx-logs default-token-s789z]

it works with teracy-dev-k8s v0.3.0, but not with the current develop branch.

should have docs how to set up logging

so that we can check logs of k8s and all the running services, follow and apply best practices

should add local to vagrant automatically generate file to match the localhost check in cluster.yml

actual:

PLAY [localhost] ***************************************************************
skipping: no hosts matched

PLAY [localhost] ***************************************************************
skipping: no hosts matched

expected: should run check ansible version and host network in kubespray

should update provisioner name so that we can call each provision with its name

teracy-dev-k8s-swapoff for the shell provision
and
teracy-dev-k8s for the ansible provision

$ vagrant provision --provision-with teracy-dev-k8s-swapoff

$ vagrant provision --provision-with teracy-dev-k8s

should work

should enable kubectl_localhost so that users can use kubectl right away

make sure to update README

should have docs how to set up prod k8s cluster on different cloud providers

we should leverage this project (or at least kubespray) to set up a prod k8s cluster to deploy on:

• gcp
• aws
• digitalocean
• bare metal
• etc,

so that we can bring local dev stuff to become prod stuff

should support ansible host_vars config from the yaml config file

so that:

• the extension can introduce useful default config
• users can override any inventory variables though config_default.yaml

This is very handy.

should update docs for KUBECONFIG to make it work on Windows

to prevent these kinds of errors when KUBECONFIG is not exported.

$ helm version
Client: &version.Version{SemVer:"v2.11.0", GitCommit:"2e55dbe1fdb5fdb96b75ff144a339489417b146b", GitTreeState:"clean"}
Error: Get http://localhost:8080/api/v1/namespaces/kube-system/pods?labelSelector=app%3Dhelm%2Cname%3Dtiller: dial tcp [::1]:8080: connectex: No connection could be made because the target machine actively refused it.

kubernetes-apps/helm : Helm | Lay Down Helm Manifests (RBAC) ------------ 0.87s
==> k8s-01: Running provisioner: guest-hosts-fixer (shell)...
    k8s-01: Running: inline script
Hoas-MacBook-Pro:k8s-dev hoavu$ helm version
Client: &version.Version{SemVer:"v2.12.0", GitCommit:"d325d2a9c179b33af1a024cdb5a4472b6288016a", GitTreeState:"clean"}
Error: Get http://localhost:8080/api/v1/namespaces/kube-system/pods?labelSelector=app%3Dhelm%2Cname%3Dtiller: dial tcp [::1]:8080: connect: connection refused

should make sure $ vagrant up/reload/resume should work without --provision

expected: don't need --provision
actual: https://github.com/teracyhq-incubator/teracy-dev-k8s#vagrant-reload

should update inventory path for ansible guest mode when vagrant == 2.1.5

there is a change I noticed on ansible guest mode:

• on vagrant == 2.1.5:
  => the inventory file is: "/tmp/vagrant-ansible/inventory/vagrant_ansible_local_inventory/vagrant_ansible_local_inventory"

• on vagrant older versions:
  => the inventory file is: "/tmp/vagrant-ansible/inventory/vagrant_ansible_local_inventory"

Not sure why this change is introduced which broke our extension settings to clean up files when switching between ansible modes.

should have ansible install version for guest mode to run

related teracyhq-incubator/teracy-dev-entry-k8s#10