Giter Site home page Giter Site logo

teracy-dev-k8s's Introduction

teracy-dev-k8s

Setting up k8s cluster on teracy-dev (v0.6) with kubespray for a production ready local k8s cluster. This can be considered a local managed k8s service that we can use it for testing, it should work the same way as any k8s cluster on the cloud.

How to use

Configure workspace/teracy-dev-entry/config_default.yaml with the following similar content:

  • Use specific version:
teracy-dev:
  extensions:
    - _id: "entry-k8s" # this _id is used for overriding by the config_override.yaml file
      path:
        extension: teracy-dev-k8s
      location:
        git:
          remote:
            origin: https://github.com/teracyhq-incubator/teracy-dev-k8s.git
          branch: v0.3.0
      require_version: ">= 0.3.0"
      enabled: true
  • Use latest stable version (auto update):
teracy-dev:
  extensions:
    - _id: "entry-k8s" # this _id is used for overriding by the config_override.yaml file
      path:
        extension: teracy-dev-k8s
      location:
        git:
          remote:
            origin: https://github.com/teracyhq-incubator/teracy-dev-k8s.git
          branch: master
      require_version: ">= 0.3.0"
      enabled: true
  • Use latest develop version (auto update):
teracy-dev:
  extensions:
    - _id: "entry-k8s" # this _id is used for overriding by the config_override.yaml file
      path:
        extension: teracy-dev-k8s
      location:
        git:
          remote:
            origin: https://github.com/teracyhq-incubator/teracy-dev-k8s.git
          branch: develop
      require_version: ">= 0.4.0-SNAPSHOT"
      enabled: true

See this example setup: https://github.com/teracyhq-incubator/teracy-dev-entry-k8s#how-to-use

Vagrant Reload

Whenever the node(s) is reloaded, make sure to provision it so that the k8s cluster is working properly

$ vagrant halt
$ vagrant up --provision
$ vagrant reload --provision

Ansible Options

By default, we copy the sample inventory from kubespray into workspace/inventory if not exists yet, so you can configure ansible from the workspace/inventory directory.

Moreover, you can also override inventory options with teracy-dev-k8s['ansible']['host_vars'], see the configuration override section below

Accessing Kubernetes API

You should see the generated artifacts within the workspace/inventory/artifacts directory

By default, the admin.conf, kubectl and kubectl.sh files are generated.

You can use:

$ cd workspace/inventory/artifacts
$ ./kubectl.sh cluster-info

You can copy the kubectl file to your local bin (/usr/local/bin/) or better to install it yourself with autocomplete.

And then set the KUBECONFIG env var for kubectl to work:

$ cd workspace/inventory/artifacts
$ export KUBECONFIG=$(pwd)/admin.conf
$ # or if you want to use existing config
$ export KUBECONFIG=$KUBECONFIG:$(pwd)/admin.conf
$ # or
$ export KUBECONFIG=$HOME/.kube/config:$(pwd)/admin.conf

You can adjust the above script for ~/.bash_profile so that it's loaded everytime a terminal session is started.

Use it:

$ kubectl config use-context admin-cluster.local
$ kubectl cluster-info

See:

Configuration Override

To override default config, you need to create workspace/teracy-dev-entry/config_override.yaml to override the values from teracy-dev-k8s/config.yaml.

For example:

teracy-dev-k8s:
  ansible:
    mode: host
    verbose: vv
    host_vars:
      kubectl_localhost: "False"
      helm_enabled: "True"
  vm_memory: 1600
  vm_cpus: 4
  num_instances: 3

How to develop

Configure workspace/teracy-dev-entry/config_override.yaml with the following similar content:

teracy-dev:
  extensions:
    - _id: "entry-k8s" # make sure the right _id matching from the config_default.yaml file
      path:
        lookup: workspace
      location:
        git:
          remote:
            origin: [email protected]:hoatle/teracy-dev-k8s.git # your forked repo
          branch: develop
      require_version: ">= 0.4.0-SNAPSHOT"

Enjoy and happy hacking!

teracy-dev-k8s's People

Contributors

hieptranquoc avatar hoatle avatar hoavt avatar phucvdb avatar

Stargazers

 avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

teracy-dev-k8s's Issues

should not ask users to fill in the sudo password with ansible host mode

With ansible host mode, ansible asks for the sudo password.
We should auto bypass this (maybe by auto-filling the sudo password)

    k8s-01: Running ansible-playbook...
PYTHONUNBUFFERED=1 ANSIBLE_FORCE_COLOR=true ANSIBLE_CONFIG='extensions/kubespray/ansible.cfg' ANSIBLE_HOST_KEY_CHECKING=false ANSIBLE_SSH_ARGS='-o UserKnownHostsFile=/dev/null -o IdentitiesOnly=yes -o ControlMaster=auto -o ControlPersist=60s' ansible-playbook --connection=ssh --timeout=30 --limit="all" --inventory-file=/Users/hoatle/k8s-dev-test/.vagrant/provisioners/ansible/inventory --become -v --forks=1 --flush-cache --ask-become-pass extensions/kubespray/cluster.yml
SUDO password:

should update inventory path for ansible guest mode when vagrant == 2.1.5

there is a change I noticed on ansible guest mode:

  • on vagrant == 2.1.5:
    => the inventory file is: "/tmp/vagrant-ansible/inventory/vagrant_ansible_local_inventory/vagrant_ansible_local_inventory"

  • on vagrant older versions:
    => the inventory file is: "/tmp/vagrant-ansible/inventory/vagrant_ansible_local_inventory"

Not sure why this change is introduced which broke our extension settings to clean up files when switching between ansible modes.

should update SSO docs by using teracy-dev-certs

we should use the generated certs from teracy-dev-certs for local dev so that we can add the CA cert to be trusted into the CA store and we can use the own CA signed cert for different domains

should add docs how to manage different type of data sources

different types of data sources:

  • data store with ceph (rook operator)
  • database
  • block storage
  • etc

how to deploy, live backup, backup verification and restore with best practices for a HA system by using helm chart; zero downtime upgrade/downgrade/backup/restore

  • mysql
  • postgresql
  • mongodb
  • minio

etc

k8s cluster should work when $ vagrant reload or $ vagrant halt && vagrant up

Basically, when the VM nodes are up and running, the k8s cluster should work.

Expect after:

$ vagrant reload

or

$ vagrant halt
$ vagrant up

then

$ kubectl cluster-info

should connect to the k8s cluster successfully.

Actual:

k8s-dev hoatle$ kubectl cluster-info
Kubernetes master is running at https://172.17.8.101:6443

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
The connection to the server 172.17.8.101:6443 was refused - did you specify the right host or port?

should update to the new location sync format

target: teracy-dev v0.6.0-a5

$ vagrant status
[TeracyDev::Location::GitSynch][WARN]: Deprecated string value at location.git of location: {"git"=>"https://github.com/kubernetes-incubator/kubespray.git", "branch"=>"v2.7.0", "lookup_path"=>"/Users/hoatle/k8s-dev/lib/../extensions", "path"=>"/Users/hoatle/k8s-dev/lib/../extensions/kubespray"}, please use location.git.remote.origin instead
/Users/hoatle/k8s-dev/lib/teracy-dev/location/git_synch.rb:26:in `sync'

[TeracyDev::Location::GitSynch][WARN]: ["branch"] of location setting has been deprecated at location: {"git"=>"https://github.com/kubernetes-incubator/kubespray.git", "branch"=>"v2.7.0", "lookup_path"=>"/Users/hoatle/k8s-dev/lib/../extensions", "path"=>"/Users/hoatle/k8s-dev/lib/../extensions/kubespray"}, please use location['git'][<branch|tag|ref|dir>] instead
/Users/hoatle/k8s-dev/lib/teracy-dev/location/git_synch.rb:38:in `sync'

should update docs for KUBECONFIG to make it work on Windows

to prevent these kinds of errors when KUBECONFIG is not exported.

$ helm version
Client: &version.Version{SemVer:"v2.11.0", GitCommit:"2e55dbe1fdb5fdb96b75ff144a339489417b146b", GitTreeState:"clean"}
Error: Get http://localhost:8080/api/v1/namespaces/kube-system/pods?labelSelector=app%3Dhelm%2Cname%3Dtiller: dial tcp [::1]:8080: connectex: No connection could be made because the target machine actively refused it.
kubernetes-apps/helm : Helm | Lay Down Helm Manifests (RBAC) ------------ 0.87s
==> k8s-01: Running provisioner: guest-hosts-fixer (shell)...
    k8s-01: Running: inline script
Hoas-MacBook-Pro:k8s-dev hoavu$ helm version
Client: &version.Version{SemVer:"v2.12.0", GitCommit:"d325d2a9c179b33af1a024cdb5a4472b6288016a", GitTreeState:"clean"}
Error: Get http://localhost:8080/api/v1/namespaces/kube-system/pods?labelSelector=app%3Dhelm%2Cname%3Dtiller: dial tcp [::1]:8080: connect: connection refused

should upgrade rook to 1.0 with rook-storage-service docs to make it work with the current develop branch

to the latest version (1.0) and make sure everything works.

Currently, I could not use the storageclass for a pod to run

  Warning  FailedMount       6s (x4 over 6m55s)  kubelet, k8s-01    Unable to mount volumes for pod "nginx-deployment-5dd94c99d-n7tpd_nginx-example(135d0da5-2193-11e9-8b79-08002702ccc4)": timeout expired waiting for volumes to attach or mount for pod "nginx-example"/"nginx-deployment-5dd94c99d-n7tpd". list of unmounted volumes=[nginx-logs]. list of unattached volumes=[nginx-logs default-token-s789z]

it works with teracy-dev-k8s v0.3.0, but not with the current develop branch.

should set min cpus is 2

Requirement of kubeadm:

  • 2 GB or more of RAM per machine (any less will leave little room for your apps).
  • 2 CPUs or more.

actual: (current config)

vm_memory: 2048
vm_cpus: 1

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.