att-comdev / openstack-helm Goto Github PK

View Code? Open in Web Editor NEW

69.0 33.0 41.0 1.44 MB

PROJECT HAS MOVED TO OPENSTACK

Home Page: https://github.com/openstack/openstack-helm

openstack-helm kubernetes minikube openstack helm

openstack-helm's Introduction

Openstack-Helm

NOTICE: This project has moved under Openstack as Openstack-Helm.

openstack-helm's People

Contributors

Stargazers

Watchers

Forkers

alanmeadows piotrprokop dtadrzak larryrensing jascott1 ss7pro wilkers-steve arashkaffamanesh pupapaik mattmceuen maris-accenture galthaus nadeemanwar akinswin naveenkunareddy ciello89 krishnaposa rmjozsa acalephstorage wilreichert v1k0d3n archyufa mstair devx renmak ellingsonmineralcompany korroot jeffaugustine bhaskarduvvuri dulek andrecolin slfletch gokulchandrap powerds akorzeni gokulpch alexxnica kryndex vikranthkongara xuehuili

openstack-helm's Issues

feat: MaaS set up to persistent volumes and k8s jobs

@larryrensing working this issue.

Ensure rolling upgrade options are present and overriddable in every chart manifest

We should set sane values for rolling dates in each chart (where relevant), ensuring that values such as maxSurge, and others are controllable by the operator.

feat: Ensure that critical services have pod disruption budgets set

http://kubernetes.io/docs/admin/disruptions/

This ensures that operators can safely drain nodes without having to understand all the complexity underneath.

Services that would require this and likely require "minAvailable:" would be:

ceph monitors (2 for quorum)
ceph osds (would think 2 but could be 1 since our default pool maintains 3 copies - this is understandably a simplification of more advanced configurations users may place on top such as pools with only 1 or 2 copies which would contradict this for now)
mariadb (2 for quorum)
All OpenStack API services: (1)
DHCP and L3 services: (1)

Kolla-Build Containers via Jenkins workflow as a Kubernetes Pod

Create a container that can then be used in a Kubernetes/Jenkins workflow. The goal should be:

Use the existing Jenkins Helm chart to deploy a complete Jenkins system that can be used for building out Kolla containers.
Add entrypoint logic (again, using Kolla-build --include-footer flags).
Should be developer initiated into the CI pipeline.
Make it useable for anyone (create sensible ENV's).

fix: correct chart typos (globally)

In the common chart we have a function called: joinListWithColon

https://github.com/att-comdev/openstack-helm/blob/master/common/templates/_funcs.tpl#L1

This should be joinListWithComma

feat/docs: Default values for development do not work on Atomic Linux

Many mount points including / are read-only on RPM-OSTREE Linux distros like Atomic, as a result, the default development values do not work: as there is no /data directory.

We should either update the documentation to tell users how to change this value, or update it to one that is acceptable for both minikube and Atomic Linux e.g: /var/lib/localkube.

https://github.com/att-comdev/openstack-helm/blob/master/mariadb/values.yaml#L22

create cinder helm charts

@DTadrzak @PiotrProkop we started to document things a little better for outside collaborators (others have expressed interest too). there's now a project under this repo that will help keep things sane.

add variables to halcyon-vagrant-kubernetes for user-defined disk sizes

@Ciello89 is working this issue.

feat: Init containers should be moved to a template in common/ utils

This would be very helpful when k8s would move init containers out of annotations.

fix: MaaS requires a database restart during upgrades (i.e. v2.0.1 to 2.0.2)

we need to do a db 'restart/db sync' in order to update maas tables when going from one version to another. it would be nice to automate this process going forward.

feat: Use PodAffinity and NodeAffinity instead of Daemonsets

In order to get rid of Daemonsets We should use a Deployments which use NodeAffinity/PodAffinity/PodAnti-Affinity.

Thanks to this approach we would be able to use only two kinds of labels (openstack-control-plane, openstack-compute-node), we don't need to take care about ovs label. It makes cluster administrating easier.

What's more It's first step to support drain command(#36) in OpenStack-helm solution.

Regarding to dynamic changes in cluster configuration. We can set replicas: 1000 so that Operators don't need to take care about increasing the amount every time, when a new server is discovered. What's more if we add PodAnti-Affinity to deployment annotations we are sure that only 1 instance per node is deployed.

security: look into restricting access to the kubernetes API (only where needed)

i would like to look at policies that could restrict access from all services having direct access to the API (wherever possible/within reason). perhaps a PoC with something like https://github.com/aporeto-inc/trireme could be used, or possibly calico/romana?

PoC: Legacy KVM under Kubernetes

@alanmeadows working this issue.

guidelines: Helm standardization of values key:pairs (and nesting)

Once the initial 0.1.0 release is cut I think it would be good idea to formally define a structure for use within values.yaml, this will make the orchestration of multiple services much easier to manage and provide a source of truth for both new contributors and services to use as a reference source.

In particular, I think this should cover (as a starting point):

Image Tags & Pull Policy
Resouces:
- Replicas
- RAM/CPU allocation
Service/Networking
- Ports
- K8s Service name/type
- Ingress
- Namespacing
Keystone
- Admin Creds
- Service Creds
Database
- URI
- Admin Creds
- User Creds
Service specific config (a generic structure layout)

NodeSelctors looking for wrong keys/values in some jobs

Commit 6f124d5 added NodeSelectors to several jobs across several components, using the value ".Vales.labels.node_selector_key". However, some components (neutron, nova) have more granular node selection, e.g. ".Values.labels.server.node_selector_key", and this is causing make to fail with errors like the following for me:

==> Linting neutron
[ERROR] templates/job-db-sync.yaml: unable to parse YAML
error converting YAML to JSON: yaml: line 9: did not find expected key
[ERROR] templates/job-init.yaml: unable to parse YAML
error converting YAML to JSON: yaml: line 9: did not find expected key
[ERROR] templates/job-post.yaml: unable to parse YAML
error converting YAML to JSON: yaml: line 9: did not find expected key

ci: repo templates for issues, automatic labeling, etc.

We need to define contribution guidelines, create issue templates and automatically label based on key words, etc.

Release 0.1.0 - Jan 10th, 2017

Foundational Elements for Openstack (charts):

Primary Openstack Services (charts):

CI Workflow(s):

Build/Push custom container images from Jenkins
- Build Docker Container-Kolla-Build Containers via Jenkins workflow as a Kubernetes Pod #20

In this release, we are able to demonstrate the following:

Demonstrate Helm-Driven Resource Quotas
Demonstrate a single service upgrade (Keystone)
Demonstrate Legacy KVM Instance: Legacy KVM artifact running as a container
Demonstrate Resiliency Aspects
- Disconnect Control Plane Host
- Re-Provision Control Plane from Scratch

PoC: Legacy KVM with Stateful Storage

docs: Document PoC Usecases for External Consumers.

We want to document the use of Registries, components, trends, Jenkins pipelines, etc for general user consumption. A good example of this is Openstack Ansible project.

helm: add resource quotas for each of the charts and default values

each of the openstack-helm charts need to have configurable resource quotas: i.e. chart/templates//deployment.yaml & chart/values.yaml

for more information: http://kubernetes.io/docs/admin/resourcequota/

chart repository: add a public-facing chart repository for project

https://github.com/kubernetes/helm/blob/master/docs/chart_repository.md#share-your-charts-with-others

this would remove the need to clone and install things locally.

Create horizon chart

Convert mariadb to a statefulset

feat: Extend rabbitmq chart with autocluster to support multiple rabbitmq instances

See https://github.com/aweber/rabbitmq-autocluster

We should investigate leveraging the DNS backend, since this should be supported natively with a kubernetes service, as opposed to using etcd.

For a working example, see https://github.com/openstack/fuel-ccp-rabbitmq - however this example uses etcd and would prefer to validate the DNS approach works.

feat: kubernetes chaos monkey chart

https://github.com/asobti/kube-monkey

Create a globals or master.yaml override harness for all independent charts

The goal here is to allow you to run:

"./tool.py master.yaml"

Where master.yaml contains:

ceph:
...
keystone:
...

This will generate:

/tmp/xk92012/ceph.yaml
/tmp/xk92012/keystone.yaml
/tmp/xk92012/horizon.yaml

You can then install charts:

helm install --values=/tmp/xk92012/horizon.yaml ./horizon-0.1.0.tgz

This is a requirement because a raw native master.yaml does not support an upper level namespace of "chartname", requiring it to be all in the same namespace if used across various charts.

This tool will be required until Helm issue #1520 is resolved or they introduce plugins that support this. Either will work.

feat: Investigate replacement for stackanetes-entrypoint

The stackanetes-entrypoint image consistently leveraged runs into issues with DAEMONSET dependencies, and has trouble with config file dependencies. These should be documented in this issue subsequently but I do not have readily available access to tracebacks at the moment.

We either need to:

Investigate whether since its creation there is a now "standard" init container for dependency checking that is being widely used
If we continue to use it, ensure we are including it in our CI/CD pipelines we will develop and be creating issues to have the issues we encountered resolved in the upstream repository: https://github.com/stackanetes/kubernetes-entrypoint

Jobs need consistent nodeSelectors to ensure they only run on the control plane

Today, in some charts, jobs may land on tenant compute infrastructure or other unintended kubernetes infrastructure. Node placement needs to be consistent for all resources including jobs.

An example job lacking nodeSelector labels below:

https://github.com/att-comdev/openstack-helm/blob/master/neutron/templates/job-db-sync.yaml

MariaDB: convert PetSets to StatefulSets in Helm Chart

@alanmeadows is working to standardize on StatefulSets, and MariaDB is impacted with this work.

migrating helm charts to kubernetes/charts

i would like to see these charts end up in kubernetes/helm upstream. in order to make this happen they need to incubate, and i would like to get eyes on them. we should talk about the low hanging fruit, and critical items (Ceph, MaaS), and work towards getting them incubated for easier consumption.

development: determine long term path

@alanmeadows we should determine a good development path for 2017. minikube supports all of the versions that we need now, however the issue will be with persistent volumes.

$ minikube get-k8s-versions
The following Kubernetes versions are available:
	- v1.5.1
	- v1.4.5
	- v1.4.3
        . . .
        . . .
        . . .

Some interesting considerations:

Updating Kubernetes for Minikube
Persistant Volumes in Minikube (See "private registries" and ingress add-on also)

I think it would be helpful for us to unblock development with an upstream, development-friendly too, and ditch Halcyon Vagrant option for development at some point, because it's just going to get hard to maintain, and will be resource intensive for developers. If our manifests are created in a way where a single daemonset is enough for consensus, than this may be a better path forward (long term). If we need changes, we can always just throw them at minikube directly.

ci: basic linting for repository

we need some basic linting and CI gating for our repository. this will make the community more confident in the changes they are submitting, and help assist reviewers before merging code into our repository as well.

feat: Template Naming Convention

Template names are global, even when used in sub-charts. As a result, it is very easy for collision to occur, with non-deterministic outcomes - which could (and has in my development work) result in effects ranging from failed services to potential security risks.

Due to this, I propose that template names, and macros, should always be scoped to the name of the chart they come from, this would require refactoring, but will I feel make the future development of charts much simpler, and more manageable.

As a result things like:

{{- define "env_ks_user_create_openrc_tpl" }}

should be written as:

{{- define "common.env_ks_user_create_openrc_tpl" }}

(from: https://github.com/att-comdev/openstack-helm/blob/master/common/templates/snippets/_ks_env_user_create_openrc.tpl#L1)

feat: Drain a Kubernetes node, and upgrade foundational elements.

PoC to demonstration draining a Kubernetes node, upgrade foundational elements (i.e. Docker, host kernel, etc), and bring back into the cluster. This is to identify a useful, and documented path for users as part of our PoC. This ends up going into Documentation after exploration.

docs: Document Contributor Guidelines and Objectives

feat: PoC snap integration with kubernetes

@andrecolin working this issue.

Token generation process for AIC-Helm

We may be able to handle secretes like this in the near future: https://github.com/andrewstuart/helm/blob/5d58b7792c4b0b56ec8eee49cfcc79980e9b0ed3/docs/chart_template_guide/accessing_files.md#secrets

Primary OpenStack services (keystone, glance, nova, neutron, heat, cinder, horizon)

Convert ceph monitors to a statefulset

The ceph monitors require reliable IP addressing, they should be converted to a statefulset.

Refactor mariadb to match keystone template layout

Convert the template layout to match the new keystone template layout, which is more readable and supports testing of injected scripts as they are not wrapped in nested YAML.

feat: Suggestion to rename bootstrap chart

@alanmeadows i'm a bit torn on this idea, because it will require some work, but I think during the 0.2.0 release we should rename the bootstrap chart to something that is a bit more clearer for new users (even thought it's lightly documented). so we can either make the documentation more clear, rename, or both. thoughts?

cLCP Demo Script

feat: Ensure init-containers are consistently used throughout every chart for dependency checks

We need to ensure that every chart consistently leverages the init-container approach.

An example of where we are doing it correctly:

https://github.com/att-comdev/openstack-helm/blob/master/keystone/templates/deployment.yaml#L11-L40

An example of something that needs conversion:

https://github.com/att-comdev/openstack-helm/blob/master/neutron/templates/deployment-server.yaml#L14-L34

[root@kub1 ~]# kubectl exec kube-controller-manager-kub1.localdomain -it -n kube-system -- /bin/bash
exec: "/bin/bash": stat /bin/bash: no such file or directory
error: error stream protocol error: invalid exit code value "-1"

however

[root@kub1 ~]# kubectl exec kube-controller-manager-kub1.localdomain -it -n kube-system -- echo $0
-bash

Any idea?

thank you