kubernetes-sigs/cluster-api

Home for Cluster API, a subproject of sig-cluster-lifecycle

Cluster API

👋 Welcome to our project! Our Book can help you get started and provides lots of in-depth information.

Useful links

• Feature proposals
• Quick Start

✨ What is the Cluster API?

Cluster API is a Kubernetes subproject focused on providing declarative APIs and tooling to simplify provisioning, upgrading, and operating multiple Kubernetes clusters.

Started by the Kubernetes Special Interest Group (SIG) Cluster Lifecycle, the Cluster API project uses Kubernetes-style APIs and patterns to automate cluster lifecycle management for platform operators. The supporting infrastructure, like virtual machines, networks, load balancers, and VPCs, as well as the Kubernetes cluster configuration are all defined in the same way that application developers operate deploying and managing their workloads. This enables consistent and repeatable cluster deployments across a wide variety of infrastructure environments.

⚙️ Providers

Cluster API can be extended to support any infrastructure (AWS, Azure, vSphere, etc.), bootstrap or control plane (kubeadm is built-in) provider. There is a growing list of supported providers available.

🤗 Community, discussion, contribution, and support

Cluster API is developed in the open, and is constantly being improved by our users, contributors, and maintainers. It is because of you that we are able to automate cluster lifecycle management for the community. Join us!

If you have questions or want to get the latest project news, you can connect with us in the following ways:

• Chat with us on the Kubernetes Slack in the #cluster-api channel
• Subscribe to the SIG Cluster Lifecycle Google Group for access to documents and calendars
• Join our Cluster API working group sessions where we share the latest project news, demos, answer questions, and triage issues
  • Weekly on Wednesdays @ 10:00 PT on Zoom
  • Previous meetings: [ notes | recordings ]

Pull Requests and feedback on issues are very welcome! See the issue tracker if you're unsure where to start, especially the Good first issue and Help wanted tags, and also feel free to reach out to discuss.

See also our contributor guide and the Kubernetes community page for more details on how to get involved.

Code of conduct

Participation in the Kubernetes community is governed by the Kubernetes Code of Conduct.

cluster-api's Issues

MachineSet controller listens for Machine resources.

From @p0lyn0mial on March 8, 2018 14:40

The current implementation of MachineSet controller watches only the desired state of the cluster. This approach is not sufficient as there are other actors in the system that can easily change the current state (e.g. delete/add a Machine) of the cluster. Thus adding event handlers for Machines to our MachineSet along with appropriate handlers (requeue) seems to be valid approach.

See also:
#535
kubernetes-retired/kube-deploy#620 (comment)
https://github.com/kubernetes/community/blob/master/contributors/devel/controllers.md

Copied from original issue: kubernetes-retired/kube-deploy#643

UX: consider renaming "cloud" param to something else

From @rsdcastro on March 2, 2018 15:20

Users select a "cloud" to determine which provider/actuator to use, but some (like Loodse's) can target multiple providers under the covers (like Digital Ocean, AWS). And in their case (see #405), the parameter in the provider specific config is "cloud". This issue is to track renaming the top-level param to select the provider/actuator to something else more general.

cc @karan @mrIncompetent

Copied from original issue: kubernetes-retired/kube-deploy#633

Backup for cluster API objects (etcd database)

From @rsdcastro on February 14, 2018 15:41

Copied from original issue: kubernetes-retired/kube-deploy#600

Set up the repository

Following the template for a new Kubernetes repository, can we please bootstrap the repo following this guide: https://github.com/kubernetes/kubernetes-template-project

MachineSet implementation for scaling up & down

From @rsdcastro on January 29, 2018 22:27

Some thoughts/questions:

• Choose a specific instance to remove rather than arbitrary instance
• Does scaling down use any smart logic to pick the node to remove?
• Does scaling down drain nodes by default? (But allow removing machines w/o draining)

Copied from original issue: kubernetes-retired/kube-deploy#536

Document how to access extension API server in case of a disaster

From @rsdcastro on February 15, 2018 16:42

Clients should be accessing cluster API via the main K8s API server. In case of a disaster with the cluster, clients should be able to connect to the ext API server. This issue is to test and document what would be required for that to happen.

(see PR with architecture diagram below to help: https://github.com/kubernetes/kube-deploy/pull/603/files)

Copied from original issue: kubernetes-retired/kube-deploy#608

Create owners file

Follow suit of kubernetes-retired/kube-deploy#481

Original context https://docs.google.com/a/heptio.com/document/d/16ils69KImmE94RlmzjWDrkmFZysgB2J4lGnYMRN89WM/edit?disco=AAAAB1xreHk

Machine controller upgrade documentation

From @rsdcastro on February 14, 2018 15:43

This should consider the different architectures where the controller might be running.

Copied from original issue: kubernetes-retired/kube-deploy#601

Extensions to machine status errors

From @roberthbailey on January 16, 2018 7:13

Extracted from kubernetes-retired/kube-deploy#298 (comment):

Should we add a new MachineStatusError of JoinClusterTimeoutMachineError (or similar)?

Copied from original issue: kubernetes-retired/kube-deploy#503

Allow other CNI providers to be installed

From kube-deploy created by rsdcastro : kubernetes-retired/kube-deploy#635

Currently CNI provider isn't specified in the API, and the GCP implementation has support for weavenet. This issue tracks API updates to support other CNI providers.

https://github.com/kubernetes/kube-deploy/blob/master/cluster-api/api/cluster/v1alpha1/types.go#L327

Conformance Tests for Clusters with Cluster API enabled

From @rsdcastro on January 29, 2018 23:15

Copied from original issue: kubernetes-retired/kube-deploy#543

Document how to access extension API server in case of a disaster

From @rsdcastro on February 15, 2018 16:42

Clients should be accessing cluster API via the main K8s API server. In case of a disaster with the cluster, clients should be able to connect to the ext API server. This issue is to test and document what would be required for that to happen.

(see PR with architecture diagram below to help: https://github.com/kubernetes/kube-deploy/pull/603/files)

Copied from original issue: kubernetes-retired/kube-deploy#608

GCP: document how to tear down a cluster created via Cluster API

From @rsdcastro on January 29, 2018 21:55

Copied from original issue: kubernetes-retired/kube-deploy#527

Machine Set implementation

From @rsdcastro on January 29, 2018 22:24

API definition here:
kubernetes-retired/kube-deploy#501

This item tracks machine set implementation. Logic should be independent of the cloud provider.

Copied from original issue: kubernetes-retired/kube-deploy#535

Design and implement a machine lifecycle state machine

From @rsdcastro on January 29, 2018 22:6

Proposal doc: https://docs.google.com/document/d/1Kz8bmAF6AZXcagjE4robuStPG93VB9YHj5xBiLI0bt0/edit

Copied from original issue: kubernetes-retired/kube-deploy#530

machine-controller: update Machine.Status when appropriate

From @pipejakob on November 7, 2017 20:8

So far, it doesn't update the Machine's Status at all.

Copied from original issue: kubernetes-retired/kube-deploy#349

Cluster Bootstrapping: document best practices (incl. supported scenarios)

From @rsdcastro on January 29, 2018 21:52

Copied from original issue: kubernetes-retired/kube-deploy#526

Remove the apiserver-boot dependency for generate certificate.

From @medinatiger on February 9, 2018 18:9

We should remove the need to run apiserver-boot to generate certificate for running ext api server. The logs of apiserver-boot shows the required openssl commands to do the job.

Copied from original issue: kubernetes-retired/kube-deploy#591

E2E tests for cluster bootstrapping

From @rsdcastro on January 29, 2018 21:49

Basic set of E2E tests for cluster bootstrapping across supported providers.

Copied from original issue: kubernetes-retired/kube-deploy#525

The clife_ prefix to the repo name is no longer required

Feel free to rename the repo.

Support multiple providers for cluster bootstrapping

From @rsdcastro on January 29, 2018 21:45

Currently only GCP is supported. This item is about - more easily - supporting other providers when bootstrapping a cluster.

Copied from original issue: kubernetes-retired/kube-deploy#524

Document how to contribute to Cluster API effort

From @rsdcastro on January 29, 2018 23:39

Copied from original issue: kubernetes-retired/kube-deploy#545

Add machine class to the Cluster API

From @rsdcastro on January 29, 2018 22:0

This issue tracks discussion on adding machine class (as used/suggested in several API discussions) and any work required to make that happen if we decide to do it.

Copied from original issue: kubernetes-retired/kube-deploy#529

Define machines deletion policies.

From @p0lyn0mial on February 28, 2018 16:57

The current implementation of MachineSet chooses the first machine from the list to delete. As this doesn't provide any guarantees of consistency regarding the order of the items. It is random. We would like to identify some strategies that would provide better approaches for deletion.

Copied from original issue: kubernetes-retired/kube-deploy#625

MachineSet controller sets MachineSetStatus

From @p0lyn0mial on March 8, 2018 14:21

The current implementation of MachineSet controller doesn't set MachineSetStatus at all. This item tracks potential implementation.

For reference implementations see:

https://github.com/kubernetes/kubernetes/blob/a0844c17bfb964e3d1709784203e8404f790116c/pkg/controller/replicaset/replica_set_utils.go#L35:1

https://github.com/gardener/machine-controller-manager/blob/master/pkg/controller/machineset.go#L518

See also #535

Copied from original issue: kubernetes-retired/kube-deploy#642

Can we set up CI/CD and testing for the new repository

Before we start transition code here, can we set up the CI/CD boiler plate for the new repository?

Tool to recover if the master is bricked

From @rsdcastro on January 29, 2018 22:45

In case metadata can't be accessed/reconstructed, this tool would allow the cluster to be restored with the cluster API components.

Related to kubernetes-retired/kube-deploy#532

Copied from original issue: kubernetes-retired/kube-deploy#538

Define validation strategy for provider config

From @rsdcastro on February 22, 2018 21:13

As per comment in #405, "validation is another concern and independent of specifying the cloud provider in the top level object or in the provider specific config. We can't bake provider specific validation logic into the API. That needs to be pluggable somehow through a webhook, admission controller, or some other mechanism. But it's a much larger discussion."

This issue tracks methods we'll allow for validation, including recommendations and examples.

cc @karan

Copied from original issue: kubernetes-retired/kube-deploy#623

Have upgrade functionality (as tool or server side)

From @rsdcastro on January 29, 2018 22:39

Copied from original issue: kubernetes-retired/kube-deploy#537

Support for automated cluster tear down via Cluster API

From @rsdcastro on January 29, 2018 21:57

Copied from original issue: kubernetes-retired/kube-deploy#528

Document plan on how API can support auto-scaling and other auto* features

From @rsdcastro on April 5, 2018 15:24

This issue tracks an evaluation of the API to make sure that it can support the use cases for auto-scaling and any other auto* features.

This issue should cover any API changes as well. Specific implementation changes should be tracked as separate issues.

cc @krousey @roberthbailey @mwielgus

Copied from original issue: kubernetes-retired/kube-deploy#666

Support adopting a cluster from a set of existing machines

From @rsdcastro on January 29, 2018 21:38

This item tracks supporting users providing a list of machines that will be used to bootstrap the cluster rather than requiring machines to be added via the cluster API.

Copied from original issue: kubernetes-retired/kube-deploy#522

Use the cluster name in cluster.yaml in admin.conf

From @karan on March 6, 2018 17:29

Currently the cluster config is created with cluster name as kubernetes. There is a cluster name in the cluster.yaml metadata.name. We should start using that name for the cluster so tools can be made to target any cluster in kubeconfig file.

Copied from original issue: kubernetes-retired/kube-deploy#636

Run more automated checks on cluster-api repo

From @krousey on February 15, 2018 19:50

We should run the following checks on PRs and in CI:

• go vet
• gofmt
• verify vendor depencies are correct (this is partially accomplished by build today, but it'd be nice to have better messaging around this)
• code doesn't need to be regenerated (deepcopy, conversion, clients, ...)
• BUILD files don't have to be regenerated

Copied from original issue: kubernetes-retired/kube-deploy#613

Support for hooks for resource cleanup during machine deletion

From @rsdcastro on January 29, 2018 22:9

Copied from original issue: kubernetes-retired/kube-deploy#531

Import path for the new repository

Traditionally in kubernetes repositories we use the k8s.io domain to redirect to the codes current hosted location.

For instance with github we would have the following redirect in place:

k8s.io/kube-deploy --> github.com/kubernetes/kube-deploy

Do we want to use the following redirect here?

Personally I think we should keep the import paths as literal as possible. The k8s.io redirect can be confusing to a newcomer. Furthermore there is still a bit of magic linking the two together, and if you don't know about it it could be hard to trace.

If we do switch code hosting providers later, we can use tools like sed and grep to update our codebases as needed.

Allow other CNI providers to be installed

From @rsdcastro on March 3, 2018 20:22

Currently CNI provider isn't specified in the API, and the GCP implementation has support for weavenet. This issue tracks API updates to support other CNI providers.

https://github.com/kubernetes/kube-deploy/blob/master/cluster-api/api/cluster/v1alpha1/types.go#L327

Copied from original issue: kubernetes-retired/kube-deploy#635

Support multi-master clusters

From @rsdcastro on January 29, 2018 22:56

Copied from original issue: kubernetes-retired/kube-deploy#539

cluster-installer - machine delete should wait for finalizers to finish before returning success

From @ajitak on November 14, 2017 21:45

When a machine CRD is marked for deletion, finalizers should delete node vm. Cluster installer should wait for finalizers to finish deleting vm whenever delete machine code path is triggered.

Copied from original issue: kubernetes-retired/kube-deploy#390

Define health check strategy for MachineSet

From @rsdcastro on March 1, 2018 21:10

This is to track discussion and documentation on how health checking will be done for machines in a set.

Copied from original issue: kubernetes-retired/kube-deploy#632

Bot for stale vendored cluster API

From @jessicaochen on January 3, 2018 19:20

Create a robot that will warn when the vendored cluster API is falling behind master.

Context:
On creation of cluster-api-gcp, there was a discussion on whether we were going to vendor the cluster-api or directly reference it. We decided to vendor it so it would be a better example of the right pattern to folks attempting to make their own machine controller. The downside to this is that the vendored cluster-api could fall behind the latest cluster-api. A robot warning about falling behind was the suggested mitigation.

Copied from original issue: kubernetes-retired/kube-deploy#471

Change the type of instance-status

From @karan on January 26, 2018 18:20

Writing the current state of the underlying VM as a machine object within another machine is confusing - https://github.com/kubernetes/kube-deploy/blob/master/cluster-api-gcp/cloud/google/instancestatus.go

Before #482 is closed, we could change the type of instance status to a type that only has the fields we care about. This would make the code a bit more readable, requiresUpdate in machine actuators a bit more explicit, and we wouldn't have to deal with recursive machines.

Copied from original issue: kubernetes-retired/kube-deploy#518

Support changing machine role (e.g. node => master)

From @rsdcastro on January 29, 2018 22:21

Copied from original issue: kubernetes-retired/kube-deploy#533

Decide how fault/availability zones should be represented in Cluster API

From @rsdcastro on February 28, 2018 20:52

This issue tracks how fault zones (or availability zones) should be represented in the Cluster API. Should it be a top-level field? Should we be able to specify provider specific config for them (separately from the current provider config)? This is both related to master and nodes that might have some affinity to resources specific to that zone.

@krousey @maisem @justinsb

Copied from original issue: kubernetes-retired/kube-deploy#627

Support upgrading/pivoting cluster to multi-master

From @rsdcastro on January 29, 2018 22:57

Copied from original issue: kubernetes-retired/kube-deploy#540

Establish a stronger link between Machines, MachineSets, and MachineDeployments with their Cluster.

From @dgoodwin on February 15, 2018 17:12

On Feb 14 Cluster API working group call, a couple parties indicated we would like to make use of the ability to have multiple clusters to exist in one namespace, which would free up namespaces to be used for other levels of organization. (one customer, one cloud provider account, etc) In this case we would not be able to assume all machines or machine sets in a namespace belong to the single cluster in the namespace, and as such a stronger link between the two could be useful for some adopters of the cluster API.

Copied from original issue: kubernetes-retired/kube-deploy#609

Have a single CONTRIBUTING.md for kube-deploy and move running instructions to its own file

From @rsdcastro on February 6, 2018 23:13

This is about having running instructions as a separate doc and have both README.md and CONTRIBUTING.md pointing to it, rather than having README.md linking to the more complex CONTRIBUTING.md doc.

Copied from original issue: kubernetes-retired/kube-deploy#576

create integration test for machine controller.

From @medinatiger on January 8, 2018 19:46

We need to have integration test to test machine controller. The following scenario is interested.

1. Basic create/update/delete through API
2. Scale up
3. Scale down
4. Stress test with combination of ScaleUp/ScaleDown
5. Upgrade.
6. Mixed upgrade and scale operation simultaneously

Copied from original issue: kubernetes-retired/kube-deploy#476

Reconcile Cluster API Cloud Provider with the effort of moving Cloud Providers out of core K8s

From @rsdcastro on February 14, 2018 18:35

This issue is to track the discussion and whether or not those should be combined in some way.

cc @kris-nova @karan @roberthbailey

Copied from original issue: kubernetes-retired/kube-deploy#604

Allow CA and key to be provided on cluster creation

From @rsdcastro on April 2, 2018 17:0

Currently kubeadm init generates a self-signed certificate authority and key, which isn't a solution that suits all users.

This issue tracks allowing CA and key to be provided by the user.

Copied from original issue: kubernetes-retired/kube-deploy#661