apoorvajagtap/kcp

kcp is a prototype of a multi-tenant Kubernetes control plane for workloads on many clusters

kcp

Overview

kcp is a Kubernetes-like control plane focusing on:

• A control plane for many independent, isolated “clusters” known as workspaces
• Enabling API service providers to offer APIs centrally using multi-tenant operators
• Easy API consumption for users in their workspaces
• Flexible scheduling of workloads to physical clusters
• Transparent movement of workloads among compatible physical clusters
• Advanced deployment strategies for scenarios such as affinity/anti-affinity, geographic replication, cross-cloud replication, etc.

kcp can be a building block for SaaS service providers who need a massively multi-tenant platform to offer services to a large number of fully isolated tenants using Kubernetes-native APIs. The goal is to be useful to cloud providers as well as enterprise IT departments offering APIs within their company.

Quickstart

Prerequisites

• kubectl
• A Kubernetes cluster (for local testing, consider kind)

Download kcp

Visit our latest release page and download kcp and kubectl-kcp-plugin that match your operating system and architecture.

Extract kcp and kubectl-kcp-plugin and place all the files in the bin directories somewhere in your $PATH.

Start kcp

You can start kcp using this command:

kcp start

This launches kcp in the foreground. You can press ctrl-c to stop it.

To see a complete list of server options, run kcp start options.

Set your KUBECONFIG

During its startup, kcp generates a kubeconfig in .kcp/admin.kubeconfig. Use this to connect to kcp and display the version to confirm it's working:

$ export KUBECONFIG=.kcp/admin.kubeconfig
$ kubectl version
WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short.  Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.4", GitCommit:"95ee5ab382d64cfe6c28967f36b53970b8374491", GitTreeState:"clean", BuildDate:"2022-08-17T18:46:11Z", GoVersion:"go1.19", Compiler:"gc", Platform:"darwin/amd64"}
Kustomize Version: v4.5.4
Server Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.3+kcp-v0.8.0", GitCommit:"41863897", GitTreeState:"clean", BuildDate:"2022-09-02T18:10:37Z", GoVersion:"go1.18.5", Compiler:"gc", Platform:"darwin/amd64"}

Configure kcp to sync to your cluster

kcp can't run pods by itself - it needs at least one physical cluster for that. For this example, we'll be using a local kind cluster. It does not have to exist yet.

In this recipe we use the root workspace to hold the description of the workload and where it goes. These usually would go elsewhere, but we use the root workspace here for simplicity.

Run the following command to tell kcp about the kind cluster (replace the syncer image tag as needed; CI now puts built images in https://github.com/orgs/kcp-dev/packages):

$ kubectl kcp workload sync kind --syncer-image ghcr.io/kcp-dev/kcp/syncer:v0.10.0 -o syncer-kind-main.yaml
Creating synctarget "kind"
Creating service account "kcp-syncer-kind-25coemaz"
Creating cluster role "kcp-syncer-kind-25coemaz" to give service account "kcp-syncer-kind-25coemaz"

 1. write and sync access to the synctarget "kcp-syncer-kind-25coemaz"
 2. write access to apiresourceimports.

Creating or updating cluster role binding "kcp-syncer-kind-25coemaz" to bind service account "kcp-syncer-kind-25coemaz" to cluster role "kcp-syncer-kind-25coemaz".

Wrote physical cluster manifest to syncer-kind-main.yaml for namespace "kcp-syncer-kind-25coemaz". Use

  KUBECONFIG=<pcluster-config> kubectl apply -f "syncer-kind-main.yaml"

to apply it. Use

  KUBECONFIG=<pcluster-config> kubectl get deployment -n "kcp-syncer-kind-25coemaz" kcp-syncer-kind-25coemaz

to verify the syncer pod is running.

Next, we need to install the syncer pod on our kind cluster - this is what actually syncs content from kcp to the physical cluster. The kind cluster needs to be running by now. Run the following command:

$ KUBECONFIG=</path/to/kind/kubeconfig> kubectl apply -f "syncer-kind-main.yaml"
namespace/kcp-syncer-kind-25coemaz created
serviceaccount/kcp-syncer-kind-25coemaz created
secret/kcp-syncer-kind-25coemaz-token created
clusterrole.rbac.authorization.k8s.io/kcp-syncer-kind-25coemaz created
clusterrolebinding.rbac.authorization.k8s.io/kcp-syncer-kind-25coemaz created
secret/kcp-syncer-kind-25coemaz created
deployment.apps/kcp-syncer-kind-25coemaz created

Bind to workload APIs and create default placement

If you are running kcp version v0.10.0 or higher, you will need to run the following commmand (continuing in the root workspace) to create a binding to the workload APIs export and a default placement for your physical cluster:

$ kubectl kcp bind compute root
Binding APIExport "root:compute:kubernetes".
placement placement-1pfxsevk created.
Placement "placement-1pfxsevk" is ready.

Create a deployment in kcp

Let's create a deployment in our kcp workspace and see it get synced to our cluster:

$ kubectl create deployment --image=gcr.io/kuar-demo/kuard-amd64:blue --port=8080 kuard
deployment.apps/kuard created

Once your cluster has pulled the image and started the pod, you should be able to verify the deployment is running in kcp:

$ kubectl get deployments
NAME    READY   UP-TO-DATE   AVAILABLE   AGE
kuard   1/1     1            1           3s

We are still working on adding support for kubectl logs, kubectl exec, and kubectl port-forward to kcp. For the time being, you can check directly in your cluster.

kcp translates the names of namespaces in workspaces to unique names in a physical cluster. We first must get this translated name; if you're following along, your translated name might be different.

$ KUBECONFIG=</path/to/kind/kubeconfig> kubectl get pods --all-namespaces --selector app=kuard
NAMESPACE          NAME                     READY   STATUS    RESTARTS   AGE
kcp-26zq2mc2yajx   kuard-7d49c786c5-wfpcc   1/1     Running   0          4m28s

Now we can e.g. check the pod logs:

$ KUBECONFIG=</path/to/kind/kubeconfig> kubectl --namespace kcp-26zq2mc2yajx logs deployment/kuard | head
2022/09/07 14:04:35 Starting kuard version: v0.10.0-blue
2022/09/07 14:04:35 **********************************************************************
2022/09/07 14:04:35 * WARNING: This server may expose sensitive
2022/09/07 14:04:35 * and secret information. Be careful.
2022/09/07 14:04:35 **********************************************************************
2022/09/07 14:04:35 Config:
{
  "address": ":8080",
  "debug": false,
  "debug-sitedata-dir": "./sitedata",

Next steps

Thanks for checking out our quickstart!

If you're interested in learning more about all the features kcp has to offer, please check out our additional documentation:

• Concepts - a high level overview of kcp concepts
• Workspaces - a more thorough introduction on kcp's workspaces
• Locations & scheduling - details on kcp's primitives that abstract over clusters
• Syncer - information on running the kcp agent that syncs content between kcp and a physical cluster
• kubectl plugin
• Authorization - how kcp manages access control to workspaces and content
• Virtual workspaces - details on kcp's mechanism for virtual views of workspace content

Contributing

We ❤️ our contributors! If you're interested in helping us out, please head over to our Contributing and Developer guides.

Getting in touch

There are several ways to communicate with us:

• The #kcp-dev channel in the Kubernetes Slack workspace
• Our mailing lists:
  • kcp-dev for development discussions
  • kcp-users for discussions among users and potential users
• Subscribe to the community calendar for community meetings and events
  • The kcp-dev mailing list is subscribed to this calendar
• See recordings of past community meetings on YouTube
• See upcoming and past community meeting agendas and notes
• Browse the shared Google Drive to share design docs, notes, etc.
  • Members of the kcp-dev mailing list can view this drive

Additional references

• KubeCon EU 2021: Kubernetes as the Hybrid Cloud Control Plane Keynote - Clayton Coleman (video)
• OpenShift Commons: Kubernetes as the Control Plane for the Hybrid Cloud - Clayton Coleman (video)
• TGI Kubernetes 157: Exploring kcp: apiserver without Kubernetes
• K8s SIG Architecture meeting discussing kcp - June 29, 2021
• Let's Learn kcp - A minimal Kubernetes API server with Saiyam Pathak - July 7, 2021