virtual-kubelet/cri

Virtual Kubelet CRI Provider

This is a Virtual Kubelet Provider implementation that manages real pods and containers in a CRI-based container runtime.

Purpose

The purpose of the CRI Provider is for testing and prototyping ONLY. It is not to be used for any other purpose!

The whole point of the Virtual Kubelet project is to provide an interface for container runtimes that don't conform to the standard node-based model. The Kubelet codebase is the comprehensive standard CRI node agent and this Provider is not attempting to recreate that.

This Provider implementation should be seen as a bare-bones minimum implementation for making it easier to test the core of the Virtual Kubelet project against real pods and containers - in other words, more comprehensive than MockProvider.

This Provider implementation is also designed such that it can be used for prototyping new architectural features which can be developed against local Linux infrastructure. If the CRI provider can be shown to work successfully within a Linux guest, there can be a much higher degree of confidence that the abstraction should work for other Providers.

Dependencies

The simplest way to run the CRI provider is to install containerd 1.1, which already has the CRI plugin installed.

Configuring

• Copy /etc/kubernetes/admin.conf from your master node and place it somewhere local to Virtual Kubelet
• Find a client.crt and client.key that will allow you to authenticate with the API server and copy them somewhere local

Running

Start containerd

sudo nohup containerd > /tmp/containerd.out 2>&1 &

Create a script that will set up the environment and run Virtual Kubelet with the correct provider

#!/bin/bash
export VKUBELET_POD_IP=<IP of the Linux node>
export APISERVER_CERT_LOCATION="/etc/virtual-kubelet/client.crt"
export APISERVER_KEY_LOCATION="/etc/virtual-kubelet/client.key"
export KUBELET_PORT="10250"
cd bin
./virtual-kubelet --provider cri --kubeconfig admin.conf

The Provider assumes that the containerd socket is available at /run/containerd/containerd.sock which is the default location. It will write container logs at /var/log/vk-cri/ and mount volumes at /run/vk-cri/volumes/. You need to make sure that you run as a user that has permissions to read and write to these locations.

Limitations

• The CRI provider does everything that the Provider interface currently allows it to do, principally managing the lifecycle of pods, returning logs and very little else.
• It will create emptyDir, configmap and secret volumes as necessary, but won't update configmaps or secrets if they change as this has yet to be implemented in the base
• It does not support any kind of persistent volumes
• It will try to run kube-proxy when it starts and can successfully do that. However, as we transition VK to a model in which it treats services and routing in the abstract, this capability will be refactored as a means of testing that feature.
• Networking should currently be considered non-functional

cri's Issues

example

is there any example??

I'm running the busybox example from teh root repo, removing the tolerations for azure...and get provider error:

time="2019-11-07T15:04:02+01:00" level=info msg="receive CreatePod \"busyecho2\""
time="2019-11-07T15:04:02+01:00" level=info msg="Updated k8s pod status" key=default/busyecho2 method=createOrUpdatePod name=busyecho2 namespace=default node=virtual-kubelet operatingSystem=Linux phase=Pending pod=busyecho2 podPhase=Pending provider=cri reason=ProviderFailed uid=52edc4f3-0167-11ea-a9a4-00505681174c watchedNamespace= workerId=10
time="2019-11-07T15:04:02+01:00" level=warning msg="requeuing [[\"default/busyecho2\"]] due to failed sync" error="failed to sync pod \"default/busyecho2\" in the provider: rpc error: code = Unknown desc = failed to setup network for sandbox \"b5b746ecf1193f3cab8e144ece3df574e5de815fba5a14da44007f25cae58ccd\": cni plugin not initialized" key=default/busyecho2 method=handleQueueItem node=virtual-kubelet operatingSystem=Linux provider=cri watchedNamespace= workerId=10```

listing the pods:

```root@uubelet20:~# kubectl  get pods
NAME                       READY   STATUS           RESTARTS   AGE
busyecho2                  0/1     ProviderFailed   0          1m
busyecho2xy                0/1     Pending          0          5m
docker-registry-1-fqsxw    1/1     Running          0          21d
registry-console-1-2xhzt   1/1     Running          0          21d
router-1-ck75m             1/1     Running          0          21d
router-1-x27sm             1/1     Running          1          21d```

any idea?

dont see cri provider

when I run
bin/virtual-kubelet providers
just get

mock

and when I run bin/virtual-kubelet --provider cri --log-level debug I get 'cri not found'

Error: provider "cri" not found
Usage:
virtual-kubelet [flags]
virtual-kubelet [command]

Available Commands:
help Help about any command
providers Show the list of supported providers
version Show the version of the program

Flags:
--cluster-domain string kubernetes cluster-domain (default is 'cluster.local') (default "cluster.local")
--disable-taint disable the virtual-kubelet node taint
--enable-node-lease use node leases (1.13) for node heartbeats
--full-resync-period duration how often to perform a full resync of pods between kubernetes and the provider (default 1m0s)
-h, --help help for virtual-kubelet
--klog.alsologtostderr log to standard error as well as files
--klog.log_backtrace_at traceLocation when logging hits line file:N, emit a stack trace (default :0)
--klog.log_dir string If non-empty, write log files in this directory
--klog.log_file string If non-empty, use this log file
--klog.log_file_max_size uint Defines the maximum size a log file can grow to. Unit is megabytes. If the value is 0, the maximum file size is unlimited. (default 1800)
--klog.logtostderr log to standard error instead of files (default true)
--klog.skip_headers If true, avoid header prefixes in the log messages
--klog.skip_log_headers If true, avoid headers when opening log files
--klog.stderrthreshold severity logs at or above this threshold go to stderr (default 2)
--klog.v Level number for the log level verbosity
--klog.vmodule moduleSpec comma-separated list of pattern=N settings for file-filtered logging
--kubeconfig string kube config file to use for connecting to the Kubernetes API server (default "/root/.kube/config")
--log-level string set the log level, e.g. "debug", "info", "warn", "error" (default "info")
--metrics-addr string address to listen for metrics/stats requests (default ":10255")
--namespace string kubernetes namespace (default is 'all')
--nodename string kubernetes node name (default "virtual-kubelet")
--os string Operating System (Linux/Windows) (default "Linux")
--pod-sync-workers int set the number of pod synchronization workers (default 10)
--provider string cloud provider
--provider-config string cloud provider configuration file
--startup-timeout duration How long to wait for the virtual-kubelet to start
--trace-exporter strings sets the tracing exporter to use, available exporters: [jaeger ocagent]
--trace-sample-rate string set probability of tracing samples
--trace-service-name string sets the name of the service used to register with the trace exporter (default "virtual-kubelet")
--trace-tag map add tags to include with traces in key=value form

Use "virtual-kubelet [command] --help" for more information about a command.

FATA[0000] provider "cri" not found