The CSI Packet plugin allows the creation and mounting of packet storage volumes as persistent volume claims in a kubernetes cluster.
Read how to deploy the Kubernetes CSI plugin for Packet here!
The basic refernce for Kubernetes CSI is found at https://kubernetes-csi.github.io/docs/
A typical sequence of the tasks performed by the Controller and Node components is
- Create Controller.CreateVolume
- Attach Controller.ControllerPublish
- Mount, Format Node.NodeStageVolume (called once only per volume)
- Bind Mount Node.NodePublishVolume
- Bind Unmount Node.NodeUnpublishVolume
- Unmount Node.NodeUnstageVolume
- Mount, Format Node.NodeStageVolume (called once only per volume)
- Detach Controller.ControllerUnpublish
- Attach Controller.ControllerPublish
- Destroy Controller.DeleteVolume
The plugin node component require particular configuration of the packet host with regard to the services that are running.
It relies on iscsid being configured correctly with the initiator name, and up multipathd running with a configuration that includes user_friendly_names yes
This setup is not perfomed by the plugin.
The files found in deploy/kubernetes/
define the deployment process, which follows the approach diagrammed in the design proposal
Packet credentials are used by the controller to manage volumes. They are configured with a json-formatted secret which contains
- an authetication token
- a project id
- a facility id
The cluster is assumed to reside wholly in one facility, so the controller includes that facility id in all of the volume-related api calls.
The file deploy/kubernetes/setup.yaml contains the serviceaccount, role and rolebinding definitions used by the various components.
The controller is deployed as a StatefulSet to ensure that there is a single instance of the pod. The node is deployed as a daemonset, which will install an instance of the pod on every un-tainted node. In most cluster deployments, the master node will have a node-role.kubernetes.io/master:NoSchedule
taint and so the csi-packet plugin will not operate there.
The CSI plugin framework is designed to be agnostic with respect to the Container Orchestrator system, and so there is no direct communication between the CSI plugin and the kubernetes api. Instead, the CSI project provides a number of sidecar containers which mediate beween the plugin and kubernetes.
The controller deployment uses
- external-attacher https://github.com/kubernetes-csi/external-attacher
- external-provisioner https://github.com/kubernetes-csi/external-provisioner
which communicate with the kubernetes api within the cluster, and communicate with the csi-packet plugin through a unix domain socket shared in the pod.
The node deployment uses
- driver-registrar https://github.com/kubernetes-csi/driver-registrar
which advertises the csi-packet driver to the cluster. There is also by unix domain socket communciation channel, but in this case it is a host-mounted directory in order to permit the kubelet process to interact with the plugin.
TODO: incorporate the liveness prob
- liveness-probe https://github.com/kubernetes-csi/livenessprobe
The node processes must interact with services running on the host in order to connect, mount and format the packet volumes. These interactions require a particular pod configuration. The driver invokes the iscsiadm and multipath client processes and they must communicate with the iscisd and multipathd systemd services. In consequence, the pod
- uses hostNetwork: true
- is privileged
- mounts /etc/
- mounts /dev
- mounts /var/lib/iscsi
- mounts /sys/devices
- mounts /run/udev/
- mounts /var/lib/kubelet
- mounts /csi
See additional documents in the docs/ directory
The plugin at present is far from production assurance, and requires testing and hardening. Even listing the known issues is premature.
The Makefile builds locally using your own installed go
, or in a container via docker run
.
The following are standard commands:
make build ARCH=$(ARCH)
- ensure vendor dependencies and build the single CSI binary asdist/bin/packet-cloud-storage-interface-$(ARCH)
make build-all
- ensure vendor dependencies and build the single CSI binary for all supported architectures. We build a binary for each arch for which aDockerfile.<arch>
exists in this directory.make build ARCH=$(ARCH) DOCKERBUILD=true
- ensure vendor dependencies and build the CSI binary as above, while performing the build inside a docker containermake build-all
- ensure vendor dependencies and build the CSI binary for all supported architectures.make image ARCH=$(ARCH)
- make an OCI image for the provided ARCHmake image-all
- make an OCI image for all supported architecturesmake ci
- build, test and create an OCI image for all supported architectures. This is what the CI system runs.make cd
- deploy images for all supported architectures, as well as a multi-arch manifest..make release
- deploy tagged release images for all supported architectures, as well as a multi-arch manifest..
All images are tagged $(TAG)-$(ARCH)
, where:
TAG
= the image tag, which always includes the current branch when merged intomaster
, and the short git hash. For git tags that are applied in master viamake release
, it also is the git tag. Thus a normal merge releases two image tags - git hash andmaster
- while adding a git tag to release formally creates a third one.ARCH
= the architecture of the image
In addition, we use multi-arch manifests for "archless" releases, e.g. :3456abcd
or :v1.2.3
or :master
.