kvaps/kube-iscsi-loop

Loop iSCSI interfaces and daemons for K8S.

kube-iscsi-loop

Loop iSCSI interfaces and daemons for K8S.

Components

Backend

The next backends supported

• SharedFS - images are simple files thats stores on some shared filesystem.

• Sheepdog - sheepdog images. Your nodes should be part of sheepdog cluster.
  However you can use containerized sheepdog.

Frontend

Project uses built-in kubernetes iSCSI driver for connect drivers into pods.

Limitations and Requirements

This implementation implies using locally runned iSCSI daemons with some distributed shared filesystem as backend, this filesystem should be directly connected to all nodes.

It's not difficult to update those scripts for use with remote runned iSCSI-daemon (take a look on sharedfs provisioners): you can add some authentication parameters, multipathing eg., and bind targets to 0.0.0.0 instead 127.0.0.1.
If you do this please send me your PR.

Otherwise you can simple use iSCSI-targetd provisioner

Images

iSCSI Daemons

Image	Build Status
iscsi

This image includes tgtd and iscsid daemons, both of them are necessary for allow publishers announce images, and connect them to your pods.

Runs as daemonsets, but can be replaced by locally runned daemons.

iSCSI Dynamic provisioners

Image	Build Status
iscsi-sharedfs-provisioner
iscsi-sheepdog-provisioner

Runs as deployment.

Waits until PersistentVolumeClaims will be created in Kubernetes.

Then creates new drives with needed parameters and announce them as PersistentVolume in Kubernetes.

iSCSI Publishers

Image	Build Status
iscsi-sharedfs-publisher
iscsi-sheepdog-publisher

Runs as daemonset.

Periodically checks some API for newly created images and creates iSCSI targets for local use only.

After that you can use built-in iSCSI driver in Kubernetes for connect your images into pods.

kube-iscsi-loop's Issues

no graceful shutdown of tgtd

When running tgtd -f, stopping the docker container causes a rude kill of the process instead of a graceful shutdown.

This is because tgtd does not listen to SIGTERM or SIGINT - so docker is sending a SIGKILL to the process.

See https://github.com/fujita/tgt/blob/master/scripts/tgtd.service for details

provisioning via `truncate` is bad

truncate -s always exits with code 0 - even if there is no space available.

root@k1:/stor# du -h 1G
1.1G    1G
root@k1:/stor# df -h /stor
Filesystem      Size  Used Avail Use% Mounted on
/dev/sde         20G   20G     0 100% /stor
root@k1:/stor# truncate -s 30G 30G
root@k1:/stor# du -h 30G
0       30G

iscisd not starting

root@ubuntu:/home/marc# docker run --net=host --privileged -v /lib/modules:/lib/modules -v /dev:/dev -v /run:/run kvaps/iscsi iscsid -f
iscsid: Can not bind IPC socket

Same problem when starting via kubectl:

k3.mbr.intern:/mnt/glusterfs/k8-orchestration/iscsi-provisioning$ kubectl logs iscsi-xqrxp iscsid
iscsid: Can not bind IPC socket

What's the problem here?