dri_device_t cannot be accessed correctly by pods using device plugins.

[mregmi]

dri_device_t cannot be accessed correctly by pods using device plugins.

[mregmi]

Tried #268 but still having issues. It is now complaining about Open.

type=AVC msg=audit(1694800707.873:1495): avc: denied { open } for pid=1073469 comm="clinfo" path="/dev/dri/by-path/pci-0000:37:00.0-render" dev="devtmpfs" ino=124588 scontext=system_u:system_r:container_t:s0:c17,c28 tcontext=system_u:object_r:dri_device_t:s0 tclass=chr_file permissive=0

[rhatdan]

Yup, that is what I thought would be an issue.

The question I have is should this be a Boolean or be allowed by default. The issue is opening the risk for all containers to be able to use dri devices. I think it is best to only allow access via a Boolean.

My previous attempt was thinking that perhaps some other process had opened the device and allow the containers to read/write them. Allowing containers to open the device along with read/write basically removes the ability to control containers from accessing DRI devices, if they escaped containerization.

[mregmi]

So i am still confused that this issue is not seen in our other device categories where its labelled device_t and is very specific to dri_device_t. The other devices work exactly the same way and just the label is different.

[mregmi]

@rhatdan So we have 2 other devices which is labelled as follows:
sh-5.1# ls -lZ /dev/sgx_provision
crw-------. 1 root root system_u:object_r:device_t:s0 10, 126 Oct 26 15:06 /dev/sgx_provision

sh-5.1# ls -lZ /dev/vfio
total 0
crw-rw----. 1 root hugetlbfs system_u:object_r:vfio_device_t:s0 235, 0 Oct 26 15:10 436

The workload containers have no issue accessing these files. But the problem is only while accessing /dev/dri/* which is labelled as dri_device_t.

Do they have different set of policies?
The boolean method works but the customers needs to set these in all the nodes in cluster. I am just trying to make sure i am not missing anything as the other two devices does not have access issues.

[rhatdan]

Are you using Podman on CRIO Within the Cluster (I am assuming cluster ==Kubernetes/OpenShift)?

[mregmi]

Yes we are using OpenShift cluster which uses CRIO.

[rhatdan]

And then executing rootless podman within it?

[mregmi]

Yes the pod is rootless but we do not directly use podman. We just use oc apply to run the pod.

The device node is made available by intel device plugin operator to the workload pod. I think it uses the 'device' field of OCI runtime.
https://github.com/opencontainers/runtime-spec/blob/main/config-linux.md#devices

[mregmi]

We experimented a bit and re-labelled the device file in question to device_t. but still got the issue. So there might be something else going on when Plugin is requesting the resource. We will look into plugin code to verify.

[mregmi]

We have narrowed down the issue to a symbolic link. When the device is accessed directly (/dev/dri/card0) there is no issue but if the access is done via symbolic link (/dev/dri/by-path/card0), the access is denied. After looking at the device files made available to the pod. it looks like its not container_file_t but dri_device_t

Not sure why the labels in by-path are not translated.
Here is how the devices show up as in host and the workload container.

Host Node:

sh-5.1# ls -lZ /dev/dri
total 0
drwxr-xr-x. 2 root root   system_u:object_r:device_t:s0          140 Oct 31 04:50 by-path
crw-rw----. 1 root video  system_u:object_r:dri_device_t:s0 226,   0 Oct 31 03:51 card0
crw-rw----. 1 root video  system_u:object_r:dri_device_t:s0 226,   1 Oct 31 04:50 card1
crw-rw----. 1 root video  system_u:object_r:dri_device_t:s0 226,   2 Oct 31 04:50 card2
crw-rw-rw-. 1 root render system_u:object_r:dri_device_t:s0 226, 128 Oct 31 04:50 renderD128
crw-rw-rw-. 1 root render system_u:object_r:dri_device_t:s0 226, 129 Oct 31 04:50 renderD129

sh-5.1# ls -lZ /dev/dri/by-path/
total 0
lrwxrwxrwx. 1 root root system_u:object_r:device_t:s0  8 Oct 31 03:51 pci-0000:02:00.0-card -> ../card0
lrwxrwxrwx. 1 root root system_u:object_r:device_t:s0  8 Oct 31 04:50 pci-0000:37:00.0-card -> ../card1
lrwxrwxrwx. 1 root root system_u:object_r:device_t:s0 13 Oct 31 04:50 pci-0000:37:00.0-render -> ../renderD128
lrwxrwxrwx. 1 root root system_u:object_r:device_t:s0  8 Oct 31 04:50 pci-0000:3c:00.0-card -> ../card2
lrwxrwxrwx. 1 root root system_u:object_r:device_t:s0 13 Oct 31 04:50 pci-0000:3c:00.0-render -> ../renderD129

Workload Pod:

sh-4.4$ ls -laZ /dev/dri   
total 0
drwxr-xr-x. 3 root root  system_u:object_r:container_file_t:s0:c27,c28      140 Nov  8 22:42 .
drwxr-xr-x. 6 root root  system_u:object_r:container_file_t:s0:c27,c28      380 Nov  8 22:42 ..
drwxr-xr-x. 2 root root  system_u:object_r:container_file_t:s0:c27,c28      120 Nov  8 22:42 by-path
crw-rw-rw-. 1 root video system_u:object_r:container_file_t:s0:c27,c28 226,   1 Nov  8 22:42 card1
crw-rw-rw-. 1 root video system_u:object_r:container_file_t:s0:c27,c28 226,   2 Nov  8 22:42 card2
crw-rw-rw-. 1 root   797 system_u:object_r:container_file_t:s0:c27,c28 226, 128 Nov  8 22:42 renderD128
crw-rw-rw-. 1 root   797 system_u:object_r:container_file_t:s0:c27,c28 226, 129 Nov  8 22:42 renderD129
sh-4.4$ ls -laZ /dev/dri/by-path/
total 0
drwxr-xr-x. 2 root root  system_u:object_r:container_file_t:s0:c27,c28      120 Nov  8 22:42 .
drwxr-xr-x. 3 root root  system_u:object_r:container_file_t:s0:c27,c28      140 Nov  8 22:42 ..
crw-rw----. 1 root video system_u:object_r:dri_device_t:s0             226,   1 Oct 31 04:50 pci-0000:37:00.0-card
crw-rw-rw-. 1 root   797 system_u:object_r:dri_device_t:s0             226, 128 Oct 31 04:50 pci-0000:37:00.0-render
crw-rw----. 1 root video system_u:object_r:dri_device_t:s0             226,   2 Oct 31 04:50 pci-0000:3c:00.0-card
crw-rw-rw-. 1 root   797 system_u:object_r:dri_device_t:s0             226, 129 Oct 31 04:50 pci-0000:3c:00.0-render

@mythi

[mythi]

Yes the pod is rootless but we do not directly use podman.

I believe there's a conflict here. The pod containers are only run as a non-root user (non-zero uid/gid), but the runtime stack is not rootless

[mythi]

After looking at the device files made available to the pod. it looks like its not container_file_t but dri_device_t

FWIW, the GPU plugin sets the symlinks via mounts[] as specified by the device plugin API.