selinux policy doesn't allow a valid pod

[kubewarden]

selinux policy doesn't allow a valid pod

[chrisns]

just validated, yeah it's mutating. one of the many reasons PSPs are awfully unusable! wicked will update the migrator and then kubewarden should 🤞 be all ✅ down the list! 💥

[chrisns]

would appear it requires the whole pod to use the same selinux config, ignores it being set on a per-container basis

this pod is allowed

apiVersion: v1
kind: Pod
metadata:
    name: nginx-selinux-allowed
    labels:
        app: nginx-selinux
spec:
  containers:
  - name: nginx
    image: nginx
    securityContext:
      seLinuxOptions:
        level: s0:c123,c456
        role: object_r
        type: svirt_sandbox_file_t
        user: system_u
  securityContext:
    seLinuxOptions:
      level: s0:c123,c456
      role: object_r
      type: svirt_sandbox_file_t
      user: system_u

[jvanz]

Okay, I've spent some time comparing PSP and Kubewearden policies. The Kubernetes SELinux PSP mutates the request if the user does not set the seLinuxOptions configuration. For example, using the following PSP configuration:

kubectl apply -f - <<EOF
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: selinuxpsp
spec:
  seLinux:
    rule: MustRunAs
    seLinuxOptions:
      user: system_u
      role: object_r
      type: svirt_sandbox_file_t
      level: s0:c123,c456

  privileged: true
  allowPrivilegeEscalation: true
  allowedCapabilities:
  - '*'
  volumes:
  - '*'
  hostNetwork: true
  hostPorts:
  - min: 0
    max: 65535
  hostIPC: true
  hostPID: true
  runAsUser:
    rule: 'RunAsAny'
  supplementalGroups:
    rule: 'RunAsAny'
  fsGroup:
    rule: 'RunAsAny'
EOF

I can apply all this resources:

Pod with seLinuxOptions in the container level.

kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
    name: nginx-selinux-allowed
    labels:
        app: nginx-selinux
spec:
  containers:
  - name: nginx
    image: nginx
    securityContext:
      seLinuxOptions:
        level: s0:c123,c456
        role: object_r
        type: svirt_sandbox_file_t
        user: system_u
EOF
kubectl get pod nginx-selinux-allowed -o json | jq ".spec.securityContext,.spec.containers[0].securityContext"

{
  "seLinuxOptions": {
    "level": "s0:c123,c456",
    "role": "object_r",
    "type": "svirt_sandbox_file_t",
    "user": "system_u"
  }
}
{
  "seLinuxOptions": {
    "level": "s0:c123,c456",
    "role": "object_r",
    "type": "svirt_sandbox_file_t",
    "user": "system_u"
  }
}

Pod with seLinuxOptions in the container and pod level.

kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
    name: nginx-selinux-allowed2
    labels:
        app: nginx-selinux
spec:
  containers:
  - name: nginx
    image: nginx
    securityContext:
      seLinuxOptions:
        level: s0:c123,c456
        role: object_r
        type: svirt_sandbox_file_t
        user: system_u
  securityContext:
    seLinuxOptions:
      level: s0:c123,c456
      role: object_r
      type: svirt_sandbox_file_t
      user: system_u
EOF
kubectl get pod nginx-selinux-allowed2 -o json | jq ".spec.securityContext,.spec.containers[0].securityContext"

{
  "seLinuxOptions": {
    "level": "s0:c123,c456",
    "role": "object_r",
    "type": "svirt_sandbox_file_t",
    "user": "system_u"
  }
}
{
  "seLinuxOptions": {
    "level": "s0:c123,c456",
    "role": "object_r",
    "type": "svirt_sandbox_file_t",
    "user": "system_u"
  }
}

Pod with no seLinuxOptions configuration.

kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
    name: nginx-selinux-allowed3
    labels:
        app: nginx-selinux
spec:
  containers:
  - name: nginx
    image: nginx
EOF
kubectl get pod nginx-selinux-allowed3 -o json | jq ".spec.securityContext,.spec.containers[0].securityContext"

{
  "seLinuxOptions": {
    "level": "s0:c123,c456",
    "role": "object_r",
    "type": "svirt_sandbox_file_t",
    "user": "system_u"
  }
}
null

And it fails if the user provides a invalid configuration:

kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
    name: nginx-selinux-allowed4
    labels:
        app: nginx-selinux
spec:
  containers:
  - name: nginx
    image: nginx
    securityContext:
      seLinuxOptions:
        level: s0:c123,c456
        role: foo
        type: svirt_sandbox_file_t
        user: system_u
EOF
Error from server (Forbidden): error when creating "STDIN": pods "nginx-selinux-allowed4" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.containers[0].securityContext.seLinuxOptions.role: Invalid value: "foo": must be object_r]

Notice that PSP mutates some requests adding the seLinuxOptions configuration. The same happens with the Kuberwarden policy:

kubectl apply -f - <<EOF
apiVersion: policies.kubewarden.io/v1alpha2
kind: ClusterAdmissionPolicy
metadata:
  name: selinux
spec:
  policyServer: default
  module: registry://ghcr.io/kubewarden/policies/selinux-psp:v0.1.5
  mutating: true
  rules:
  - apiGroups: [""]
    apiVersions: ["v1"]
    resources: ["pods"]
    operations:
    - CREATE
    - UPDATE
  settings:
    rule: MustRunAs
    user: system_u
    role: object_r
    type: svirt_sandbox_file_t
    level: s0:c123,c456
EOF
clusteradmissionpolicy.policies.kubewarden.io/selinux created

kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
    name: nginx-selinux-allowed
    labels:
        app: nginx-selinux
spec:
  containers:
  - name: nginx
    image: nginx
    securityContext:
      seLinuxOptions:
        level: s0:c123,c456
        role: object_r
        type: svirt_sandbox_file_t
        user: system_u
EOF
pod/nginx-selinux-allowed created
kubectl get pod nginx-selinux-allowed -o json | jq ".spec.securityContext,.spec.containers[0].securityContext"

{
  "seLinuxOptions": {
    "level": "s0:c123,c456",
    "role": "object_r",
    "type": "svirt_sandbox_file_t",
    "user": "system_u"
  }
}
{
  "seLinuxOptions": {
    "level": "s0:c123,c456",
    "role": "object_r",
    "type": "svirt_sandbox_file_t",
    "user": "system_u"
  }
}

kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
    name: nginx-selinux-allowed2
    labels:
        app: nginx-selinux
spec:
  containers:
  - name: nginx
    image: nginx
    securityContext:
      seLinuxOptions:
        level: s0:c123,c456
        role: object_r
        type: svirt_sandbox_file_t
        user: system_u
  securityContext:
    seLinuxOptions:
      level: s0:c123,c456
      role: object_r
      type: svirt_sandbox_file_t
      user: system_u
EOF
pod/nginx-selinux-allowed2 created
kubectl get pod nginx-selinux-allowed2 -o json | jq ".spec.securityContext,.spec.containers[0].securityContext"

{
  "seLinuxOptions": {
    "level": "s0:c123,c456",
    "role": "object_r",
    "type": "svirt_sandbox_file_t",
    "user": "system_u"
  }
}
{
  "seLinuxOptions": {
    "level": "s0:c123,c456",
    "role": "object_r",
    "type": "svirt_sandbox_file_t",
    "user": "system_u"
  }
}

kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
    name: nginx-selinux-allowed3
    labels:
        app: nginx-selinux
spec:
  containers:
  - name: nginx
    image: nginx
EOF
pod/nginx-selinux-allowed3 created
kubectl get pod nginx-selinux-allowed3 -o json | jq ".spec.securityContext,.spec.containers[0].securityContext"

{
  "seLinuxOptions": {
    "level": "s0:c123,c456",
    "role": "object_r",
    "type": "svirt_sandbox_file_t",
    "user": "system_u"
  }
}
{
  "seLinuxOptions": {
    "level": "s0:c123,c456",
    "role": "object_r",
    "type": "svirt_sandbox_file_t",
    "user": "system_u"
  }
}

And it fails when the given value is invalid:

 kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
    name: nginx-selinux-allowed4
    labels:
        app: nginx-selinux
spec:
  containers:
  - name: nginx
    image: nginx
    securityContext:
      seLinuxOptions:
        level: s0:c123,c456
        role: foo
        type: svirt_sandbox_file_t
        user: system_u
EOF

Error from server: error when creating "STDIN": admission webhook "selinux.kubewarden.admission" denied the request: SELinux validation failed

Therefore, everything seem to be fine with the policy. I guess you are just missing the mutating: true in the policy configuration.

[chrisns]

curious, hadn't noticed it mutating, i wonder what the behaviour is with multiple containers in a pod with differing selinux profiles? is that possible? ¯_(ツ)_/¯

[jvanz]

curious, hadn't noticed it mutating, i wonder what the behaviour is with multiple containers in a pod with differing selinux profiles? is that possible? ¯_(ツ)_/¯

I do not think so. This is the output from the cluster running the Kubernetes PSP:

kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
    name: nginx-selinux-allowed
    labels:
        app: nginx-selinux
spec:
  containers:
  - name: nginx
    image: nginx
    securityContext:
      seLinuxOptions:
        level: s0:c123,c456
        role: object_r
        type: svirt_sandbox_file_t
        user: system_u
  - name: nginx2
    image: nginx
    securityContext:
      seLinuxOptions:
        level: s0:c123,c456
        role: foo
        type: svirt_sandbox_file_t
        user: system_u
EOF
Error from server (Forbidden): error when creating "STDIN": pods "nginx-selinux-allowed" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.containers[1].securityContext.seLinuxOptions.role: Invalid value: "foo": must be object_r]

I can not see how to configure that in the PSP. I guess is not supported.