Policy to block access to secrets

[kyverno]

Policy to block access to secrets

[chipzoller]

I'll be glad to take this one if I can be assigned. I've already done half this anyway with an existing sample. Needs sample policy label.

[chipzoller]

Here's a modified version of this sample policy I contributed, augmented to additionally disallow secrets from volumes.

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: secrets-not-from-env-vars
spec:
  background: false
  validationFailureAction: enforce
  rules:
  - name: secrets-not-from-env-vars
    match:
      resources:
        kinds:
        - Pod
    validate:
      message: "Secrets cannot come from environment variables."
      pattern:
        spec:
          containers:
          - name: "*"
            =(env):
            - =(valueFrom):
                X(secretKeyRef):
  - name: secrets-not-from-vol
    match:
      resources:
        kinds:
        - Pod
    validate:
      message: "Secrets cannot come from volumes."
      pattern:
        spec:
          =(volumes):
          - X(secret): "null"

Because a service account token is mounted to all Pods by default, this second rule would essentially block any Pod from being created. So the question is would this be an acceptable solution to block access to secrets within a namespace, or would it need to be more focused to allow tokens but deny everything else?

[chipzoller]

Policies such as this one are waiting on the fix here (kyverno/kyverno#1314) to be effective.

[chipzoller]

I'd like your thoughts on this comment, @JimBugwadia

[chipzoller]

Thinking more about how this should get designed, it seems like such a policy would need to block only specific Secrets from being mounted into Pods and not disallow all Secrets (see the SA token above). I think it could cause more problems than it solved if such a policy just blocked wholesale any and all Secrets. The workflow logic would probably need to be:

• Secrets which should be disallowed need to be labelled/annotated with some identifier.
• Policy needs to check that Pods cannot mount any of these labelled/annotated Secrets unless the request comes from someone either with/without a given Role.
• Kyverno needs to check Pod request, get all Secrets in request, check for any Secrets with given label/annotation, block request if greater than zero are found if Role is/isnot X.

So this would require API object lookup, but I'm not sure that given how that's going to be implemented this would be initially possible.

[chipzoller]

[chipzoller]

apiVersion: kyverno.io/v1
kind: Policy
metadata:
  name: no-secrets
  annotations:
    policies.kyverno.io/title: Disallow all Secrets
    policies.kyverno.io/category: Other
    policies.kyverno.io/description: >-
      Disables the use of all Secrets in a Pod definition. In order to work effectively,
      this Policy needs a separate Policy or rule to require `automountServiceAccountToken=false`
      since this would otherwise result in a Secret being mounted.
spec:
  validationFailureAction: enforce
  rules:
  - name: secrets-not-from-env
    match:
      resources:
        kinds:
        - Pod
    validate:
      message: "No Secrets from env."
      pattern:
        spec:
          containers:
          - name: "*"
            =(env):
            - =(valueFrom):
                X(secretKeyRef): "null"
  - name: secrets-not-from-envfrom
    match:
      resources:
        kinds:
        - Pod
    validate:
      message: "No Secrets from envFrom."
      pattern:
        spec:
          containers:
          - name: "*"
            =(envFrom):
            - X(secretRef): "null"
  - name: secrets-not-from-volumes
    match:
      resources:
        kinds:
        - Pod
    validate:
      message: "No Secrets from volumes."
      pattern:
        spec:
          =(volumes):
          - X(secret): "null"

[chipzoller]

apiVersion: kyverno.io/v1
kind: Policy
metadata:
  name: only-safe-secrets
  annotations:
    policies.kyverno.io/title: Only safe Secrets
    policies.kyverno.io/category: Other
    policies.kyverno.io/description: >-
      Ensures that only Secrets beginning with the name "safe-" can be consumed by Pods.
      In order to work effectively,
      this Policy needs a separate Policy or rule to require `automountServiceAccountToken=false`
      since this would otherwise result in a Secret being mounted.
spec:
  validationFailureAction: enforce
  rules:
  - name: safe-secrets-from-env
    match:
      resources:
        kinds:
        - Pod
    validate:
      message: "Only Secrets beginning with `safe-` may be consumed in env statements."
      pattern:
        spec:
          containers:
          - name: "*"
            =(env):
            - =(valueFrom):
                =(secretKeyRef):
                    name: safe-*
  - name: safe-secrets-from-envfrom
    match:
      resources:
        kinds:
        - Pod
    validate:
      message: "Only Secrets beginning with `safe-` may be consumed in envFrom statements."
      pattern:
        spec:
          containers:
          - name: "*"
            =(envFrom):
            - =(secretRef):
                name: safe-*
  - name: safe-secrets-from-volumes
    match:
      resources:
        kinds:
        - Pod
    validate:
      message: "Only Secrets beginning with `safe-` may be consumed in volumes."
      pattern:
        spec:
          =(volumes):
          - =(secret):
              secretName: safe-*

[chipzoller]

Working PoC for 3. Only looks at Secrets from volumes, only the first volume (index 0), and is only applicable to Pods (no higher-level controllers). Blocks a Pod definition if its first volume mounts a secret containing a label foo=bar. Needs enhancements outlined in #1589 and #1587 to make this type of solution viable.

apiVersion: kyverno.io/v1
kind: Policy
metadata:
  name: select-secrets
spec:
  validationFailureAction: enforce
  rules:
  - name: select-secrets-from-volumes
    match:
      resources:
        kinds:
        # - Deployment
        - Pod
    context:
    - name: volsecret
      apiCall:
        #for Deployment
        # urlPath: "/api/v1/namespaces/{{request.object.metadata.namespace}}/secrets/{{request.object.spec.template.spec.volumes[0].secret.secretName}}"
        # for Pod
        urlPath: "/api/v1/namespaces/{{request.object.metadata.namespace}}/secrets/{{request.object.spec.volumes[0].secret.secretName}}"
        jmesPath: "metadata.labels.foo"
    preconditions:
    - key: "{{ request.operation }}"
      operator: Equals
      value: "CREATE"
    validate:
      #for Deployment
      # message: "The Secret named {{request.object.spec.template.spec.volumes[0].secret.secretName}} is restricted and may not be used."
      #for Pod
      message: "The Secret named {{request.object.spec.volumes[0].secret.secretName}} is restricted and may not be used."
      deny:
        conditions:
        - key: "{{ volsecret }}"
          operator: Equals
          value: "bar"

[chipzoller]

Option 3 is going to be the "best" option but will require the proposal as outlined here.

[chipzoller]

Tracking for 1.5.0

[chipzoller]

@JimBugwadia now that this feature is implemented, if you'd be able and willing to supple a brief PoC for this policy using foreach that'd be wonderful.

[JimBugwadia]

Will do.

[chipzoller]

I'm clearly not understanding how to reference nested foreach loops correctly. This is what I have so far:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: secret-lookup
  annotations:
    policies.kyverno.io/title: WIP check secrets
    policies.kyverno.io/category: Other
    policies.kyverno.io/severity: medium
    policies.kyverno.io/minversion: 1.5.0
    policies.kyverno.io/subject: Pod
    policies.kyverno.io/description: >-
      WIP     
spec:
  background: false
  validationFailureAction: enforce
  rules:
  - name: secrets-lookup-from-env
    match:
      resources:
        kinds:
        - Pod
    preconditions:
      all:
      # check if any secrets are in env statements
      - key: "{{ request.object.spec.containers[].env[].valueFrom.secretKeyRef | length(@)}}"
        operator: GreaterThanOrEquals
        value: 1
      - key: "{{request.operation}}"
        operator: In
        value:
        - CREATE
        - UPDATE
    validate:
      message: These Secrets are restricted.
      foreach:
      # get every secret, then do an API lookup on each one checking for protected labels.
      # deny the request if any of those secrets have protected=true.
      - list: "request.object.spec.containers[].env[].valueFrom.secretKeyRef"
        context:
        - name: isprotected
          apiCall:
            jmesPath: "metadata.labels.protected"
            urlPath: "/api/v1/namespaces/{{request.object.metadata.namespace}}/secrets/{{element.name}}"
        deny:
          conditions:
          - key: "{{ isprotected }}"
            operator: Equals
            value: "true"

A test resource:

apiVersion: v1
kind: Pod
metadata:
  name: kyvernopod
  labels:
    name: kyvernopod
spec:
  automountServiceAccountToken: false
  # initContainers: 
  # - name: initbusybox-harbor
  #   image: foapbmshyw:1.28
  #   command: ["sleep", "9999"]
  #   env:
  #   - name: initsomething
  #     valueFrom:
  #       secretKeyRef:
  #         name: safe-foo
  #         key: bar
  #   envFrom:
  #   - secretRef:
  #       name: safe-secured
  containers: 
  - name: busybox-harbor
    image: harbor2.zoller.com/library/busybox:1.28
    command: ["sleep", "9999"]
    env:
    - name: something
      valueFrom:
        secretKeyRef:
          name: foo
          key: bar
    - name: somethingelse
      valueFrom:
        secretKeyRef:
          name: umbra
          key: bissel
    envFrom:
      - secretRef:
          name: safe-secured
  volumes:
  - name: supersecret
    secret:
      secretName: safe-testing
  - name: vol1
    secret:
      secretName: safe-secret
  - name: vol2
    secret:
      secretName: unsafe-secret

$ k get secret --show-labels
NAME                  TYPE                                  DATA   AGE    LABELS
default-token-kllds   kubernetes.io/service-account-token   3      54m    <none>
foo                   Opaque                                1      9m8s   protected=true

[chipzoller]

Kyverno complains because I'm using the context variable isprotected and it says it's invalid. It seems like this should be correct though. The sequence is list => context => deny where the next logic references a variable in the previous. I'm not sure how else to make such a policy work if writing it like this isn't possible.

Error from server: error when creating "labeled-secrets.yaml": admission webhook "validate-policy.kyverno.svc" denied the request: policy contains invalid variables: Rule "secrets-lookup-from-env" has forbidden variables. Allowed variables are: {{request.*}}, {{serviceAccountName}}, {{serviceAccountNamespace}}, {{@}} and ones defined by the context

[chipzoller]

Tested and now working thanks to @JimBugwadia.

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: secret-lookup
  annotations:
    policies.kyverno.io/title: WIP check secrets
    policies.kyverno.io/category: Other
    policies.kyverno.io/severity: medium
    policies.kyverno.io/minversion: 1.5.0
    policies.kyverno.io/subject: Pod
    policies.kyverno.io/description: >-
      WIP     
spec:
  background: false
  validationFailureAction: enforce
  rules:
  - name: secrets-lookup-from-env
    match:
      resources:
        kinds:
        - Pod
    preconditions:
      all:
      # check if any secrets are in env statements
      - key: "{{ request.object.spec.containers[].env[].valueFrom.secretKeyRef | length(@)}}"
        operator: GreaterThanOrEquals
        value: 1
      - key: "{{request.operation}}"
        operator: In
        value:
        - CREATE
        - UPDATE
    validate:
      message: These Secrets are restricted.
      foreach:
      # get every secret, then do an API lookup on each one checking for the `status` labels.
      # deny the request if any of those secrets don't have `status=protected`.
      - list: "request.object.spec.containers[].env[].valueFrom.secretKeyRef"
        context:
        - name: status
          apiCall:
            jmesPath: "metadata.labels.status"
            urlPath: "/api/v1/namespaces/{{request.object.metadata.namespace}}/secrets/{{element.name}}"
        deny:
          conditions:
          - key: "{{ status }}"
            operator: NotEquals
            value: protected
  - name: secrets-lookup-from-envfrom
    match:
      resources:
        kinds:
        - Pod
    preconditions:
      all:
      # check if any secrets are in envfrom statements
      - key: "{{ request.object.spec.containers[].envFrom[].secretRef | length(@)}}"
        operator: GreaterThanOrEquals
        value: 1
      - key: "{{request.operation}}"
        operator: In
        value:
        - CREATE
        - UPDATE
    validate:
      message: These Secrets are restricted.
      foreach:
      # get every secret, then do an API lookup on each one checking for the `status` labels.
      # deny the request if any of those secrets don't have `status=protected`.
      - list: "request.object.spec.containers[].envFrom[].secretRef"
        context:
        - name: status
          apiCall:
            jmesPath: "metadata.labels.status"
            urlPath: "/api/v1/namespaces/{{request.object.metadata.namespace}}/secrets/{{element.name}}"
        deny:
          conditions:
          - key: "{{ status }}"
            operator: NotEquals
            value: protected
  - name: secrets-lookup-from-volumes
    match:
      resources:
        kinds:
        - Pod
    preconditions:
      all:
      # check if any secrets are in volume statements
      - key: "{{ request.object.spec.volumes[].secret | length(@)}}"
        operator: GreaterThanOrEquals
        value: 1
      - key: "{{request.operation}}"
        operator: In
        value:
        - CREATE
        - UPDATE
    validate:
      message: These Secrets are restricted.
      foreach:
      # get every secret, then do an API lookup on each one checking for the `status` labels.
      # deny the request if any of those secrets don't have `status=protected`.
      - list: "request.object.spec.volumes[].secret"
        context:
        - name: status
          apiCall:
            jmesPath: "metadata.labels.status"
            urlPath: "/api/v1/namespaces/{{request.object.metadata.namespace}}/secrets/{{element.secretName}}"
        deny:
          conditions:
          - key: "{{ status }}"
            operator: NotEquals
            value: protected

apiVersion: v1
kind: Pod
metadata:
  name: kyvernopod
  labels:
    name: kyvernopod
spec:
  automountServiceAccountToken: false
  containers: 
  - name: busybox-harbor
    image: aobuuenbys:1.28
    command: ["sleep", "9999"]
    env:
    - name: something
      valueFrom:
        secretKeyRef:
          name: safe-foo
          key: bar

$ k get secret safe-foo --show-labels
NAME       TYPE     DATA   AGE   LABELS
safe-foo   Opaque   1      20m   junky=cars,status=something

$ k apply -f pod.yaml 
Error from server: error when creating "pod.yaml": admission webhook "validate.kyverno.svc-fail" denied the request: 

resource Pod/default/kyvernopod was blocked due to the following policies

secret-lookup:
  secrets-lookup-from-env: validation failed in foreach rule for These Secrets are
    restricted.

[chipzoller]

Still same problem: kyverno/kyverno#2665 (comment)

[chipzoller]

After enhancing the policy with input from kyverno/kyverno#2665 , this is fully working as tested on Kyverno 1.6.0 dev:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: restrict-secrets-by-label
  annotations:
    policies.kyverno.io/title: Restrict Secrets by Label
    policies.kyverno.io/category: Other
    policies.kyverno.io/severity: medium
    # Need autogen for deny rules
    policies.kyverno.io/minversion: 1.6.0
    policies.kyverno.io/subject: Pod, Secret
    policies.kyverno.io/description: >-
      Secrets often contain sensitive information and their access should be carefully controlled.
      Although Kubernetes RBAC can be effective at restricting them in several ways,
      it lacks the ability to use labels on referenced entities. This policy ensures
      that only Secrets not labeled with `status=protected` can be consumed by Pods.
spec:
  background: false
  validationFailureAction: enforce
  rules:
  - name: secrets-lookup-from-env
    match:
      resources:
        kinds:
        - Pod
    preconditions:
      all:
      # check if any secrets are in env statements
      - key: "{{ request.object.spec.[containers, initContainers, ephemeralContainers][].env[].valueFrom.secretKeyRef || '' | length(@) }}"
        operator: GreaterThanOrEquals
        value: 1
      - key: "{{request.operation}}"
        operator: NotEquals
        value: DELETE
    validate:
      message: These Secrets are restricted.
      foreach:
      # get every secret, then do an API lookup on each one checking for the `status` label.
      # deny the request if any of those secrets have `status=protected`.
      - list: "request.object.spec.[containers, initContainers, ephemeralContainers][].env[].valueFrom.secretKeyRef"
        context:
        - name: status
          apiCall:
            jmesPath: "metadata.labels.status || ''"
            urlPath: "/api/v1/namespaces/{{request.namespace}}/secrets/{{element.name}}"
        deny:
          conditions:
          - key: "{{ status }}"
            operator: Equals
            value: protected
  - name: secrets-lookup-from-envfrom
    match:
      resources:
        kinds:
        - Pod
    preconditions:
      all:
      # check if any secrets are in envfrom statements
      - key: "{{ request.object.spec.[containers, initContainers, ephemeralContainers][].envFrom[].secretRef || '' | length(@) }}"
        operator: GreaterThanOrEquals
        value: 1
      - key: "{{request.operation}}"
        operator: NotEquals
        value: DELETE
    validate:
      message: These Secrets are restricted.
      foreach:
      # get every secret, then do an API lookup on each one checking for the `status` label.
      # deny the request if any of those secrets have `status=protected`.
      - list: "request.object.spec.[containers, initContainers, ephemeralContainers][].envFrom[].secretRef"
        context:
        - name: status
          apiCall:
            jmesPath: "metadata.labels.status || ''"
            urlPath: "/api/v1/namespaces/{{request.namespace}}/secrets/{{element.name}}"
        deny:
          conditions:
          - key: "{{ status }}"
            operator: AnyIn
            value: protected
  - name: secrets-lookup-from-volumes
    match:
      resources:
        kinds:
        - Pod
    preconditions:
      all:
      # check if any secrets are in volume statements
      - key: "{{ request.object.spec.volumes[].secret || '' | length(@) }}"
        operator: GreaterThanOrEquals
        value: 1
      - key: "{{request.operation}}"
        operator: NotEquals
        value: DELETE
    validate:
      message: These Secrets are restricted.
      foreach:
      # get every secret, then do an API lookup on each one checking for the `status` label.
      # deny the request if any of those secrets have `status=protected`.
      - list: "request.object.spec.volumes[].secret"
        context:
        - name: status
          apiCall:
            jmesPath: "metadata.labels.status || ''"
            urlPath: "/api/v1/namespaces/{{request.namespace}}/secrets/{{element.secretName}}"
        deny:
          conditions:
          - key: "{{ status }}"
            operator: Equals
            value: protected

Rules not copied for Pod controllers as this is now included in 1.6 via kyverno/kyverno#2947 .