undistro/marvin

Marvin is a CLI tool that scans a k8s cluster by performing CEL expressions to report potential issues, misconfigurations and vulnerabilities.

Marvin is a CLI tool designed to help Kubernetes cluster administrators ensure the security and reliability of their environments.

Using a comprehensive set of CEL (Common Expression Language) expressions, Marvin performs extensive checks on cluster resources, identifying potential issues, misconfigurations, and vulnerabilities that could pose a risk to the system. It helps ensure that your Kubernetes clusters are always in compliance with best practices and industry standards.

Marvin is also used as a plugin in Zora.

• Installation
  • Manually
  • Install via script
  • Install via Krew
  • Install from source
• Usage
  • Built-in checks
  • Custom checks
  • Skipping resources
  • RBAC
• Contributing
• License

Please star ⭐ the repo if you want us to continue developing and improving Marvin! 😁

Installation

The pre-compiled binaries are available in GitHub releases page and can be installed manually, via script or as a kubectl plugin with Krew.

Manually

1. Download the file for your system/architecture from the GitHub releases page
2. Unpack the downloaded archive (e.g tar -xzf marvin_Linux_x86_64.tar.gz)
3. Make sure the binary has execution bit turned on (chmod +x ./marvin)
4. Move the binary somewhere in your $PATH (e.g sudo mv ./marvin /usr/local/bin/)

Install via script

The process above can be automated by the following script:

curl -sSfL https://raw.githubusercontent.com/undistro/marvin/main/install.sh | sh -s -- -b $HOME/.local/bin

Install via Krew

You can install Marvin as a kubectl plugin via Krew:

kubectl krew install marvin

Then you can use Marvin with kubectl prefix:

kubectl marvin version

Install from source

go install github.com/undistro/marvin@latest

Usage

Built-in checks

Scan the current-context Kubernetes cluster performing the built-in checks:

marvin scan

SEVERITY   ID      CHECK                                                   STATUS   FAILED   PASSED   SKIPPED 
High       M-101   Host namespaces                                         Failed   8        25       0         
High       M-104   HostPath volume                                         Failed   8        25       0         
High       M-201   Application credentials stored in configuration files   Failed   2        45       0         
High       M-102   Privileged container                                    Failed   2        31       0         
High       M-103   Insecure capabilities                                   Failed   2        31       0         
High       M-100   Privileged access to the Windows node                   Passed   0        33       0         
High       M-105   Not allowed hostPort                                    Passed   0        33       0         
Medium     M-113   Container could be running as root user                 Failed   33       0        0         
Medium     M-407   CPU not limited                                         Failed   31       2        0         
Medium     M-406   Memory not limited                                      Failed   27       6        0         
Medium     M-404   Memory requests not specified                           Failed   26       7        0         
Medium     M-402   Readiness and startup probe not configured              Failed   25       8        0         
Medium     M-403   Liveness probe not configured                           Failed   25       8        0         
Medium     M-405   CPU requests not specified                              Failed   23       10       0         
Medium     M-106   Forbidden AppArmor profile                              Passed   0        33       0         
Medium     M-107   Forbidden SELinux options                               Passed   0        33       0         
Medium     M-108   Forbidden proc mount type                               Passed   0        33       0         
Medium     M-109   Forbidden seccomp profile                               Passed   0        33       0         
Medium     M-110   Unsafe sysctls                                          Passed   0        33       0         
Medium     M-112   Allowed privilege escalation                            Passed   0        33       0         
Medium     M-114   Container running as root UID                           Passed   0        33       0         
Medium     M-200   Image registry not allowed                              Passed   0        33       0         
Medium     M-400   Image tagged latest                                     Passed   0        33       0         
Medium     M-408   Sudo in container entrypoint                            Passed   0        33       0         
Medium     M-409   Deprecated image registry                               Passed   0        33       0         
Medium     M-500   Workload in default namespace                           Passed   0        33       0         
Medium     M-410   Not allowed restartPolicy                               Passed   0        18       0         
Low        M-116   Not allowed added/dropped capabilities                  Failed   33       0        0         
Low        M-202   Automounted service account token                       Failed   33       0        0         
Low        M-115   Not allowed seccomp profile                             Failed   29       4        0         
Low        M-300   Root filesystem write allowed                           Failed   29       4        0         
Low        M-111   Not allowed volume type                                 Failed   8        25       0         
Low        M-203   SSH server running inside container                     Passed   0        39       0         
Low        M-401   Unmanaged Pod                                           Passed   0        15       0         

The default output format is table which represents a summary of checks result. You can provide json or yaml in the -o/--output flag to get more details.

Run marvin scan --help to see all available options.

Custom checks

Marvin allows you to write your own checks by using CEL expressions in a YAML file like the example below.

id: CUSTOM-001
severity: Medium
message: "Replicas limit"
match:
  resources:
    - group: apps
      version: v1
      resource: deployments
validations:
  - expression: >
      object.spec.replicas <= 5
    message: "Deployment with more than 5 replicas"

If an expression evaluates to false, the check fails.

This is how built-in Marvin checks are defined as well. You can see all the built-in checks in the internal/builtins folder for examples.

If you want to quickly test CEL expressions from your browser, check out the CEL Playground.

Then provide the directory path with your custom check files in the -f/--checks flag:

marvin scan --disable-builtin --checks ./examples/

SEVERITY   ID           CHECK            STATUS   FAILED   PASSED   SKIPPED 
Medium     CUSTOM-001   Replicas limit   Passed   0        2        0         

The flag --disable-builtin disables the built-in Marvin checks.

If the check matches a PodSpec (Pod, ReplicationController, ReplicaSet, Deployment, StatefulSet, DaemonSet, Job or CronJob) the podSpec and allContainers inputs are available for expressions.

The allContainers input is a list of all containers including initContainers and ephemeralContainers.

Skipping resources

You can use annotations to skip certain checks for specific resources in your cluster. By adding the marvin.undistro.io/skip annotation to a resource, you can specify a comma-separated list of check IDs to skip.

Example:

kubectl annotate deployment nginx marvin.undistro.io/skip='M-202, M-111'

By default, Marvin will respect the marvin.undistro.io/skip annotation when performing checks. However, you can disable this behavior by using the --disable-annotation-skip flag. This flag will cause Marvin to perform all checks on all resources.

If you prefer to use a different annotation to skip checks, you can use the --skip-annotation flag to specify the annotation name. Example: --skip-annotation='my-company.com/skip-checks'

RBAC

Currently, the built-in checks look for the below resources and Marvin needs view (get and list) permission to verify them.

• v1/pods
• v1/configmaps
• v1/services
• apps/v1/deployments
• apps/v1/daemonsets
• apps/v1/statefulsets
• apps/v1/replicasets
• batch/v1/cronjobs
• batch/v1/jobs

Here is a sample `ClusterRole` for Marvin:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: marvin
rules:
  - apiGroups: [ "" ]
    resources:
      - configmaps
      - pods
      - services
    verbs: [ "get", "list" ]
  - apiGroups: [ "apps" ]
    resources:
      - daemonsets
      - deployments
      - statefulsets
      - replicasets
    verbs: [ "get", "list" ]
  - apiGroups: [ batch ]
    resources:
      - jobs
      - cronjobs
    verbs: [ "get", "list" ]

Note You can write a custom check to look at any resource. But Marvin needs view permission. Remember to update RBAC for new resources you want to check.

Contributing

We appreciate your contribution. Please refer to our contributing guideline for further information. This project adheres to the Contributor Covenant code of conduct.

License

Marvin is available under the Apache 2.0 license. See the LICENSE file for more info.

marvin's Issues

Updating vulnerable images at runtime

Feature Description

Hability to update packages at pod runtime.

Is your feature request related to a problem?

Usually reports are too long and prone to be ignored by developers. This proposal is to be used as a last resource in order to minimize vulnerable code to run.

Thes idea is as follow:

• Marvin/Zora/whatever scans the image and generates a vulnerability report.
• A workload ValidatingWebhook inside the cluster checks for matches with vulnerable imagens from the reports
• For a given match, webhook changes each containers.command with a pre-exec script/binary which updates vulnerable dependencies (add, remove, update packages/files/configs) and exec() the original command (either explicitly set in the original containers.command or from image COMMAND.
• Upon failure, it could fallback to dont-update and leave the next pod untuched by labeling the owner workload object.

What alternatives have you considered?

Slim detects and minifies the image by using probe heuristics. The drawback is you need to interact with the application in order to generate data for slim to detect (access files, network activity, exec commands...)

https://github.com/slimtoolkit/slim

It's also an standalone tool, running outside of the cluster, thus it have no knowledge of the environment the app runs.

Additional Context

For example the webhook could change a deployment to this:

kind: Deployment:
...
spec:
  template:
    spec:
      container:
      - name: web
        commands:
        - sh
        - -c
        - 'curl https://undistro.io/update/alpine/3.2 | sh /app/server --listen 8080'

The update script will "fix" the image and exec() the original command /app/server from the image's COMMAND or from the original Deployment object.

To ignore certain deployments, a label marvin.io/dont-auto-update may be set to it. This could be used to ignore subsequent auto-updates to be applied by the webhook.

Caveats

This is certainly an antipattern since on-the-fly updated images will not be immutable anymore.
Also, this only work when pod.spec.containers.securityContext.readOnlyRootFilesystem=false.

An alternative solution is to have a daemonset agent to rebuild the images locally in all nodes with a new name.
All the webhook now has to do is mutate the workload's image name to the secured one.

CVEs found in the base image of alpine:3.17.2

Scanning the main image of Marvin, trivy found some CVEs in two packages:

• libcrypto3
• libssl3

Suggestion: update this base image to 3.18.4 version and update those packages to fix the vulnerabilities.

FROM alpine:3.18.4

RUN addgroup -g 8494 -S nonroot && adduser -u 8494 -D -S nonroot -G nonroot \
    && apk add libcrypto3=3.1.4-r0 libssl3=3.1.4-r0 # fix CVE-2023-5363

RFE: offline manifest scanning

Feature Description

Ability to scan rendered manifests offline

Is your feature request related to a problem?

In my automated GitOps workflow that deploys resources to the cluster, I want to verify any resource changes against the cluster's PSA policies at build time, before the changes are merged and applied by ArgoCD.

What alternatives have you considered?

Alternatives are conftest and kyverno CLI

Additional Context

https://undistrocommunity.slack.com/archives/C05AF6BDAAU/p1701831391878269