mittwald/kube-av

AntiVirus automation on Kubernetes

KubeAV: AntiVirus automation on Kubernetes

KubeAV is a Kubernetes operator that automates malware detection on Kubernetes. This is potentially useful when you're managing (and serving) untrusted data in your Kubernetes volumes.

---

⚠️ COMPATIBILITY NOTICE: This project is a prototypical implementation that is under heavy development and not considered stable. Breaking changes may occur at any time and without notice.

---

Table of contents

• Installation
• Architecture
• Usage
  • Starting an AV scan on demand
  • Scheduling an AV scan for periodic execution
• Future features

Installation

Install this operator using Helm:

$ helm repo add mittwald https://helm.mittwald.de
$ helm repo update
$ kubectl create namespace kubeav-system
$ helm install kubeav mittwald/kubeav --namespace kubeav-system

Architecture

This operator consists of several components:

• The KubeAV operator runs the main controller loop. It watches for VirusScan and ScheduledVirusScan resources created by users (or itself).
• The KubeAV updater is a DaemonSet that is created by the manager to run on every node. It maintains a local copy of the ClamAV database on each node in your cluster.
• The KubeAV agent is run in Job resources that are managed by creating a VirusScan custom resource. The agent contains the actual virus scanner which uses the signature database which is maintained by the updater.

                            ┌────────────────┐
              creates       │ KubeAV updater │
           ┌───────────────▶│   (DaemonSet)  │
           │                └────────────────┘
┌──────────┴──────┐
│ KubeAV operator │
└──────────┬──────┘
           │  creates       ┌───────────────────┐                           ┌──────────────┐
  ┌────────────────────────▶|     VirusScan     ├──────────────────────────▶│ KubeAV agent │
  │        ├───────────────▶| (Custom Resource) │  creates (via operator)   │    (Job)     │
  │        │  watches       └───────────────────┘                           └──────────────┘
  │        │                          ▲
  │        │                          │ creates (via operator)
  │        │                          │
  │        │  creates       ┌─────────┴──────────┐
  ├────────────────────────▶│ ScheduledVirusScan │
  │        └───────────────▶│ (Custom Resource)  │
  │           watches       └────────────────────┘

  O
 /|\ User
 / \

Usage

Starting an AV scan on demand

An on-demand scan is modelled using the VirusScan custom resource (API group av.mittwald.de/v1beta1). In the .spec of a virus scan you can specify which files to scan and which engine to use (currently, only ClamAV is supported):

apiVersion: av.mittwald.de/v1beta1
kind: VirusScan
metadata:
  name: example-virusscan
spec:
  # supported values: ["ClamAV"]
  engine: ClamAV

  # list of volumes to scan
  targets:

    # "volume" may be any kind of VolumeSource that you'd also use in
    # a PodSpec.
    - volume:
        persistentVolumeClaim:
          claimName: my-pvc
      subPath: path/to/subdir

A VirusScan resource will be mapped to a Job (of the batch/v1 API group), which will in turn result in a Pod that runs the configured AV engine and that has all the specified volumes mounted.

The results of the AV scan will be written back into the .status property of the VirusScan resource:

$ kubectl get virusscans
NAME                SUMMARY                        SCHEDULED   COMPLETED   AGE
example-virusscan   Completed (1 infected files)   44s         11s         44s

The .status.scanResults property in the CR lists the individual files found by the scanner:

apiVersion: av.mittwald.de/v1beta1
kind: VirusScan
metadata:
  name: example-virusscan
spec: # ...
status:
  conditions:
    Completed:
      type: Completed
      status: "True"
    Positive:
      type: Positive
      status: "True"
  scanResults:
  # filePath:
  #   path to the infected file
  # matchingSignature:
  #   name of the detected signature as reported by the AV engine.
  - filePath: /scan/scan-target-0/infected-file
    matchingSignature: Eicar-Signature

Scheduling an AV scan for periodic execution

Periodic scanning can be configured using the ScheduledVirusScan resource.

apiVersion: av.mittwald.de/v1beta1
kind: ScheduledVirusScan
metadata:
  name: example-scheduledvirusscan
spec:
  # this is a standard cron schedule
  schedule: "0 */3 * * *"

  # how many "VirusScan" resources that were created from this
  # schedule should be kept.
  historySize: 3

  # template for a "VirusScan" resource
  template:
    spec:
      engine: ClamAV
      targets:
        - volume:
            hostPath:
              path: /
          subPath: root/virus

For these resources, KubeAV will create new VirusScan resources from the configured template at the specified interval.

Future features

• Alerting (maybe by adding a metric for counting infected files? Or by directly integrating something like the Prometheus alert manager)
• On-Access Scanning (Todo: determine feasibility in containerized environment)

kube-av's Issues

A suggestion + a question about how to scan a pod on demand.

Hey, I would like to use kube-av for an internship project and I found a potential alteration for the readme. For the example I had to add a claimName so that it becomes this:

apiVersion: av.mittwald.de/v1beta1
kind: VirusScan
metadata:
  name: example-virusscan
spec:
  # supported values: ["ClamAV"]
  engine: ClamAV

  # list of volumes to scan
  targets:

    # "volume" may be any kind of VolumeSource that you'd also use in
    # a PodSpec.
    - volume:
        persistentVolumeClaim:
            claimName: "claimName"
            path: my-pvc
        subPath: path/to/subdir  

Perhaps that could be an update for the readme?

For my question, I want to scan specific pods. How would I know the volume of a pod that I want to scan?
for example if my pod was created from this yaml file (from the official Kubernetes docs;

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 2 # tells deployment to run 2 pods matching the template
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80

How would I know what volume to add so I can start an on demand scan?

Also would it be possible to create a scan on demand through the Kubernetes c# client using a version of this method from the official c# client?

Do recursive scans

Is your feature request related to a problem? Please describe.
It seems that clamscan is told not to do recursive scans of the scan targets. This should be default or at least be an option you can set somehow. I cannot find anywhere how to set this, but perhaps there is a way?

Describe the solution you'd like
I would like that the -r is added to the list of arguments when running clamscan to scan files in all subdirectoies.

Describe alternatives you've considered
Perhaps an option to be able to set that recursive scan should be done.

Additional context
I have been using the latest images available (alpha-14)

Helm chart not available on helm.mittwald.de

Describe the issue
I am trying to install using the Helm chart as described in the readme but I get "Error: failed to download "mittwald/kubeav""

Additional context
If I do a
helm search repo mittwald
the kubeav chart is not in the listing.

kube-av is not listed in Mittwald's helm chart.

I followed the installation guide from the readme but the mittwald repo doesn't contain kube-av. When I use helm repo search mittwald it lists these:

mittwald/aptly
mittwald/brudi-operator 
mittwald/harbor-operator 
mittwald/kube-mail       
mittwald/kubernetes-replicator
mittwald/kubernetes-secret-generator   
mittwald/pypiserver               
mittwald/servicegateway    

Would it be possible for you to add kube-av to the mittwald repo?

Changes in configmap for freshclam is overwritten when reconciling updater DaemonSet. Bug or feature?

Describe the bug
I found the configmap kubeav-updater and saw that I could use this to add additional signature databases. But the complete configmap are updated and the content is reset when the code for reconciling updater DaemonSet is run every 30 minutes. Since this project is in it's early stages (but works well) it is perhaps not correct to call it a bug. But I do not know if I would call it a feature either.

To Reproduce
Make some changes in the configmap called kubeav-updater. Wait for about 30 minutes or until the controller do "reconciling updater DaemonSet". The configmap data is overwritten.

Expected behavior
I would like that my added signature databases was left intact in the configmap. A simple solution could be to only create the configmap if it does not exist. That would make the configmap a lot more useful.

Operator context
Helm, latest

Kubernetes context
Kubernetes Server version: 1.19

kubeav operator pod cannot start. Kubernetes reports ContainerCannotRun

Describe the bug
After install or upgrade with helm using version alpha8 or 9 the operator pod cannot start. kubectl describe of the pod give us the event:
Error: failed to start container "kubeav": Error response from daemon: OCI runtime create failed: container_linux.go:370: starting container process caused: exec: "/usr/local/bin/kubeav": stat /usr/local/bin/kubeav: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type

To Reproduce
install with helm with --version parameter: v1.0.0-alpha9 or v1.0.0-alpha8

Expected behavior
Container to run

Operator context
helm install kubeav mittwald/kubeav --namespace kubeav-system --version v1.0.0-alpha9

Kubernetes context
Kubernetes v1.19.6 (EKS)

Additional context
If a run the helm install but using --version v1.0.0-alpha7 it works as expected.