anchorfree/vmbackup-sidecar

Backup sidecar container for VictoriaMetrics

Warning: use this tool only with VictoriaMetrics version < 1.29.0. Starting from v1.29.0 VictoriaMetrics added native vmbackup and vmrestore tools for fast backups on GCS and S3.

vmbackup-sidecar

Provides a sidecar backup service for VictoriaMetrics vmstorage component.

How it works

vmbackup-sidecar provides /backup/create API endpoint. After receiving GET request, it creates VictoriaMetrics vmstorage snapshot via snapshot API and syncs it with S3 bucket using aws s3 sync. When sync with S3 is completed, all vmstorage snapshots are removed to free up space. Depending on the number of vmstorage instances (pods) running, snapshots will be stored in S3 bucket in following tree:

<s3-bucket-name>
├── <vmstorage-pod-1>
│   ├── data
│   └── indexdb
├── <vmstorage-pod-2>
│   ├── data
│   └── indexdb
└── <vmstorage-pod-3>
    ├── data
    └── indexdb

Docker image

$ docker pull anchorfree/vmbackup-sidecar:latest

Configuration

Parameters for communicating with vmstorage instance and credentials for accessing S3 bucket are stored in environment variables:

VMSTORAGE_HOST: localhost      # vmstorage hostname (should be available via localhost as both containers are in the same Pod)
VMSTORAGE_PORT: "8482"         # which port vmstorage is listening for API requests
VM_SNAPSHOT_BUCKET: my-bucket  # S3 bucket name for syncing snapshot
VM_STORAGE_DATA_PATH: "/vmstorage-data"  # Corresponds to --storageDataPath flag in VictoriaMetrics setup
ENVIRONMENT: "dev"            # either "prod" or "dev", affects logs output (structlog for prod, plain for dev)
AWS_ACCESS_KEY_ID: <access_key>
AWS_SECRET_ACCESS_KEY: <secret_key>

Command-line options:

./backup-vm -h
Usage of ./backup-vm:
  -help
    Show usage
  -only-show-errors
    Only errors and warnings are displayed. All other output is suppressed
  -port int
    Port to listen (default 8488)

Deployment

vmbackup-sidecar container is supposed to be running in the same pod as VictoriaMetrics vmstorage component.

Here is a helm-chart which implements such kind of vmbackup-sidecar deployment: https://github.com/AnchorFree/helm-charts/blob/master/stable/victoria-metrics/templates/vmstorage-statefulset.yaml

vmbackup-sidecar's Issues

Remove filling empty dirs with .keep file

Starting from v1.28.3, empty directories structure is recreated automatically on the first run (restoring from backup scenario). The trick with .keep file can be removed.