Giter Site home page Giter Site logo

docker-automysqlbackup's Introduction

Docker AutoMySQLBackup

A lightweight image for creating and managing scheduled MySQL backups. Runs a slightly modified AutoMySQLBackup utility.

Supported tags and respective Dockerfile links

Version

This image uses AutoMySQLBackup 2.5 from Debian Linux source repository as a base, branched at 2.6+debian.4-1 tag. Original source can be cloned from git://anonscm.debian.org/users/zigo/automysqlbackup.git or taken at the appropriate Debian package page.

Custom modifications:

  • passed logging to stdout/stderr
  • removed error logs mailing code
  • made default configuration more suitable for docker container

Image usage

Let's create a bridge network and start a MySQL container as an example.

docker network create dbtest
docker run --name some-mysql --network dbtest \
    -e MYSQL_ROOT_PASSWORD=my-secret-pw -d mysql:latest

For the basic one-shot backup, you can run a container like this:

docker run --network dbtest \
    -v '/var/lib/automysqlbackup:/backup' \
    -e DBHOST=some-mysql \
    -e DBNAMES=all \
    -e USERNAME=root \
    -e PASSWORD=my-secret-pw \
    -e DBNAMES=all \
    automysqlbackup:2.7.0

Container will create dumps of all datebases from MySQL inside /var/lib/automysqlbackup directory and exit.

To run container in a scheduled mode, populate CRON_SCHEDULE environment variable with a cron expression.

docker run --network dbtest \
    -v '/var/lib/automysqlbackup:/backup' \
    -e DBHOST=some-mysql \
    -e DBNAMES=all \
    -e USERNAME=root \
    -e PASSWORD=my-secret-pw \
    -e DBNAMES=all \
    -e CRON_SCHEDULE="0 0 * * *" \
    automysqlbackup:2.7.0

Instead of passing environment variables though docker, you can also mount a file with their declarations as volume. See defaults file in this image's git repository for the example.

docker run --network dbtest \
    -v '/var/lib/automysqlbackup:/backup' \
    -v '/etc/default/automysqlbackup:/etc/default/automysqlbackup:ro' \
    automysqlbackup:2.7.0

Usage with docker-compose

For the example of using this image with docker-compose, see docker-compose.yml file in the image's repository.

Quick tips:

  • You can call automysqlbackup binary directly for the manual backup: docker-compose exec mysqlbackup automysqlbackup
  • Use only YAML dictionary for passing CRON_SCHEDULE environment variable CRON_SCHEDULE: "0 0 * * *" as YAML sequence - CRON_SCHEDULE="0 * * * *" will preserve quotes breaking go-cron (Issue #1).

Environment variables

  • CRON_SCHEDULE
    If set to cron expression, container will start a cron daemon for scheduled backups.

  • USERNAME
    Username to access the MySQL server.

  • PASSWORD
    Password to access the MySQL server.

  • DBHOST
    Host name (or IP address) of MySQL server.

  • DBPORT
    Port of MySQL server.

  • DBNAMES
    List of space separated database names for Daily/Weekly Backup. Set to all for all databases.
    Default value: all

  • BACKUPDIR
    Backup directory location. Folders inside this one will be created (daily, weekly, etc.), and the subfolders will be database names.
    Default value: /backup

  • MDBNAMES
    List of space separated database names for Monthly Backups.
    Will mirror DBNAMES if DBNAMES set to all.

  • DBEXCLUDE
    List of DBNAMES to exclude if DBNAMES are set to all (must be in " quotes).

  • IGNORE_TABLES
    List of space separated table names in a format of db_name.tbl_name to exclude from backup (must be in " quotes).

  • CREATE_DATABASE
    Include CREATE DATABASE in backup?
    Default value: yes

  • SEPDIR
    Separate backup directory and file for each DB? (yes or no).
    Default value: yes

  • DOWEEKLY
    Which day do you want weekly backups? (1 to 7 where 1 is Monday).
    Default value: 6

  • COMP
    Choose Compression type. (gzip or bzip2)
    Default value: gzip

  • COMMCOMP
    Compress communications between backup server and MySQL server?
    Default value: no

  • LATEST
    Additionally keep a copy of the most recent backup in a seperate directory.
    Default value: no

  • MAX_ALLOWED_PACKET
    The maximum size of the buffer for client/server communication. e.g. 16MB (maximum is 1GB)

  • SOCKET
    For connections to localhost. Sometimes the Unix socket file must be specified.

  • PREBACKUP
    Command to run before backups

  • POSTBACKUP
    Command run after backups

  • ROUTINES
    Backup of stored procedures and routines
    Default value: yes

  • EXTRA_OPTS
    Pass any arbitrary flags to mysqldump, e.g. --single-transaction.

  • USER_ID Run the backup process from the specified user id. Allows matching backup files permissions with the host's user.
    Default value: 1 (root).

  • GROUP_ID Run the backup process from the specified group id. Allows matching backup files permissions with the host's group.
    If empty, matches USER_ID.

Docker Secrets

As an alternative to passing sensitive information via environment variables, _FILE may be appended to some of the previously listed environment variables, causing the initialization script to load the values for those variables from files present in the container. In particular, this can be used to load passwords from Docker secrets stored in /run/secrets/<secret_name> files. For example:

docker run --name automysqlbackup -e USERNAME=root -e PASSWORD_FILE=/run/secrets/mysql_root_password automysqlbackup

Currently, this is only supported for USERNAME and PASSWORD.

FAQ

  • Will you add support for AutoMySQLBackup 3?
    No. AutoMySQLBackup 3 was a complete rewrite of the script with much higher complexity but was abandoned in 2011 before it released. There are multiple repositories which try to support it by fixing bugs and ensuring compatibility with newer MySQL versions but I don't have time to track changes in those to properly support docker image.

  • Can you add CONFIG_* option
    Those options appeared in AutoMySQLBackup 3. See the above question.

License

Similar to the original automysqlbackup script, all sources for this image are licensed under GPL-2.0.

docker-automysqlbackup's People

Contributors

guillaume-gambs avatar jamesgol avatar pmontepagano avatar selim13 avatar thomasgoirand avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

docker-automysqlbackup's Issues

Add Support for Ports

Hello,

is it possible for you to add support for custom / bespoke ports please as the SQL service we use, doesn't run on the default port.

Tom

How to send error notications?

The readme says "removed error logs mailing code"
What is an alternative to receive notifications about errors? Is it somehow possible to call a webservice after error?

selim13/automysqlbackup:2.6-3-debian doesn't do full backups

I pulled this image and ran it. I backed up the first table of the database and quit.

I pulled image selim13/automysqlbackup:2.6-4-debian and it seems to be working properly.

I only bring this up because your docker-compose example has selim13/automysqlbackup:2.6-3-debian

start.sh - bad substitution

Starting gives:

/usr/local/bin/start.sh: 13: /usr/local/bin/start.sh: Bad substitution

Pinned this down to usage of #!/bin/sh instead of #!/bin/bash since sh is now symlinked to dash. Please change the first line to #!/bin/bash.

Startup jobs (cron/backup) don't work

I have a problem, might be something simple.

I'm using docker-compose like below (i'm not writing all the input/output since i have multiple files/containers etc). The mysqlbackup container works fine, i can use:
docker exec -it mysqlbackup sh and run automysqlbackup manually - but the cron jobs or even starting `docker-compose up -d" without CRON_SCHEDULE specified doesn't do anything for me. Thanks for any input.

  mysqlbackup:
    container_name: mysqlbackup
    image: selim13/automysqlbackup
    volumes:
      - "mysqlbackup-data:/backup"
    environment:
      - USERNAME=user
      - PASSWORD=password
      - DBHOST=typo3_db
      - CRON_SCHEDULE="0 * * * *"
    depends_on:
      - typo3_db

docker ps

CONTAINER ID        IMAGE                                  COMMAND                  CREATED             STATUS              PORTS                           NAMES
3200d1e77a3d        selim13/automysqlbackup                "start.sh"               6 minutes ago       Up 6 minutes                                        mysqlbackup

docker logs mysqlbackup gives:

2017/10/06 22:24:07 Running version:
2017/10/06 22:24:07 new cron: 0 "0 * * * *"
2017/10/06 22:24:07 Opening port 18080 for health checking

Can the container cron wake up the host?

If the host machine is asleep, what is the state of the container? Is it suspended, or can cron still execute at a given clock time? I would like to run this when I'm not actively using the machine but when I'm not using it, it goes to sleep. Thanks.

File and directory rights of the backuped files.

I have a problem with the rights of the backuped files. My dockers run as docker with group users rwxr.x.r.x , but docker-automysqlbackup puts the backuped databases as root:root rw...... so when i try to backup with Duplicati it is not possible. Is there a way to solve this from the docker?

Add Retention Options

In AutoMySQL backup, there is retention preferences, could you include them in this container please?

`
Set rotation of daily backups. VALUE*24hours
If you want to keep only today's backups, you could choose 1,
i.e. everything older than 24hours will be removed.
CONFIG_rotation_daily=6

Set rotation for weekly backups. VALUE*24hours
CONFIG_rotation_weekly=35

Set rotation for monthly backups. VALUE*24hours
CONFIG_rotation_monthly=150
`

How to start selim13/automysqlbackup as a daemon?

How to start selim13/automysqlbackup as a daemon?

I am new in Docker.

This works fine but requires opened terminal:
sudo docker run
--name automysqlbackup
--network my-network
-v '/var/xxxxxxxx/backup/docker:/backup'
-e DBHOST=mariadb
-e DBNAMES=all
-e USERNAME=root
-e PASSWORD=xxxxxxxx
-e DBNAMES=all
-e CRON_SCHEDULE="10 06 * * *"
selim13/automysqlbackup:2.6-7-debian

I tried to start it as daemon (-d --restart unless-stopped) but it doesn't work unfortunatelly:
sudo docker run
--name automysqlbackup
--network my-network
-v '/var/xxxxxxxx/backup/docker:/backup'
-e DBHOST=mariadb
-e DBNAMES=all
-e USERNAME=root
-e PASSWORD=xxxxxxxx
-e DBNAMES=all
-e CRON_SCHEDULE="10 06 * * *"
-d --restart unless-stopped
selim13/automysqlbackup:2.6-7-debian

What do I wrong? Thanks

Not creating any backups

installed it as docker compose using the following info

  automysqlbackup:
    image: selim13/automysqlbackup
    container_name: automysqlbackup
    restart: unless-stopped
    environment:
      - PGID=${PGID}
      - PUID=${PUID}
      - TZ=${TZ}
    volumes:
      - ${DOCKERCONFDIR}/automysqlbackup:/backup
      - /etc/localtime:/etc/localtime:ro
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      USERNAME: username
      PASSWORD: "password"
      DBHOST: mariadb
      DBPORT: 3306
      CRON_SCHEDULE: '*/15 * * * *'
    depends_on:
      - mariadb

it startup without any errors as seen in the log

2019-01-27T12:13:03.631221633Z 2019/01/27 13:13:03 Running version: 


2019-01-27T12:13:03.631254193Z 2019/01/27 13:13:03 new cron: 0 */15 * * * *


2019-01-27T12:13:03.631375409Z 2019/01/27 13:13:03 Opening port 18080 for health checking


2019-01-27T12:15:00.002352370Z 2019/01/27 13:15:00 13 cmd: automysqlbackup 


2019-01-27T12:15:00.039995182Z 2019/01/27 13:15:00 13: ======================================================================


2019-01-27T12:15:00.040023246Z AutoMySQLBackup VER 2.5


2019-01-27T12:15:00.040043647Z http://sourceforge.net/projects/automysqlbackup/


2019-01-27T12:15:00.040049210Z 


2019-01-27T12:15:00.040053320Z Backup of Database Server - mariadb


2019-01-27T12:15:00.040057604Z ======================================================================


2019-01-27T12:15:00.041068087Z 2019/01/27 13:15:00 13: Backup Start Time Sun Jan 27 13:15:00 CET 2019


2019-01-27T12:15:00.041081012Z ======================================================================


2019-01-27T12:15:00.041567320Z 2019/01/27 13:15:00 13: Backup End Sun Jan 27 13:15:00 CET 2019


2019-01-27T12:15:00.041577601Z ======================================================================


2019-01-27T12:15:00.041582569Z Total disk space used for backup storage..


2019-01-27T12:15:00.041586882Z Size - Location


2019-01-27T12:15:00.045698045Z 2019/01/27 13:15:00 13: 16K /backup


2019-01-27T12:15:00.045712103Z 


2019-01-27T12:15:00.045715605Z ======================================================================


2019-01-27T12:15:00.045718670Z If you find AutoMySQLBackup valuable please make a donation at


2019-01-27T12:15:00.045721590Z 2019/01/27 13:15:00 13: http://sourceforge.net/project/project_donations.php?group_id=101066


2019-01-27T12:15:00.045724504Z ======================================================================

it has created the folders where the backup should happen.
but nothing is getting back-upped.

releases?

Hi, do you plan to create release tags for this project?

MySQL 8: caching_sha2_password not supported

Using the container with mysql 8 I get following error:

ERROR 2059 (HY000): Authentication plugin 'caching_sha2_password' cannot be loaded: Error loading shared library /usr/lib/mysql/plugin/caching_sha2_password.so: No such file or directory

Got error: 1556: You can't use locks with log tables when using LOCK TABLES

Running this against a MariaDB 10.4.8 I get the error in the subject and the backup of the mysql database does not contain any tables.

  mysqlbackup:
    image: selim13/automysqlbackup:2.6-6-debian
    volumes:
      - "mariadb-backup:/backup"
    environment:
      USERNAME: root
      PASSWORD: "..."
      DBHOST: cloudserver_db
      DBEXCLUDE: "performance_schema information_schema"
      CRON_SCHEDULE: "22 22 * * *"
    depends_on:
      - db
# docker-compose exec mysqlbackup automysqlbackup
mysql: [Warning] Using a password on the command line interface can be insecure.
======================================================================
AutoMySQLBackup VER 2.5
http://sourceforge.net/projects/automysqlbackup/

Backup of Database Server - cloudserver_db
======================================================================
Backup Start Time Wed Oct 2 21:08:36 UTC 2019
======================================================================
Daily Backup of Database ( mysql )
Rotating last weeks Backup...

mysqldump: [Warning] Using a password on the command line interface can be insecure.
mysqldump: Got error: 1556: You can't use locks with log tables when using LOCK TABLES

Backup Information for /backup/daily/mysql/mysql_2019-10-02_21h08m.Wednesday.sql
         compressed        uncompressed  ratio uncompressed_name
                535                1088  56.0% /backup/daily/mysql/mysql_2019-10-02_21h08m.Wednesday.sql

The dump contains just a bunch of comments and then

--
-- Current Database: `mysql`
--

CREATE DATABASE /*!32312 IF NOT EXISTS*/ `mysql` /*!40100 DEFAULT CHARACTER SET latin1 */;

USE `mysql`;

Other databases are dumped just fine. Changes between 2.6-6-debian and latest don't seem to affect anything relevant.

Thanks

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.