holland-backup/holland

Holland Backup Manager

Holland README

Holland is an Open Source backup framework originally developed by Rackspace and written in Python. Its goal is to help facilitate backing up databases with greater configurability, consistency, and ease. Holland currently focuses on MySQL, however future development will include other database platforms and even non-database related applications. Because of its plugin structure, Holland can be used to backup anything you want by whatever means you want.

Documentation
Install Instructions
Holland Configuration Guide

LICENSE

holland-core, the basic backup framework is licensed under the New-BSD ("3-clause") license. See LICENSE for more details.

Each of these are under their respective licenses. Please see LICENSE for more information.

holland's Issues

Preserve existing environment with scripts/mkvirtenv.py

If ~/holland-test already exists, only reinstall packages if absolutely necessary. Also, try not override the existing test config which may have been altered previously.

Foo

bar

Uncaught Exception for Unkown Database

The core issue ended up being a MySQL problem, though it looks like we are not capturing exceptions thrown due to a MySQL error. The Holland output is: http://gist.github.com/392572

The specific issue was due to MySQL doing something silly with lower_case_table_names but the fact that Holland did not behave all that well seems to be a legitimate issue.

'purge' command not working as [not] advertised

 $ holland bk 
 Holland 1.0.0 started with pid 31758
 Acquired lock /Users/wdierkes/holland-test/etc/holland/backupsets/sqlite.conf :   '/Users/wdierkes/holland-test/etc/holland/backupsets/sqlite.conf'
 Validating config: sqlite
 Checking that SQLite backups can run. 
 Purged sqlite/20100517_185143
 Purged sqlite/20100515_145006
 2 backups purged 
 Estimated Backup Size: 4.00KB
 Starting backup[sqlite/20100701_123614] via plugin sqlite
 SQLite binary is [/usr/bin/sqlite3]
 Backing up SQLite database at [/Users/wdierkes/sqlite_database1.db]
 Backing up SQLite database at [/Users/wdierkes/sqlite_database2.db]
 Final on-disk backup size 402.00B 9.81% of estimated size 4.00KB
 Backup completed in 0.14 seconds
 Extremely low free space on env/var/spool/holland/sqlite/20100701_123614's filesystem  ( /).
 9.01GB of 185.99GB [4.85%] remaining
 Released lock /Users/wdierkes/holland-test/etc/holland/backupsets/sqlite.conf
 
 $ holland lb
 Holland 1.0.0 started with pid 31765
 Backupset[sqlite]:
         sqlite/20100517_190847
         sqlite/20100701_123614
 
 $ holland purge sqlite/20100517_190847
 Holland 1.0.0 started with pid 31766
 Would purge single backup 'sqlite/20100517_190847' 201.00B
 
 $ holland lb
 Holland 1.0.0 started with pid 31771
 Backupset[sqlite]:
         sqlite/20100517_190847
         sqlite/20100701_123614


 $ holland purge sqlite/20100517_190847 --force
 Holland 1.0.0 started with pid 31806
 Purged sqlite/20100517_190847

 $ holland lb
 Holland 1.0.0 started with pid 31808
 Backupset[sqlite]:
         sqlite/20100701_123614

scripts/build_rpms.py should use git archive

The default behavior of scripts/build_rpms.py of creating a random directory under $HOME is slightly annoying as I have to track down and clean these later. Often when this fails I fix something and try a second time, which creates another directory.

There is --tmpdir which solves part of this problem. However, providing an explicit --tmpdir under my git tree (say in an unversioned staging environment) will cause shutil.copytree to go in an infinite loop in certain cases and eat up all my disk space. :)

Rather than copytree, I suggest we simply do the much simpler, git archive --prefix=holland-${version}/ HEAD > $dest_srcdir/SOURCES/$name.tar.gz. Also with --tmpdir, it would be nice (IMHO) to have this be the actual %_topdir and not simply be the prefix of mkdtemp.

holland rpm leaves empty directories on uninstall

$ find /usr/lib/python2.4/site-packages/holland/
find: /usr/lib/python2.4/site-packages/holland/: No such file or directory
$ python scripts/build_rpms.py --topdir $PWD/staging/
...
$ rpm -Uhv staging/RPMS/noarch/holland-*noarch.rpm
$ find /usr/lib/python2.4/site-packages/holland/
/usr/lib/python2.4/site-packages/holland/
/usr/lib/python2.4/site-packages/holland/restore
/usr/lib/python2.4/site-packages/holland/restore/mysqlrestore
/usr/lib/python2.4/site-packages/holland/restore/mysqlrestore/py23
/usr/lib/python2.4/site-packages/holland/restore/mysqlrestore/mysqldump
/usr/lib/python2.4/site-packages/holland/restore/mysqlrestore/test
/usr/lib/python2.4/site-packages/holland/lib
/usr/lib/python2.4/site-packages/holland/lib/archive
/usr/lib/python2.4/site-packages/holland/backup
/usr/lib/python2.4/site-packages/holland/backup/lvm
/usr/lib/python2.4/site-packages/holland/backup/lvm/cli
/usr/lib/python2.4/site-packages/holland/backup/lvm/py23
/usr/lib/python2.4/site-packages/holland/backup/lvm/pylvm
/usr/lib/python2.4/site-packages/holland/backup/lvm/actions
/usr/lib/python2.4/site-packages/holland/backup/lvm/actions/archive
/usr/lib/python2.4/site-packages/holland/backup/lvm/actions/mysql
/usr/lib/python2.4/site-packages/holland/backup/lvm/util
/usr/lib/python2.4/site-packages/holland/backup/mysqldump
/usr/lib/python2.4/site-packages/holland/backup/mysqldump/mysql
/usr/lib/python2.4/site-packages/holland/backup/mysqldump/util
/usr/lib/python2.4/site-packages/holland/backup/mysqldump/mock
/usr/lib/python2.4/site-packages/holland/core
/usr/lib/python2.4/site-packages/holland/core/backports
/usr/lib/python2.4/site-packages/holland/core/backports/logging
/usr/lib/python2.4/site-packages/holland/core/command
/usr/lib/python2.4/site-packages/holland/core/config
/usr/lib/python2.4/site-packages/holland/core/util
/usr/lib/python2.4/site-packages/holland/commands

So essentially all directories remain after uninstall since moving to the setup.py --record magic. I guess we need to explicitly assign directories to packages? I guess my rpm foo is weak. :)

Write a functional test suite

Currently the functional (non-unit) test_suite is broken and not easily maintainable. This needs to be thrown out and redone, preferably using a full featured scripting language (such as Python) as the original suite was written in BASH and limited some of the functionality.

The ultimate goal is to have Hudson automatically apply tests for new builds of Holland.

mysqldump --dry-run reporting 'bad file descriptor'

2010-07-09 04:06:11,243 [INFO] Starting backup[default/20100709_040611] via plugin mysqldump
2010-07-09 04:06:11,273 [INFO] Running in dry-run mode.
...
2010-07-09 04:06:11,298 [INFO] Executing: /usr/bin/mysqldump --defaults-file=/var/spool/holland/default/20100709_040611/my.cnf --lock-tables mysql
2010-07-09 04:06:11,300 [ERROR] Bad file descriptor

This appears to be due to a simple typo in how the output stream is opened and checked for errors on stderr in --dry-run mode.

Integrate Violations/pylint into Hudson

Pylint support in hudson would be a nice feature.

http://wiki.hudson-ci.org/display/HUDSON/Violations
http://www.logilab.org/857

Old blog post describing this:
http://redsolo.blogspot.com/2007/11/hudson-embraces-python.html

Request to change 'Starting backup run' line as it confuses github

So, posting output from holland potentially breaks in github issues because of the following lines:

---> Starting backup run <---

As you can see, even the trailing "<---" doesn't show up, becasue it starts an HTML comment... so anything after that line doesn't show up.

--help does not work with a busted config file

Looks like Holland does not allow using --help if the configuration file does not exist:

[tim@holland holland-virtualenv]$ bin/holland --help
Failed to load holland config: Config file not found: "/etc/holland/holland.conf".

This could be confusing for users who may have installed Holland improperly. Oddly enough, --version actually does work:

[tim@holland holland-virtualenv]$ bin/holland --version

Holland Backup v0.9.9
Copyright (c) 2008-2010 Rackspace US, Inc.
More info available at http://hollandbackup.org

[[[[[[[]]]]]]] [[[[[[[]]]]]]]
[[[[[[[]]]]]]]       [[[[[[[]]]]]]]
[[[[[[[]]]]]]] [[[[[[[]]]]]]]
[[[[[[[]]]]]]] [[[[[[[]]]]]]]

lost+found and mysql4.1

If the mysql datadir is on the root of a mountpoint such that there is a lost+found directory, a show table status against this directory will fail. However, mysqldump will silently ignore any unreadable databases. When using the information_schema, unreadable directories are silently skipped (although such a directory will show up in SHOW DATABASES).

For mysqldump compatibility we must ignore unreadable databases (sqlstate = 1018).

backups-to-keep not working as advertised

In Holland 1.0 (trunk) setting 'backups-to-keep' to 1 is actually keeping two backupsets:

http://gist.github.com/460267

The results in my case are two backup sets even though there should be one (as the backups were both successful).

holland backup --dry-run generating empty backup

Regression from v1.0.0.

mysql:client -> port not quoted properly in some cases

Regression from v1.0, introduced when fixing a few other quoting issues. :(

[mysql-lvm] tar archive log should be saved to a separate file

The backup process for mysql-lvm simply runs a (g)tar against the snapshot datadir with --verbose --totals. This logs all the files backed up and the final throughput, which is useful information. However, this data is logged to the general holland.log and can produce quite a lot of data for deployments with a large number of tables.

I propose the tar information output be logged to $backupset/archive.log and saved along with the backupset. This reduces the amount of data being logged to holland.log, this data can be logged more efficiently than going through the general logging framework and being next to the backupset provides a useful tool for restores.

holland/lvm does not parse snapshot-size the same in 1.0 and 0.9.8

In 0.9.8 a snapshot-size without units was parsed as megabytes, but in 1.0 there seems to be a regression where this is now parsed as bytes.

This causes a problem if someone relied on the implicit MB behavior where "snapshot-size = 20000" is now interpreted as ~20K rather than ~20G.

Allow specifying a MySQL password from a file

It would be nice if we could specify a password file to read a password from. This might be maintained by some other system that doesn't understand my.cnf files. I propose adding a 'file:' prefix to the [mysql:client] password field to read the contents of a file and subtitute the password.

This would look like:
[mysql:client]
password = file:/etc/holland/secret.txt

Here, the "file:" prefix will be stripped and the remaining string will be treated as a path which will be read and the entire filecontents treated as the password. If the path is relative, it will be evaluated relative to the current working directory.

holland lvm snapshots + mysqldump

Provide a method to perform mysqldump backups off of an LVM snapshot. Ideally this would use the existing mysqldump plugin and provide most, if not all of the functionality in that plugin and simply handoff to the mysqldump plugin after the snapshot is mounted and a bootstrap mysqld configured.

Splitting out schema from data?

One option that might be nice to have in the Holland mysqldump provider is a way to split schema from data. Basically, making two dumps, using the -t and -d options. I found myself running into this problem while trying to convert from InnoDB to PBXT; or trying to convert schemas to the more simpler Drizzle format.

This can be done manually using multiple backup-sets, but having this as an option for a single set would be nice.

lvm snapshot fails on rhel/centos4

Traceback (most recent call last):
  File "/usr/lib/python2.3/site-packages/holland/core/command/command.py", line 201, in dispatch
    return self.run(self.optparser.prog, opts, *args)
  File "/usr/lib/python2.3/site-packages/holland/commands/backup.py", line 95, in run
    runner.backup(name, config, opts.dry_run)
  File "/usr/lib/python2.3/site-packages/holland/core/backup/base.py", line 106, in backup
    plugin.backup()
  File "/usr/lib/python2.3/site-packages/holland/backup/mysql_lvm/plugin/raw/plugin.py", line 140, in backup
    snapshot.start(volume)
  File "/usr/lib/python2.3/site-packages/holland/lib/lvm/snapshot.py", line 32, in start
    self._apply_callbacks('initialize', self)
  File "/usr/lib/python2.3/site-packages/holland/lib/lvm/snapshot.py", line 145, in _apply_callbacks
    callback_list.sort(reverse=True)
TypeError: sort() takes no keyword arguments

python-2.3 doesn't support sort(reverse=bool).

scripts/build_rpms.py option parsing broken (via python 2.6 keyword 'with')

'with' is a keyword in Python 2.6 and can't be used this way...

$ python ./scripts/build_rpms.py --help
  File "./scripts/build_rpms.py", line 129
    retcode = build_rpms(cli_opts.with)
                                     ^
SyntaxError: invalid syntax

devtools/build_debs.py doesn't use addons/ACTIVE or plugins/ACTIVE properly

The build debs script doesn't use the ACTIVE files to determine which addons/plugins to build.

Add pigz to the list of supported output mechanisms

On modern multi-core machines, this gives much faster compression that gzip, for similar size.

devtools/mkvirtenv.py failure on EL5

]# devtools/mkvirtenv.py
New python executable in /root/holland-test/bin/python
Installing setuptools..............done.
[INFO] Installed holland-core.
[INFO] Installing holland addons
[INFO] Installing holland plugins
[INFO] Installed plugin holland.lib.common
[ERROR] Failed to install plugin holland.lib.mysql
[INFO] Installed plugin holland.backup.example
[INFO] Installed plugin holland.backup.maatkit
[INFO] Installed plugin holland.backup.mysqldump
[INFO] Installed plugin holland.backup.mysqlhotcopy
[INFO] Installed plugin holland.backup.mysql-lvm
Traceback (most recent call last):
 File "devtools/mkvirtenv.py", line 186, in ?
   sys.exit(main())
 File "devtools/mkvirtenv.py", line 179, in main
   install_configs(home_dir)
 File "devtools/mkvirtenv.py", line 151, in install_configs
   join(env_root, 'etc', 'holland'))
 File "/usr/lib64/python2.4/shutil.py", line 111, in copytree
   os.mkdir(dst)
OSError: [Errno 2] No such file or directory: '/root/holland-test/etc/holland'
]# 

Remove --databases (USE) flag from mysqldump (in certain cases)

Based upon our discussion and feedback, we need to move away from using the --databases option (and thus having "USE db" in the resulting backup) from the mysqldump provider under cases where only a single database is being backed up.

This would be when a backup-set only contains a single database as well as when using the 'file-per-database' option.

I propose we just rip it out for now and can talk about reintroducing this functionality by a configuration option at a later date.

holland-mysqldump should be able to skip metadata disk space estimates

I propose adding a estimate-method = plugin|const: attribute to mysqldump to entirely skip the potentially costly metadata check.

This allows an admin to specify a rough guesstimate that the plugin will always return rather than calculating the guesstimate from metadata (via show table status or information_schema). For those instances with thousands (or millions) of tables/databases this can make the pain of disk space checks much less costly.

const:size will be evaluated accordingly to standard MySQL syntax. "estimate-method = const:4G" means ensure at least 4_1024_1024*1024 bytes are available in $backup_directory prior to performing a backup. With holland 1.0 this will still be subject to the estimate-size-factor fudging.

Update comment about snapshot size calculation

This is to suggest that the following comment in mysql-lvm.conf be updated to reflect the recently added 15 GB limit for automatically calculated snapshot sizes:

The size of the snapshot itself. By default it is 20% of the size of

the MySQL LVM mount or the remaining free-space in the Volume Group

(if there is less than 20% available).

If snapshot-size is defined, the number represents the size of the

snapshot in megabytes.

holland locking behavior needs to be more configurable

holland backup presently acquires an exclusive lock on /etc/holland/holland.conf (as an easy place to flock). This can be disabled on the command-line with --no-lock, but this is not presently exposed through the configuration file.

This probably should be exposed through a more generic mechanism in the future. For now, I think just adding a lock-file parameter in holland.conf (or perhaps, better, the backupset [holland:backup] section) will make this sufficiently configurable.

mysqldump bin-log-position misbehavior

The mysqldump backup plugin is not recording binary log status to the backup.conf file in all cases. This looks to be a regression from a couple typos in 0.9.9.

password = 'file:...' does not handle symbols

Using special symbols in a password file appears to prevent Holland from being able to login to MySQL properly:

http://gist.github.com/468108

Not sure if we discussed this as a possibility but I admittedly did test the file function, but not with crazy characters :/

I think this is a minor to medium issue since, in those cases, we can just use the /root/.my.cnf instead of a password file. That may break cases where, say, a customer is running a well known admin panel and changes their password, but that would just mean the /root/.my.cnf needs to be updated manually.

Holland does not allow --max-allowed-packet when used with additional-options

I am trying to set max_allowed_packet to 256M due to mysqldump, however I am running into issues. http://gist.github.com/392779

This affects both 0.9.8 and 0.9.9. 0.9.9 gives the odd error noted in the gist, but 0.9.8 simply says:

2010-05-06 17:24:16,812 [ERROR] Invalid mysqldump option: '--max-allowed-packet=256M'

Putting this in the /root/.my.cnf does not resolve the issue either.

MySQL-python dependency issue with CentOS 5 in 0.9.9 (master)

Trying to test some new Holland features but ran into an issue that did not seem to previously exist where Holland cannot find my MySQL-python package:

[root@holland noarch]# rpm -Uvh *rpm
Preparing...                ########################################### [100%]
   1:holland                ########################################### [ 14%]
   2:holland-common         ########################################### [ 29%]
   3:holland-example        ########################################### [ 43%]
   4:holland-maatkit        ########################################### [ 57%]
   5:holland-mysqldump      ########################################### [ 71%]
   6:holland-mysqlhotcopy   ########################################### [ 86%]
   7:holland-mysqllvm       ########################################### [100%]
[root@holland noarch]# holland bk
Holland 0.9.9 started with pid 12573
-----> Starting backup run <----
No backupsets defined.  Please specify in /etc/holland/holland.conf or specify a name of a backupset in /etc/holland/backupsets
[root@holland noarch]# holland mk-config mysqldump
Holland 0.9.9 started with pid 12574
Failed to load backup plugin 'mysqldump': Could not find dependency 'MySQL-python'
[root@holland holland]# holland bk mysqldump
Holland 0.9.9 started with pid 12743
-----> Starting backup run <----
Acquired backup lock.
Loading config for backupset mysqldump
Loading plugin mysqldump
Failed to load plugin mysqldump: Could not find dependency 'MySQL-python'
Backup 'mysqldump' failed: Could not find dependency 'MySQL-python'
One or more backupsets failed to run successfully
This backup run of 1 backupset took 0.01 seconds
-----> Ending backup run <----
[root@holland holland]# rpm -qa | grep -i mysql-python
MySQL-python-1.2.1-1
[root@holland holland]# cat /etc/redhat-release 
CentOS release 5.4 (Final)
[root@holland holland]# yum upgrade
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* addons: linux.mirrors.es.net
* base: mirror.team-cymru.org
* epel: linux.mirrors.es.net
* extras: linux.mirrors.es.net
* updates: mirrors.cmich.edu
Setting up Upgrade Process
No Packages marked for Update

(Sorry if that's ugly...I fail at Markdown apparently...)

[mysql-lvm] innodb-recovery's mysqld bootstrap should define --log-error

If a my.cnf defines --log-error in the [mysqld] section, mysql-lvm's bootstrap process will write to this error log (usually with a InnoDB recovery and shutdown message). So it is essentially "leaking" into the production error log and causing confusion. FWIW, this is repeating mylvmbackup's error (fixed in their 0.12 release).

I recommend innodb-recovery writes its error log to $backupdir/innodb_recovery.log, so the log is fully saved with the backupset. Otherwise, even in a "good" case (where log-error is not defined), we are only logging the mysqld stderr to the global holland.log which may be long gone by the time one wants to reference this information - for instance to recover binlog position/offset.

build_debs.py fails with cryptic error

The DEB build-script is failing to build the Debian packages on the latest snapshot (2010-04-14) tarball with cryptic errors:

root@misc:~/holland-20100414.041271222548# lsb_release -r
Release:    9.10
root@misc:~/holland-20100414.041271222548# devtools/build_debs.py 
0
Traceback (most recent call last):
  File "devtools/build_debs.py", line 118, in 
    sys.exit(main())
  File "devtools/build_debs.py", line 115, in main
    cleanup_tree()
  File "devtools/build_debs.py", line 88, in cleanup_tree
    subprocess.call(args)
  File "/usr/lib/python2.6/subprocess.py", line 470, in call
    return Popen(*popenargs, **kwargs).wait()
  File "/usr/lib/python2.6/subprocess.py", line 621, in __init__
    errread, errwrite)
  File "/usr/lib/python2.6/subprocess.py", line 1126, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

--dry-run bombs if the backup job failed

Using the sqlite plugin as an example:

$ holland bk --dry-run
Holland 0.9.9 started with pid 19874
Acquired backup lock.
Loading config for backupset sqlite
Loading plugin sqlite
Prepared backup spool env/var/spool/holland/sqlite/20100517_154545
Validating config: sqlite
Checking that SQLite backups can run.
Unable to open database "/Users/wdierkes/johnny": unable to open database file

Estimated Backup Size: 4.00KB
Starting backup[sqlite/20100517_154545] via plugin sqlite
SQLite binary is [/usr/bin/sqlite3]
Backing up SQLite database at [/Users/wdierkes/sqlite_database1.db] (dry run)
Skipping invalid SQLite database at [/Users/wdierkes/johnny]
Backing up SQLite database at [/Users/wdierkes/sqlite_database2.db] (dry run)
Dry-run completed in 0.02 seconds
Purging this backup (sqlite/20100517_154545) due to failure
Unexpected exception caught.  This is probably a bug.  Please report to the holland development team.
Traceback (most recent call last):
  File "/Users/wdierkes/devel/holland/holland/commands/backup.py", line 101, in run_backup
    backup(jobname, dry_run)
  File "/Users/wdierkes/devel/holland/holland/core/backup.py", line 196, in backup
    backup_job.purge()
  File "/Users/wdierkes/devel/holland/holland/core/spool.py", line 227, in purge
    shutil.rmtree(self.path)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/shutil.py", line 208, in rmtree
    onerror(os.listdir, path, sys.exc_info())
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/shutil.py", line 206, in rmtree
    names = os.listdir(path)
OSError: [Errno 2] No such file or directory: 'env/var/spool/holland/sqlite/20100517_154545'
One or more backupsets failed to run successfully
This backup run of 1 backupset took 0.05 seconds

Feature Request: File backup plugin

Holland has some sophisticated MySQL backup plugins but in many cases just having a way to archive simple files with configurable compression would be a useful feature and work well in tandem with the database backups.

mysql-lvm provider not built on Ubuntu 10.04

Spoke a bit too soon about not running into issues on 10.04:

tim@lucid:~/git$ ls -1 *deb
holland_0.9.9-local-201018180618_all.deb
holland-common_0.9.9-local-201018180618_all.deb
holland-example_0.9.9-local-201018180618_all.deb
holland-maatkit_0.9.9-local-201018180618_all.deb
holland-mysqlcmds_0.9.9-local-201018180618_all.deb
holland-mysqldump_0.9.9-local-201018180618_all.deb
holland-mysqlhotcopy_0.9.9-local-201018180618_all.deb

For some reason, MySQL-LVM is not building, though I suspect due to something fairly simple since the build process shows no errors.

tim@lucid:~/git/holland$ scripts/build_debs.py 
0
 dpkg-buildpackage -rfakeroot -D -us -uc
dpkg-buildpackage: set CFLAGS to default value: -g -O2
dpkg-buildpackage: set CPPFLAGS to default value: 
dpkg-buildpackage: set LDFLAGS to default value: -Wl,-Bsymbolic-functions
dpkg-buildpackage: set FFLAGS to default value: -g -O2
dpkg-buildpackage: set CXXFLAGS to default value: -g -O2
dpkg-buildpackage: source package holland
dpkg-buildpackage: source version 0.9.9-local-201018180618
dpkg-buildpackage: source changed by Unknown 
dpkg-buildpackage: host architecture amd64
 fakeroot debian/rules clean
dh_testdir
dh_testroot
rm -f build-stamp configure-stamp
cd /home/tim/git/holland ; \
        python setup.py -q clean; \
    done
dh_clean 
 dpkg-source -b holland
dpkg-source: warning: source directory 'holland' is not - 'holland-0.9.9-local'
dpkg-source: info: using source format `1.0'
dpkg-source: info: building holland in holland_0.9.9-local-201018180618.tar.gz
dpkg-source: info: building holland in holland_0.9.9-local-201018180618.dsc
 debian/rules build
dh_testdir
touch configure-stamp
dh_testdir
touch build-stamp
 fakeroot debian/rules binary
dh_testdir
dh_testroot
dh_clean -k
dh_installdirs
cd /home/tim/git/holland ; \
        python setup.py -q install \
            --no-compile \
            --prefix=/usr \
            --root=/home/tim/git/holland/debian/tmp \
            --install-scripts=/usr/sbin \
            --single-version-externally-managed \
            --install-layout=deb; \
    done
Skipping installation of /home/tim/git/holland/debian/tmp/usr/lib/python2.6/dist-packages/holland/__init__.py (namespace package)
Skipping installation of /home/tim/git/holland/debian/tmp/usr/lib/python2.6/dist-packages/holland/lib/__init__.py (namespace package)
Skipping installation of /home/tim/git/holland/debian/tmp/usr/lib/python2.6/dist-packages/holland/commands/__init__.py (namespace package)
Skipping installation of /home/tim/git/holland/debian/tmp/usr/lib/python2.6/dist-packages/holland/backup/__init__.py (namespace package)
mkdir -p /home/tim/git/holland/debian/tmp/etc/holland
mkdir -p /home/tim/git/holland/debian/tmp/etc/holland/backupsets
mkdir -p /home/tim/git/holland/debian/tmp/etc/holland/providers
dh_testdir
dh_testroot
dh_installchangelogs
dh_installdocs
dh_installexamples
dh_install
dh_movefiles
dh_installman
dh_pysupport
dh_link
dh_strip
dh_compress
dh_fixperms
dh_installdeb
dh_gencontrol
dpkg-gencontrol: warning: unused substitution variable ${python:Versions}
dh_md5sums
dh_builddeb
dpkg-deb: building package `holland' in `../holland_0.9.9-local-201018180618_all.deb'.
dpkg-deb: building package `holland-common' in `../holland-common_0.9.9-local-201018180618_all.deb'.
dpkg-deb: building package `holland-mysqldump' in `../holland-mysqldump_0.9.9-local-201018180618_all.deb'.
dpkg-deb: building package `holland-example' in `../holland-example_0.9.9-local-201018180618_all.deb'.
dpkg-deb: building package `holland-maatkit' in `../holland-maatkit_0.9.9-local-201018180618_all.deb'.
dpkg-deb: building package `holland-mysqlhotcopy' in `../holland-mysqlhotcopy_0.9.9-local-201018180618_all.deb'.
dpkg-deb: building package `holland-mysqlcmds' in `../holland-mysqlcmds_0.9.9-local-201018180618_all.deb'.
 dpkg-genchanges  >../holland_0.9.9-local-201018180618_amd64.changes
dpkg-genchanges: including full source code in upload
dpkg-buildpackage: full upload; Debian-native package (full source is included)
Now running lintian...
W: holland source: native-package-with-dash-version
W: holland source: diff-contains-git-control-dir .git
W: holland source: debhelper-but-no-misc-depends holland
W: holland source: debhelper-but-no-misc-depends holland-common
W: holland source: debhelper-but-no-misc-depends holland-maatkit
W: holland source: debhelper-but-no-misc-depends holland-example
W: holland source: debhelper-but-no-misc-depends holland-mysqldump
W: holland source: debhelper-but-no-misc-depends holland-mysqlhotcopy
W: holland source: debhelper-but-no-misc-depends holland-mysqlcmds
W: holland source: missing-debian-source-format
E: holland source: clean-should-be-satisfied-by-build-depends python | python-dev | python-all | python-all-dev | python2.4 | python2.4-dev | python2.5 | python2.5-dev | python2.6 | python2.6-dev
W: holland source: out-of-date-standards-version 3.8.0 (current is 3.8.4)
W: holland-mysqlcmds: empty-binary-package
Finished running lintian.
dh_testdir
dh_testroot
rm -f build-stamp configure-stamp
cd /home/tim/git/holland ; \
        python setup.py -q clean; \
    done
dh_clean 

Ughm... No way to assign developers to issues?

WTF?

stop-slave+bin-log-position could cause an error if the slave was not configured.

This seems to have been an issue for a while, but no one seems to use this functionality.

Add make_release script to generate a source distribution

VIEWs break --single-transaction on MySQL 5.0, not 5.1 (1.0.0 regression)

Creating a VIEW in a transactional DB (ie one using all InnoDB) appears to cause Holland to not properly detect when it is safe to use --single-transaction. This only affects MySQL 5.0 however (or specifically 5.0.77 and 5.0.91). MySQL 5.1 (5.1.47 tested) seems to work without issue.

Simple test case:

http://gist.github.com/468157

Licensing Audit

Need to audit holland-core and all plugins to ensure each have an appropriate LICENSE file, and ensure the READMEs reference licensing properly.

build_rpms.py script bailing on git

When downloading the tarball of holland 1.0.2, I attempt to run the build_rpms.py script to create the packages, but git errors out:

[root@hollandcent holland-1.0.2]# python scripts/build_rpms.py
git archive --prefix=holland-1.0.2/ HEAD > /root/holland-buildroot/SOURCES/holland-1.0.2.tar.gz
fatal: Not a git repository

Regressions with Views in 0.9.9

This is a continuation of http://github.com/holland-backup/holland/issues/closed#issue/11 as the original issue has been resolved.

The problem I now have is the following:

Tested and I think this bug itself may be resolved. I was able to generate a new mysqldump configuration file using mk-config (which previously did not work), but now I get this when trying to run a backup:

[root@holland ~]# holland bk default
Holland 0.9.9 started with pid 6907
-----> Starting backup run <----
Acquired backup lock.
Loading config for backupset default
Loading plugin mysqldump
Prepared backup spool /var/spool/holland/default/20100426_091228
Loading /root/.my.cnf [/root/.my.cnf]
Loading ~/.my.cnf [/root/.my.cnf]
Unexpected exception caught.  This is probably a bug.  Please report to the holland development team.
Traceback (most recent call last):
  File "/usr/lib/python2.4/site-packages/holland/commands/backup.py", line 101, in run_backup
    backup(jobname, dry_run)
  File "/usr/lib/python2.4/site-packages/holland/core/backup.py", line 126, in backup
    estimated_size = plugin.estimate_backup_size()
  File "/usr/lib/python2.4/site-packages/holland/backup/mysqldump/plugin.py", line 93, in estimate_backup_size
    self.schema.refresh(db_iter=db_iter, tbl_iter=tbl_iter)
  File "/usr/lib/python2.4/site-packages/holland/lib/mysql/schema/base.py", line 112, in refresh
    for table in tbl_iter(database.name):
  File "/usr/lib/python2.4/site-packages/holland/lib/mysql/schema/base.py", line 249, in __call__
    for metadata in self.client.show_table_metadata(database):
  File "/usr/lib/python2.4/site-packages/holland/lib/mysql/client/base.py", line 174, in show_table_metadata
    return self._show_table_metadata50(database)
  File "/usr/lib/python2.4/site-packages/holland/lib/mysql/client/base.py", line 124, in _show_table_metadata50
    row['is_transactional'] = row['engine'].lower() in ['innodb']
AttributeError: 'NoneType' object has no attribute 'lower'
One or more backupsets failed to run successfully
This backup run of 1 backupset took 0.82 seconds
-----> Ending backup run <----

A fix for this has been pushed, but now I end up with:

[root@holland noarch]# holland bk default
Holland 0.9.9 started with pid 11249
-----> Starting backup run <----
Acquired backup lock.
Loading config for backupset default
Loading plugin mysqldump
Prepared backup spool /var/spool/holland/default/20100426_102107
Loading /root/.my.cnf [/root/.my.cnf]
Loading ~/.my.cnf [/root/.my.cnf]
Unexpected exception caught.  This is probably a bug.  Please report to the holland development team.
Traceback (most recent call last):
  File "/usr/lib/python2.4/site-packages/holland/commands/backup.py", line 101, in run_backup
    backup(jobname, dry_run)
  File "/usr/lib/python2.4/site-packages/holland/core/backup.py", line 126, in backup
    estimated_size = plugin.estimate_backup_size()
  File "/usr/lib/python2.4/site-packages/holland/backup/mysqldump/plugin.py", line 94, in estimate_backup_size
    return sum([db.size for db in self.schema.databases])
  File "/usr/lib/python2.4/site-packages/holland/lib/mysql/schema/base.py", line 163, in size
    return sum([table.size for table in self.tables if not table.excluded])
  File "/usr/lib/python2.4/site-packages/holland/lib/mysql/schema/base.py", line 200, in size
    return self.data_size + self.index_size
TypeError: unsupported operand type(s) for +: 'NoneType' and 'NoneType'
One or more backupsets failed to run successfully
This backup run of 1 backupset took 0.75 seconds
-----> Ending backup run <----

These appear to be possibly related since they both involve the estimate_backup_size function. Andy mentioned the first error was caused by VIEWs so I suspect this may be the case here as ewll.

Test DBs are:

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema | 
| employees          | 
| menagerie          | 
| mysql              | 
| sakila             | 
| test               | 
| world              | 
+--------------------+
7 rows in set (0.00 sec)

Configurable Backup Set Purging Policy

Logging this based upon previous conversations just to we have it documented. Currently the Holland purge policy is conservative - backups are not purged until the next successful backup finishes. In some cases, this is not ideal, such as with large databases and when another backup method (such as tape-backup) is being used to copy the backups elsewhere. Under these cases, disk space can fill up if Holland runs into too many failures since both full and partial backups can be left on the server.

So, having an option to make Holland more aggressive about purging backups would be ideal. Implementation is still something to be discussed.

Standardize plugins via a HollandBackupPlugin() base class

Suggest something similar to the following that backup plugins should subclass.

class HollandBackupPlugin(object):
    def __init__(self, name, config, target_directory, dry_run=False):
        self.name = name
        self.config = config
        self.target_directory = target_directory
        self.dry_run = dry_run

    def info(self):
        """Returns [str] information about the plugin. [optional]"""
        return "No information available."

    def pre(self):
        """Called before check() and backup()."""
        pass

    def check(self):
        """Perform any checks to verify a backup can run. [optional]"""
        pass

    def estimate_backup_size(self):
        """Estimate disk space [in bytes] required to perform backup. [required]"""
        raise HollandRuntimeError, "estimate_backup_size() must be subclassed."

    def backup(self):   
        """Perform backup of data. [required]"""
        raise HollandRuntimeError, "backup() must be subclassed."

    def post(self):
        """Called after backup() is run. [optional]"""
        pass

build_debs.py fails with cryptic error

The DEB build-script is failing to build the Debian packages on the latest snapshot (2010-04-14) tarball with cryptic errors:

{{{
root@misc:/holland-20100414.041271222548# lsb_release -r
Release: 9.10
root@misc:/holland-20100414.041271222548# devtools/build_debs.py
0
Traceback (most recent call last):
File "devtools/build_debs.py", line 118, in
sys.exit(main())
File "devtools/build_debs.py", line 115, in main
cleanup_tree()
File "devtools/build_debs.py", line 88, in cleanup_tree
subprocess.call(args)
File "/usr/lib/python2.6/subprocess.py", line 470, in call
return Popen(_popenargs, *_kwargs).wait()
File "/usr/lib/python2.6/subprocess.py", line 621, in init
errread, errwrite)
File "/usr/lib/python2.6/subprocess.py", line 1126, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
}}}

single-quoted values in my.cnf are not always parsed correctly

# cat ~/.my.cnf 
[client]
password = 'crazysh!t$%&#&%^##%('

holland is failing to unquote the single-quotes correctly. With double-quotes this works without issue. This appears to have been a problem in 0.9.8 as well.

holland mysql-lvm should also skip-relay-log-index

When a relay-log-index is specified but relay-log is not, MySQL will generate a bad '0.00001' relay log file. This can temporarily break replication.

This only affects cases where a relay-log-index is explicitly specified (this is not necessary on any recent version of MySQL > 5.0.51). Holland is making a mistake by specifying skip-relay-log but not also adding skip-relay-log-index.

This affects both MySQL 5.0 and 5.1 (and probably 4.1), and only when using the innodb-recovery mechanism in mysql-lvm when an explicit relay-log-index is specified in the global /etc/[mysql/]my.cnf.