kraymer / cronicle Goto Github PK
View Code? Open in Web Editor NEWUse cron to rotate files and keep time-spaced copies using symlinks.
License: MIT License
Use cron to rotate files and keep time-spaced copies using symlinks.
License: MIT License
A custom periodicity can't just have the number of archives defined, we need to define the min number of days between two archives too.
This number will be appended as a suffix to the period name.
So if i want to have a folder last
containing the last archive, the config would look like :
/home/johndoe/dumps/mydb-*.dump:
daily: 7
monthly: 2
last|0: 1
last|0
meaning keep just 1 file, no days spacing requirement between archives
Tasks :
cronicle
name on pypi index cf pypi/support#317Hi Kraymer,
Would you please like to enhance this tool for taking a quarterly backup of a file?
So, in a Quarterly folder/dir, only four files will be present. And at end of the year only latest (last quarter) backup file will be moved to Yearly dir.
Every quarter, latest backup file of the last month of each quarter will be moved to the Quarterly folder/dir.
======================================================================
FAIL: test_number_of_archives (test_cronicle.Test)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/mock/mock.py", line 1305, in patched
return func(*args, **keywargs)
File "/home/travis/build/Kraymer/cronicle/test/test_cronicle.py", line 104, in test_number_of_archives
{"foo_2019-12-01_09h", "foo_2020-01-01_09h", "foo_2020-02-01_09h",},
AssertionError: Items in the first set but not the second:
'foo_2019-12-31_09h'
'foo_2020-01-30_09h'
Items in the second set but not the first:
'foo_2020-02-01_09h'
'foo_2020-01-01_09h'
I was trying to see if I could get this to work with some hourly backups that I have. (The TL;DR is: yes, hourly|0: 5
works and gives me 5 hourly backups, aside from the issue in this question.)
Where it breaks is in rotate
, because archives_create_days
returns a Dict split by day, so it only considers one file from any given day.
Is there a reason it needs to work this way? When it's used here:
Line 117 in 060cb58
it's just to skip the n most recent, so if there isn't more than one file per day, it acts the same as a plain directory listing. When it's used here:
Line 83 in 060cb58
it's just to get the last date a file was created, which could be done by calling file_create_day
on the last filename.
Did I miss something else? Would you be open to a PR that changes this? I don't really mind if there's no built-in support for hourly rotations, as long as I can make the custom periodicity work.
Also, this is a cool project, thanks for building this! It's much nicer than the Bash script I was considering writing myself, haha.
If a link is broken (underlying file has been deleted manually) in a periodic dir, program will crash at execution.
Would be better to dleete the link in such a case
exemple: MONTHLY/
created whether a monthly
attribute is present in the config or not
the --dry-run
argument does not seem to work for me. I used your shell script to create those empty sample files and then ran cronicle --remove --dry-run /tmp/x/log/foobar*.txt
.
I expected it to do nothing, but instead it already removed the files. Also the --verbose
flag does not seem to do anything. There is no output at all...
I copied my terminal output below.
(venv) ubuntu@ubuntu:/tmp/x/log/DAILY
$ python --version
Python 3.6.9
(venv) ubuntu@ubuntu:/tmp/x/log/DAILY
$ pip freeze
click==7.1.2
confuse==1.4.0
cronicle==0.3.1
ordereddict==1.1
python-dateutil==2.8.1
PyYAML==5.3.1
six==1.15.0
(venv) ubuntu@ubuntu:/tmp/x/log
$ ls -lt
total 0
-rw-r--r-- 1 ubuntu Domain Users 0 Dez 31 2007 foobar_2007-12-31.txt
-rw-r--r-- 1 ubuntu Domain Users 0 Dez 30 2007 foobar_2007-12-30.txt
-rw-r--r-- 1 ubuntu Domain Users 0 Dez 29 2007 foobar_2007-12-29.txt
-rw-r--r-- 1 ubuntu Domain Users 0 Dez 28 2007 foobar_2007-12-28.txt
-rw-r--r-- 1 ubuntu Domain Users 0 Dez 27 2007 foobar_2007-12-27.txt
-rw-r--r-- 1 ubuntu Domain Users 0 Dez 26 2007 foobar_2007-12-26.txt
-rw-r--r-- 1 ubuntu Domain Users 0 Dez 25 2007 foobar_2007-12-25.txt
-rw-r--r-- 1 ubuntu Domain Users 0 Dez 24 2007 foobar_2007-12-24.txt
-rw-r--r-- 1 ubuntu Domain Users 0 Dez 23 2007 foobar_2007-12-23.txt
-rw-r--r-- 1 ubuntu Domain Users 0 Dez 22 2007 foobar_2007-12-22.txt
-rw-r--r-- 1 ubuntu Domain Users 0 Dez 21 2007 foobar_2007-12-21.txt
-rw-r--r-- 1 ubuntu Domain Users 0 Dez 20 2007 foobar_2007-12-20.txt
-rw-r--r-- 1 ubuntu Domain Users 0 Dez 19 2007 foobar_2007-12-19.txt
-rw-r--r-- 1 ubuntu Domain Users 0 Dez 18 2007 foobar_2007-12-18.txt
-rw-r--r-- 1 ubuntu Domain Users 0 Dez 17 2007 foobar_2007-12-17.txt
-rw-r--r-- 1 ubuntu Domain Users 0 Dez 16 2007 foobar_2007-12-16.txt
-rw-r--r-- 1 ubuntu Domain Users 0 Dez 15 2007 foobar_2007-12-15.txt
-rw-r--r-- 1 ubuntu Domain Users 0 Dez 14 2007 foobar_2007-12-14.txt
-rw-r--r-- 1 ubuntu Domain Users 0 Dez 13 2007 foobar_2007-12-13.txt
-rw-r--r-- 1 ubuntu Domain Users 0 Dez 12 2007 foobar_2007-12-12.txt
-rw-r--r-- 1 ubuntu Domain Users 0 Dez 11 2007 foobar_2007-12-11.txt
-rw-r--r-- 1 ubuntu Domain Users 0 Dez 10 2007 foobar_2007-12-10.txt
-rw-r--r-- 1 ubuntu Domain Users 0 Dez 9 2007 foobar_2007-12-09.txt
-rw-r--r-- 1 ubuntu Domain Users 0 Dez 8 2007 foobar_2007-12-08.txt
-rw-r--r-- 1 ubuntu Domain Users 0 Dez 7 2007 foobar_2007-12-07.txt
-rw-r--r-- 1 ubuntu Domain Users 0 Dez 6 2007 foobar_2007-12-06.txt
-rw-r--r-- 1 ubuntu Domain Users 0 Dez 5 2007 foobar_2007-12-05.txt
-rw-r--r-- 1 ubuntu Domain Users 0 Dez 4 2007 foobar_2007-12-04.txt
-rw-r--r-- 1 ubuntu Domain Users 0 Dez 3 2007 foobar_2007-12-03.txt
-rw-r--r-- 1 ubuntu Domain Users 0 Dez 2 2007 foobar_2007-12-02.txt
-rw-r--r-- 1 ubuntu Domain Users 0 Dez 1 2007 foobar_2007-12-01.txt
(venv) ubuntu@ubuntu:/tmp/x/log
$ cat ~/.config/cronicle/config.yaml
/tmp/x/log/foo*.txt:
daily: 7
monthly: 4
(venv) ubuntu@ubuntu:/tmp/x/log
$ cronicle --remove --dry-run /tmp/x/log/foobar*.txt
(venv) ubuntu@ubuntu:/tmp/x/log
$ ls -lt
total 8
drwxr-xr-x 2 ubuntu Domain Users 4096 Dez 20 16:45 DAILY
drwxr-xr-x 2 ubuntu Domain Users 4096 Dez 20 16:45 MONTHLY
-rw-r--r-- 1 ubuntu Domain Users 0 Dez 31 2007 foobar_2007-12-31.txt
-rw-r--r-- 1 ubuntu Domain Users 0 Dez 30 2007 foobar_2007-12-30.txt
-rw-r--r-- 1 ubuntu Domain Users 0 Dez 29 2007 foobar_2007-12-29.txt
-rw-r--r-- 1 ubuntu Domain Users 0 Dez 28 2007 foobar_2007-12-28.txt
-rw-r--r-- 1 ubuntu Domain Users 0 Dez 27 2007 foobar_2007-12-27.txt
-rw-r--r-- 1 ubuntu Domain Users 0 Dez 26 2007 foobar_2007-12-26.txt
-rw-r--r-- 1 ubuntu Domain Users 0 Dez 25 2007 foobar_2007-12-25.txt
-rw-r--r-- 1 ubuntu Domain Users 0 Dez 1 2007 foobar_2007-12-01.txt
(venv) ubuntu@ubuntu:/tmp/x/log
create a docker image which allows to use this program on any file based backup program
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.