Comments (1)
You're correct that the backups are more resilient because buttersink does occasional full backups, and tries to reduce the length of the chain of differences required to restore any particular snapshot.
The algorithm for this is heuristic. Buttersink uses some factors for the cost of transferring the data to S3, of storing the data in S3, and of the likelihood of failure based on the amount of data and number of files that could possible be corrupted. It then attempts to find the "differences" that create the most resilient backups for the least cost. Because it is a heuristic algorithm, the exact scheme differs based on the data and the current set of differences in S3. It would be hard to predict the results in advance.
You have probably notices that the differences are labeled with a "to...from" scheme, so you can see what differences are dependent on what other differences.
Though I have not used Glacier personally, I did think about it during the design of buttersink. My vision of how to use it would be to set up an AWS policy to rotate your S3 differences into Glacier. As long as you leave the ".bs" alone, buttersink should be able to continue backing-up your snapshots. When you restore, you may need to request that AWS transfer a file from glacier back to S3.
Another possibility would be to backup to two separate buckets -- one for normal S3 and another where you have Glacier configured. Then any restores from the plain S3 bucket would never need a transfer from Glacier.
As you can see, buttersink hasn't been tested with Glacier much, so if you try it, I'd be interested in your experiences with either of the two methods above.
In answer to your question about btrfs receive
, yes, the files are pure btrfs send data, and so should work normally with btrfs receive
. FYI, buttersink does some extra things to enable things like restoring from a combination of local snapshots and remote "send" backup data, which btrfs does not support.
Thanks!
from buttersink.
Related Issues (20)
- Remove destination folder if it exists HOT 4
- Migrate to python3
- Refactoring: automated unit testing HOT 2
- Btrfs <-> Btrfs transfer optimization and multiple sources HOT 7
- Store.listVolumes() unreliable when subvolid=5 mounted somewhere else
- buttersink can't rely on Received UUID uniqueness HOT 9
- Buttersink causes lock up on Kernel 4.17 HOT 3
- S3 endpoint iterates through untargeted folders HOT 1
- Partial snapshot updates? HOT 7
- empty stream is not considered valid HOT 9
- relative path while 'keeping' - no such file or directory HOT 4
- Detect snapper cleanup timer
- Issue with quota HOT 4
- buttersink fails transferring large snapshot HOT 3
- error File exists HOT 2
- Buttersink measures unrelated snapshots when syncing. HOT 1
- .bs files automatic removal HOT 2
- S3 /trash moves fail > 5GB
- What is the SSH syntax to use a different port? HOT 1
- S3 compatible storage HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from buttersink.