Comments (2)
Have the same issue; using compression_format: tar
helps for me since clickhouse data is already highly compressed you don't really need to compress it. Makes it 3 times as fast for me;
from clickhouse-backup.
Hi, @Slind14!
-
Not everyone have databases of several terabytes. My database size is 220GiB, full backup is uploaded for 2h47m and increments are uploaded for ~6m. v0.4.1 has good performance improvements such as non-blocking pipes and multi-threading gzip support that reduces uploading of full backup of my database to 1h10m and increments to ~2m (after switching from lz4 to gzip).
-
Upload to s3 is optional feature protecting you from hardware failures. If you want to defend yourself only from destructive operations like DROP DATABASE or ALTER TABLE you may just create periodical backups with
clickhouse-backup create
keep it local and not put to cloud. -
If you have large database and want to be protected from hardware or whole DC failure you may create instance of ClickHouse on another DC and have it fresh by clickhouse-copier combined with periodical backups that will be stored locally for protection from destructive operations.
from clickhouse-backup.
Related Issues (20)
- implements `--partitions=db.table:part_name1,part_name2` and `--partitions=db2.table2:*` to allow more flexible backup logic HOT 1
- create system.backup_version, add version to log, add GET /backup/version endpoint
- Implements `X/Y tables` logging for `done` logging HOT 1
- How to backup replica clickhouse cluster correctly ? HOT 1
- `acccess` and `configs` download should use archieve extensions the same manner as main data, for example allow download zstd instead of .tar
- API server should restart if `watch` command fails
- Unable to clean remote broken (can't stat metadata.json) HOT 6
- Multipart Upload to Minio Fails in Newest Minio Release HOT 1
- Permission denied while doing restore_remote if backup is also present locally HOT 10
- Authentication Failed When Configuring Automatic Backup HOT 12
- implements embedded BACKUP/RESTORE ON CLUSTER approach
- not supporter users with dot in name HOT 6
- URL is getting malformed while running restore_remote command with azure_blob_store HOT 4
- create / restore stuck when CLICKHOUSE_MAX_CONNECTIONS=0
- Apply rate limiting as wrapper around Reader \ Writer interface instead of calculate it after upload / download finished
- Implementation --restore-table-mapping HOT 2
- `$` character in database and table name can't make backup
- validation config failures for API server after apply --env in one command
- Replcace AsyncStatus.commands []ActionRow to combination map + slice to allow store only last XXX action items
- Add support for dynamically created disks
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from clickhouse-backup.