Comments (3)
Dump URL/data format (WIP)
Backup file : <prefix>
/[bchn
|bu
|fulcrum
|slpdb
]/<blockheight>
/<fname>
Manifest : <prefix>
/[bchn
|bu
|fulcrum
|slpdb
]/<blockheight>
/manifest.json
Signature : <prefix>
/[bchn
|bu
|fulcrum
|slpdb
]/<blockheight>
/sig.json
Example:
Lets say our storage root (prefix) is https://example.com/devkit-dumps/
, and we have a dump of bitcoin unlimited at block 1,500,000
- This dump would be available at
https://example.com/devkit-dumps/bchu/1500000/bu.tar.gz
- Manifest file at
https://example.com/devkit-dumps/bchu/1500000/manifest.json
- Signature file at
https://example.com/devkit-dumps/bchu/1500000/sig.json
Dump format : Depends on the application, the devkit needs to know how to deal with it
manifest.json proposed format
{
"height": 1500000, //Last known block included in the dump
"filename": "bu.tar.gz", //Filename of the actual dump
"sha256sum" : "c98c24b677eff44860afea6f493bbaec5bb1c4cbb209c6fc2bbb47f66ff2ad31", //sha256sum of bu.tar.gz
"version": "1.9.2", //Version used to make this dump. The users version must be same or newer
"devkit_version": "1.2.3", //The version of devkit this dump supports. Restore script would take this into account to determine how to process this dump
"time" : "2021-12-20T11:38:39.730026" //Timestamp when this dump was created
}
sig.json proposed format
{
"signer": "qp3wjpa3tjlj042z2wv7hahsldgwhwy0rq9sywjpyy", //The key used to sign the manifest, the user needs to trust this signer.
"signature": "IBOvIfsAs/da1e36W8kw1cQOPqPVXCW5zJgNQ5kI8m57FycZXdeFmeyoIqJSREzE4W7vfDmdmPk0HokuJPvgPPE=" //Entire manifest.json signed by signer
}
User flow when restoring (lets say they want to do bitcoin unlimited)
- Do directory listing of
https://example.com/devkit-dumps/bchu/
- Find directory name with highest block number
- Fetch manifest.json and sig.json
- Validate that signer is someone user trusts, and check if manifest.json is signed correctly.
- Check if
version
is compatible. If not find an alternate block number and goto step 3. - Download the file referred in
filename
. - Validate sha256sum of downloaded file is same as mentioned in the manifest
- Restore Bitcoin Unlimited node, wait for it to startup and stabilize (
initialblockdownload
turns to false)
Note: We either need directory listing enabled, or some application server that processes some logic to point the user to the correct directory.
from bch-devsuite.
Updated work-in-progress example for a smartbch restore
- Ensure BCH node is synced up
- Fetch index
https://devkit-dumps.actorforth.org/mainnet/smartbch/
- Observe newest directory is
2808567
. Directory names are approximate last-known block number at the time of making the dump. - Fetch manifest from
https://devkit-dumps.actorforth.org/mainnet/smartbch/2808567/manifest.json
- check if constraints are satisfied, i.e. smartbch >= 0.3.5 and dump made for devsuite 1.1.0 is compatible with currently running devsuite or not. If not then retry from step 2 using an older dump.
5. fetch signature filehttps://devkit-dumps.actorforth.org/mainnet/smartbch/2808567/sig.json
(currently its a placeholder)
6. Compare if signature matches(check previous comment) and the signer is trusted. - Check if /tmp/smartbch.tar.gz already exists and sha256sum matches manifest. if matches GOTO 10
- ^ If not then download
https://devkit-dumps.actorforth.org/mainnet/smartbch/2808567/smartbch.tar.gz
. Its better to use a download accelerator (likeaxel
) instead of wget,axel
allows for multi-connection downloads, and allows download from several mirrors at once. - check sha256sum matches, if not its an error
- proceed to untaring it as mentioned in #50
from bch-devsuite.
Restore step for slpdb is similar.
Find the file https://devkit-dumps.actorforth.org/mainnet/slpdb/710040/slpdb.mongo.gz
then run
mongorestore --drop --authenticationDatabase admin --archive=/tmp/slpdb.mongo.gz --gzip mongodb://mongouser:mongopass@mongodb-slpdb:27017
This needs to run somewhere (ideally in a container) that has access to the dump file and can connect to mongodb
from bch-devsuite.
Related Issues (20)
- Create a landing page for DevSuite
- Proper docker image tag management
- Update the README's structure
- Support SmartBCH mainnet
- Make setup script option to deploy both rest.bitcoin.com and bch-api at the same time
- SmartBCH support Testnet
- Support remote BCH RPC
- Potential security vulnerabilities in docs dependency HOT 1
- Expose smartbch p2p port to external network when running testnet or mainnet
- ./clean does not cleanup smartbchd directory
- Dump and restore smartbch
- Bitcoin Cash Node regtest Cli unable to update balance when transaction in mempool
- Upgrade smartBCH to v0.4.2
- Do not reindex BCHN each time
- Perform a full successful sync of smartbch(mainnet) and provide step-by-step instructions
- Urgent: Upgrade Bitcoincash node and Bitcoin Unlimited nodes
- smartBCH does not sync anymore
- Cant init without smartbch selected
- Upgrade bitcoin unlimited to 2.0.0.0
- Upgrade bitcoincash node to v26.0.0
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bch-devsuite.