Dshackle is an L7 Blockchain API Load Balancer. It provided a high level aggregated API on top of several underlying upstreams, i.e., blockchain nodes or providers, such as Geth, Parity, Infura, etc. It automatically verifies their availability and the current status of the network, executes commands making sure that the response is consistent and/or sent data successfully broadcasted to the network.
-
Standard Ethereum JSON RPC API, plus advanced gRPC-based API
-
Secure TLS with optional client authentication
-
Blockchain-aware caching in memory and in Redis
-
Routing based on data availability (peers, height, sync status)
-
Data consistency, always gives the most actual state
-
Automatic failover and retry
-
Separate public blockchain nodes from your internal servers
Dshackle allows to build a mesh network of interconnected Dshackle servers for building blockchain based services that needs to have fast, secure, stable and fail-proof access to blockchain APIs.
Dshackle connects to several upstreams via JSON RPC, Websockets, or gRPC protocol. The server verifies if a node ("upstream") is fully synchronized (not in initial sync mode), has enough peers, and its height is not behind other nodes. If upstream lags behind others, lost peers below required, started to resync or went down, then Dshackle temporarily excludes it from requests and returns it when the upstream problem is fixed.
-
❏ Support Bitcoin RPC
-
❏ External logging
-
❏ Access to ERC-20 tokens on asset level
-
❏ Subscription to bitcoind notification over gRPC (instead of ZeroMQ)
-
❏ Prometheus monitoring
-
❏ BIP-32 Pubkey
-
❏ Lightweight sidecar node connector
-
❏ Configurable upstream roles
Create file dshackle.yaml
with the following content:
version: v1
port: 2449
tls:
enabled: false
proxy:
host: 0.0.0.0
port: 8545
routes:
- id: eth
blockchain: ethereum
- id: kovan
blockchain: kovan
cluster:
upstreams:
- id: infura-eth
chain: ethereum
connection:
ethereum:
rpc:
url: "https://mainnet.infura.io/v3/${INFURA_USER}"
ws:
url: "wss://mainnet.infura.io/ws/v3/${INFURA_USER}"
- id: infura-kovan
chain: kovan
connection:
ethereum:
rpc:
url: "https://kovan.infura.io/v3/${INFURA_USER}"
Which sets the following:
-
gRPC access through 0.0.0.0:2449
-
TLS security is disabled (please don’t use in production!)
-
-
JSON RPC access through 0.0.0.0:8545
-
proxy requests to Ethereum and Kovan upstreams
-
request path for Ethereum Mainnet is
/eth
, for Kovan is/kovan
-
i.e. call Mainnet by
POST http://127.0.0.0:8545/eth
with JSON RPC payload
-
-
two upstreams, one for Ethereum Mainnet and another for Kovan Testnet (both upstreams are configured to use Infura endpoint)
-
for Ethereum Mainnet it connects using JSON RPC and Websockets connections, for Kovan only JSON RPC is used
-
Infura authentication config is omitted for this demo
-
${INFURA_USER}
will be provided through environment variable
Official Docker image you can find at: emeraldpay/dshackle
export INFURA_USER=...
docker run -p 2449:2449 -p 8545:8545 -v $(pwd):/etc/dshackle -e "INFURA_USER=$INFURA_USER" emeraldpay/dshackle
Now it listen on port 2449 at the localhost and can be connected from any gRPC compatible client. Tools such as gRPCurl can automatically parse protobuf definitions and connect to it (actual Protobuf sources are located in a separate repository which you can find at https://github.com/emeraldpay/proto)
Alternatively you can connect to port 8545 with traditional JSON RPC requests
Dshackle implements standard JSON RPC interface, providing additional caching layer, upstream readiness/liveness checks, retry and other features for building Fault Tolerant services.
curl --request POST \
--url http://localhost:8545/eth \
--header 'content-type: application/json' \
--data '{"jsonrpc":"2.0", "method":"eth_getBalance", "id":1, "params":["0x690b2bdf41f33f9f251ae0459e5898b856ed96be", "latest"]}'
{"jsonrpc":"2.0","id":1,"result":"0x72fa5e0181"}
Dshackle provides a custom gRPC based API, which provides additional methods and other features such as streaming responses. Please refer to the documentation: gRPC Methods
grpcurl -import-path ./proto/ -proto blockchain.proto -d "{\"type\": 100}" -plaintext 127.0.0.1:2449 io.emeraldpay.api.Blockchain/SubscribeHead
{ "chain": "CHAIN_ETHEREUM", "height": 8396159, "blockId": "fc58a258adccc94466ae967b1178eea721349b0667f59d5fe1b0b436460bce75", "timestamp": 1566423564000, "weight": "AnMcf2VJB5kOSQ==" } { "chain": "CHAIN_ETHEREUM", "height": 8396160, "blockId": "787899711b862b77df8d2faa69de664048598265a9f96abf178d341076e200e0", "timestamp": 1566423574000, "weight": "AnMch35tO6hSGg==" } ... ...
The output above is for a streaming subscription to all new blocks on Ethereum Mainnet. It’s one of services provided by Dshackle, in additional to standard methods provided by RPC JSON of underlying nodes.
For detailed documentation see docs/ directory.
Dshackle should be compatible with all standard libraries that use Ethereum JSON RPC over HTTP.
repositories {
maven {
url "https://dl.bintray.com/emerald/emerald-grpc"
}
}
dependencies {
compile "io.emeraldpay:emerald-grpc:0.6.0-0.2"
}
"dependencies": {
"@emeraldpay/grpc-client": "0.11.0-0.2",
}
See more in the documentation for Client Libraries.
gradle jib -Pdocker=gcr.io/myproject
Gradle will prepare a Docker image and upload it to your custom Docker Registry at gcr.io/myproject
(please change to address of your actual registry)
Want to support the project, prioritize a specific feature, or get commercial help with using Dshackle in your project? Please contact [email protected] to discuss the possibility
Copyright 2019 ETCDEV GmbH
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.