josehu07/summerset

Distributed, Replicated, Protocol-generic Key-value Store in Async Rust

Summerset

Summerset is a distributed, replicated, protocol-generic key-value store supporting a wide range of state machine replication (SMR) protocols for research purposes. More protocols are actively being added.

List of currently implemented protocols...

Name	Description
RepNothing	Simplest protocol w/o any replication
SimplePush	Pushing to peers w/o consistency guarantees
ChainRep	Bare implementation of Chain Replication (paper)
MultiPaxos	Classic MultiPaxos protocol (paper)
RSPaxos	MultiPaxos w/ Reed-Solomon erasure code sharding (paper)
Raft	Raft with explicit log and strong leadership (paper)
CRaft	Raft w/ erasure code sharding and fallback support (paper)

Formal TLA+ specification of some protocols are provided in tla+/.

Why is Summerset different from other codebases...

• Async Rust: Summerset is written in Rust and demonstrates canonical usage of async programming structures backed by the tokio framework;
• Event-based: Summerset adopts a channel-oriented, event-based system architecture; each replication protocol is basically just a set of event handlers plus a tokio::select! loop;
• Modularized: Common components of a distributed KV store, e.g. network transport and durable logger, are cleanly separated from each other and connected through channels.
• Protocol-generic: With the above two points combined, Summerset is able to support a set of different replication protocols in one codebase, with common functionalities abstracted out.

These design choices make protocol implementation in Summerset straight-forward and understandable, without any sacrifice on performance. Comments / issues / PRs are always welcome!

Build

Install the Rust toolchain if haven't. For *nix:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

Build everything in debug or release (-r) mode:

cargo build [-r] --workspace [--features ...]

Run all unit tests:

cargo test --workspace

Generate & open documentation for the core library in browser:

cargo doc --open

Usage

Launch a Cluster

First, launch the cluster manager oracle (which only serves setup & testing purposes and does not participate in any of the protocol logic):

cargo run [-r] -p summerset_manager -- -h

Then, launch server replica executables:

cargo run [-r] -p summerset_server -- -h

The default logging level is set as >= info. To display debugging or even tracing logs, set the RUST_LOG environment variable to debug or trace, e.g.:

RUST_LOG=debug cargo run ...

Run Client Endpoints

To run a client endpoint executable:

cargo run [-r] -p summerset_client -- -h

Currently supported client utility modes include: repl for an interactive CLI, bench for performance benchmarking, and tester for correctness testing.

Helper Scripts

Some helper scripts for running Summerset processes are available. First, install dependencies:

pip3 install toml

You can find the scripts for running Summerset locally in scripts/:

python3 scripts/local_cluster.py -h
python3 scripts/local_clients.py -h

And for running on a set of distributed machines (requiring correctly filled scripts/remote_hosts.toml file):

python3 scripts/distr_cluster.py -h
python3 scripts/distr_clients.py -h

TODO List

• async event-loop foundation
• implementation of Chain Replication
  • failure detection & recovery
  • TLA+ spec
• implementation of MultiPaxos
  • TLA+ spec
• implementation of RS-Paxos
• implementation of Raft
  • TLA+ spec
• implementation of CRaft
  • TLA+ spec
• long-term planned improvements
  • use a sophisticated storage backend
  • efficient state-transfer snapshotting
  • more robust TCP msg infrastructure
  • membership discovery & view change
  • multi-versioning & stale reads
  • partitioned groups service structure
• client-side utilities
  • interactive REPL
  • benchmarking client
  • unit tester
  • linearizability fuzzer
• better README & documentation

---

Lore: Summerset Isles is the name of an elvish archipelagic province in the Elder Scrolls series.