postech-dao/simperby

The ultimate BFT blockchain engine for decentralized organizations with powerful trustless interoperability.

simperby's Issues

Design chat system

The ideal medium of the human communication for the DAO (which's using simperby)
will be p2p gossip, not Discord or something.
In order to realize it, simperby should support a chatting system leveraging the existing p2p network.
Also, chat logs for a height will be an important historical record that deserves to be saved in the block of the very height..

Add CSV test that covers `Diff::General`

Support non-validator votemint client.

i.e. HeightInfo::this_node_index must be Option.

Investigate storage tests

They sometimes fail in CI, so currently #[ignore] is set.

Implement in-process version of P2P network

• No file system, No TCP
• Use in-process communication channel such as tokio::sync::mpsc
• Ignore storage directory
• Use socket addr as an routing identifier of mpsc. A trivial implementation would have a global HashMap<SockAddr, Send>

Use indices instead of keys for header

https://github.com/postech-dao/simperby/blob/main/common/src/lib.rs#L30

Update
pub delegation: Vec<(PublicKey, PublicKey)>,
to
pub delegation: Vec<(usize, usize)>,
where usize is the index to validator_set.
This will decrease the header size which's important.

Add WASM compile CI for `simperby-common`

Those two crates will be imported by the contract side, so there must be no dependencies that breaks WASM compatibility.
Having a CI pipeline that checks such integrity would help.

Keep only CSV-acceptable branches in `clean()`

Make the initial broadcaster not responsible for the task

Broadcasted packet must be re-broadcasted even if the original broadcaster goes offline.

Define interface and scheme for the Merkle tree

Research tendermint-rs

Link

As Tendermint is the most likely consensus to adopt, we have to research about the existing implementation of it.
We could either directly use it, or just take it as a reference for our original implementation.

The followings are what we have to research:

1. There are few unique conditions for our system (described in the tracking issue), so one should thoroughly survey whether tendermint-rs is capable of them.
2. We should find out how tendermint-rs abstracted (or embed) the network layer, and should be aware of how it differs from our abstraction of networking.
3. A summary of the final interface they provide is required as well.
4. What can and cannot be customized.

Create a comparison table with Hyperledger

We need comprehensive comparisons with the Hyperledger protocol.

Improve delegation protocol

mitigation of the prevention of multi-level delegation would be helpful

Return `valid_round` in ConsensusResponse

Tracking issue for the storage research

Our requirements for the storage

1. Key-Value
2. JSON-like schema
3. Simple, Easy, and widely used in other blockchain projects
4. Rust binding

Tracked issues

• #10
• #9

Define unit tests for peer discovery without internet

Make DMS more isolated in a single height

1. tag the height on the dms key
2. message (unit of public key sign) contains target height
3. take height-consistent data in the DMS state (e.g., eligible network users)

Implement simple-format de/serializer for crypto types

They should encode into a pretty and simple way, like 0123456789ABCDEF, using the hex crate.
You will have to remove #[derive(Serialize, Deserialize] and manually implement them.

Rename vote to non-nil-vote

Due to the special property of Vetomint, it's common to refer both nil votes and non nil votes.
Here we decided the naming rule that is applied universally throughout our project.

1. Nil vote (NilPrevote, NilPrecommit): the vote for skipping the round.
2. Non nil vote (NonNilPrevote, NonNilPrecommit): the vote for the proposal
3. Vote (Prevote, Precommit): either nil vote or non nil vote.

Integrate all commit hashes in block header

/// The hash of aggregation of all the commits for this block.
pub commit_hash: Hash256,
/// The Merkle root of transactions.
pub tx_merkle_root: Hash256,
/// The Merkle root of the chat log.
pub chat_merkle_root: Hash256,
/// The Merkle root of the non-essential state.
pub repository_merkle_root: Hash256,

to

/// The merkle root of all the commits for this block.
pub commit_merkle_root: Hash256,
/// The Merkle root of the state.
pub repository_merkle_root: Hash256,

1. All commits (tx, agenda, agenda proof, extra-agenda tx, chat) will be wrapped in a single merkle tree. As discussed, I think it's not necessary and even confusing to have separate roots
2. repository_merkle_root now contains the reserved state as well, though it will not be calculated in the MVP anyway.

Rename the branch name `main` to `finalized` for repo

Define a new broadcast interface using GossipSub

Motivation

As I talked with @junha1, FloodSub seems to be inefficient, fragile, and an experimental feature, so we decided to use libp2p-GossipSub instead of FloodSub.

⚠️ Although several changes including this are planned to be made to simperby-network, I will focus on the broadcasting module in this issue.

GossipSub

A brief intro of the functionalities

todo

Notable properties

todo

Use in simperby network

todo

Implement vetomint

Tracking issue for the consensus research

Simperby's consensus MUST be implemented as a simple, safe, and live BFT algorithm.

Conditions

Here are some conditions and relaxations (compared to ordinary blockchain consensuses)

1. No need for slashing conditions. (No strong accountability needed)
2. Instant (one-block) finalization.
3. On-demand block production. (No empty blocks)
4. Consensus-level yes-or-no choice (This becomes trivial for an asynchronous consensus, as voting for 'reject' equals to the case in the asynchrony, where a node pretends that he's missing the block proposal (which, in fact, he dislikes) because of the network asynchrony.)
5. Fair chances of block-proposing right are NOT required. (stable leader is OK, and actually desired)
6. BFT threshold 1/3 is allowed to become smaller.
7. Synchrony assumptions is also OK to take.
8. There MUST be a support for the light client verification. (i.e., it must be based on cryptography, not some live observation protocol)

Tracked issues

• #7
• #6
• #5

Implement p2p network

Remove unnecessary unwrap in vetomint

let proposal = if state.valid_value.is_some() {
    state.valid_value.unwrap()
}

can be rewritten as

let proposal = if let Some(x) = state.valid_value {
    x
}

example: https://github.com/postech-dao/simperby/blob/main/vetomint/src/progress.rs#L354

Relax timestamp condition for CSV

Change the comparison condition to be inclusive.

This will be helpful in testing or if it's not easy to determine the increasing timestamp sequence,
while not compromising any security.

Add unit tests for vetomint

Make network broadcast trial be able to be stopped

Make `progress()` not take `HeightInfo`

HeightInfo must be carried by ConsensusState itself.

Add 'check_broadcast_status(token)` for network interface

The node operator would check how well his message is broadcasted, and decide whether to turn off his node.
The return value could be

1. Number of successfully relayed nodes
2. Some kind of a 'score' that considers how well the relayed nodes will actually relay well, according to their past activities.

I think the option 1 will be sufficient for now.

Remove KVStorage

It is not used for now.

Add `preserve_order` for all serde_json deps

Make a summary for those leaderless BFTs

If the leaderless design makes the implementation simpler, we can consider to use.
(Additionally, being leaderless makes the UX better in our case)

Add total size query in KVStorage

/// Queries the total size of bytes occupied in the file system.
async fn get_total_size_in_bytes(&self) -> Result<u64, Error>;

Add badges in README.md

Discord, Build status, ...

test

Design `ConsensusStatus`

simperby_node::ConsensusStatus

It must contain useful information for the user.
Note that ConsensusStatus is strictly for a single height.
It must be print in a nice and human-readable way.

Introduce git-managed chain state

Check how PBFT can be safe without lock mechanism

Compared to Tendermint.

Setup CI for markdown

That checks every markdown file in the repo for

• lint
• word-wrap

Summarize how Geth implement history state storage

Change the Crypto scheme to the same one that Ethereum uses

It will make the EVM light client much more cheap on its gas price.

• hash: KECCAK-256
• keypair: secp256k1

Implement CommitSequenceVerifier

Add `Member::expelled`

/// If true, all voting powers are ignored.
/// Note that once granted, Simperby keeps all members forever in the reserved state.
/// If you want to remove a member, you must set this to true instead of removing the member.
///
/// TODO: consider this in various verification functions.
pub expelled: bool,

Implement in-memory KVStorage

This will be useful for testing and special configurations.
depends on #57

Tracking issue for the consensus candidate `interactive-tendermint`

This is a variation of the Tendermint protocol, with few small tweaks.

1. There is no round timeout. This breaks liveness, but it doesn't really matter because we don't produce block regularly anyway (we even assume that almost every participating nodes stay offline for most of the time)
2. Pre-vote and Nil-vote are interactive. That is, validators (human) will be requested whether to pre-vote or nil-vote for the given received block proposal. Since there is no timeout, nil-vote is not automatically performed and can be used as a 'veto' as well. Pre-commit will be automatically performed if the user has decided to pre-vote.
3. The leader round robin order is specified in the blockchain state, and this order won't be shuffled for each block. This is for the stable leader, which makes the consensus process efficient for our case. Note that the order can be explicitly changed by a transaction anyway.

Handle asynchrony for KVStorage

Currently it is using blocking methods (which MUST be fixed)

Survey applicable functions in libp2p

We need a brief overview of the functionality of the library, which should be enough to finalize the interface of networking module

Rename vetomint to votemint