massalabs/massa

The Decentralized and Scaled Blockchain

Massa: The Decentralized and Scaled Blockchain

About Massa

Massa is a new blockchain based on a multithreaded technology that supports more than 10'000 transactions per second in a fully decentralized network with thousands of nodes. A short introduction video is available here.

Massa's purpose is to make it easy to deploy fully decentralized applications. We've developed two key technologies that are exclusive to Massa to help make this possible: autonomous smart contracts and native front-end hosting.

Here is a list of tools to easily build applications on the Massa blockchain:

• JS Client library to connect to the Massa blockchain from your applications.
• AssemblyScript SDKs to write smart contracts.
• Examples of applications built on Massa.
• Explorer.
• Interactive API specification.
• Lots of documentation, from web3 development to Massa's architecture.

Become a node runner

With decentralization as a core value, we've gone to great lengths to lower the barrier of entry for community participation in our mainnet, and we invite you to join in by becoming a node runner!

Community

We have active community discussions on several platforms, and we invite you to join the conversations. If you wish to go into some depth about technical aspects, speak with people working in the ecosystem full-time, or just have a chat with other like-minded people, these can be good places to start:

• Discord
• Telegram
• Twitter

Contributing

We welcome contributions from the community at large, from anywhere between seasoned OSS contributors, to those hoping to try it for the first time.

If you would like some help to get started, reach out to us on our community Discord server. If you're comfortable enough to get started on you're own, check out our good first issue label.

Contributors

A list of all the contributors can be found here

Community Charter

The Massa Community Charter is designed to protect decentralization. You can find it here: COMMUNITY_CHARTER.md

Initial distribution files

The following initial distribution files:

• massa-node/base_config/initial_ledger.json
• massa-node/base_config/deferred_credits.json
• massa-node/base_config/initial_rolls.json

Are copied from https://github.com/Massa-Foundation/genesis-ledger/tree/main/node_initial_setup at commit hash 9bb16c286d2bdc830490bd0af70571207d34921c.

massa's Issues

Add timer in fn_node_controller

In GitLab by @AureliaDolo

Add a timer that ask periodically for peer list in the select in fn_node_controller

Prevent 10 sec delay on peer db shutdown

In GitLab by @damip

Select! the timeout OR None Or Some(None) to break timeout

fn handshake

In GitLab by @AureliaDolo

todo parameters: connection ID, socket

TODO
- socket.split_into
let (mut socket_reader, mut socket_writer) = socket.split_into();
let (reader, writer) = (ReadBinder::new(socket_reader), WriteBinder::new(socket_writer));
- wrap each side with the corresponding binder
- do the crypto handshake (do not forget timeouts !)
- return (ConnectionId, Ok(NodeId, writer, reader)) or (ConnectionId, Err) if error

Setup docker testing environment

In GitLab by @damip

Setup a docker-based testing environment with network simulation

Add pub to crypto module

In GitLab by @pdelrieu

All the missing pub to method signature in crypto module

Refactor trace

In GitLab by @damip

Standardize trace! => use json

Add NodeCommand

In GitLab by @AureliaDolo

Add following node commands :

• SendBlock
• SendTransaction

Process them inside fn_node_controller.

do not panic on chrono problem

In GitLab by @damip

Create massa_trace macro

In GitLab by @AureliaDolo

We use traces to follow what's happening during tests.

Here is what a trace looks like for now :

trace!(
    "massa_trace:{}",
    serde_json::json!({
        "origin": concat!(module_path!(), "::ProtocolController::new"),
        "event": "self_node_id",
        "parameters": {
            "node_id": self_node_id
        }
    })
    .to_string()
)

It takes a lot of space and makes the code less readable.

We are willing to implement a macro that we could use like that :

massa_trace!("self_node_id", {
    "node_id": self_node_id
})

It will take as argument the event name and a struct of parameters, with values. It could use the crate function_name to retrieve the name of the function we are inside.

ProtocolControler Main Loop protocol_controller_fn and extern communication

In GitLab by @pdelrieu

Implement new and protocol_controller_fn function.
Implements Protocol controler outside communication

Document peer_info_database

In GitLab by @damip

Write the peer_info_database documentation

Dump and load peer list

In GitLab by @damip

In network/peer_database.rs, reimplement the load_peers and dump_peers functions so that:

• dump_peers:

  • dumps the peers as a json file
  • if the peer status is Banned, keep it as Banned in the file
  • if the peer status is anything else, dump it as Idle

• laod_peers:

  • load peers form json file
  • if the status in the file is anything else than Idle or Banned, throw an error
  • if zero peers have been loaded, throw an error

All file reads/write need to be asynchronous (use tokio::fs async functions)

Serde must be used as much as possible. Avoid implementing stuff manually

Consensus module first draft

In GitLab by @AureliaDolo

Establish first draft of consensus module.

Queries have a unique id.

• Protocol is sending blocks to consensus
  • Check if incoming block is new and valid
    • if yes propagate node
    • else discard block
  • Save block*
• Protocol ask for block by hash block**
  • return Option(Block)
• Block creation for testing purpose

* for now everything is in memory

** at the end : thread number + slot number + hash

Everything is async.

Implement tests for peer_info_database

In GitLab by @damip

Test that the various functions yield the right return values and the expected modifications on a known peer database.

Add log when denying connection from a banned peer

In GitLab by @AureliaDolo

Implement feedback functions in NetworkController

In GitLab by @damip

Rename PeerConnected as PeerAlive

Implement methods transparently wrapping the corresponding peer feedback messages:

• feedback_peer_list(hashset) sends PeerFeedbackEvent.PeerList
• feedback_peer_closed(Ipaddr, reason) sends PeerFeedbackEvent.ConnectionClosed
• feedback_peer_alive(IpAddr) sends PeerFeedbackEvent.PeerAlive

Replace f32 by Duration when needed

In GitLab by @AureliaDolo

For now fields containing seconds, like message_timeout_seconds in ProtocolConfig, are represented by f32. Find all those fields and replace them by Duration.

Add code coverage info

In GitLab by @damip

Add code coverage info to repository

Prevent connection loops

In GitLab by @damip

Prevent infinite connection loops when connections fail

High level networking

In GitLab by @damip

Create a peer database, peer inventory watcher, and implement node discovery.

Crypto wrapper

In GitLab by @damip

• find rust validated libraries implementing ECDSA secp256k1 https://docs.rs/rust-crypto/0.2.36/crypto/
• implement a wrapper module that alows us to:
  • generate a random private key
  • derive a public key from a private key
  • sign data
  • verify a signature
  • load or dump a public or private key as raw bytes
  • load or dump a public or private key as base58check (use a crate bs58)

Mock the node for test

In GitLab by @pdelrieu

Implement what is defined in the wiki page Functional testing of one node

Add hashes to crypto

In GitLab by @damip

Add a hashes.rs file alongside signature.rs, that wraps the bitcoin_hashes crate : https://docs.rs/bitcoin_hashes/0.9.4/bitcoin_hashes/index.html

Proceed in a similar way as they did:

• define HashEngine that wraps bitcoin_hashes::HashEngine
• define Hash that behaves like bitcoin_hashes::Hash
  • implement two specializations : sha256 and ripemd160

The Hash trait must be serializable/deserializable as bs58check when human readable, and as binary otherwise

Network restructuration 2

In GitLab by @damip

Simplify tokio::selects

In GitLab by @AureliaDolo

Now when we have a loop with a tokio::select we do everything in it. Split what is inside the select branches in new functions.

Implement get_connector_candidate_ips in peer database

In GitLab by @damip

Return a list of candidate IPs to connect to.

missing_out_connections = max(0, target_out_connections - count_peers_with_status(outconnected)) warning: usize is unsigned !
connection_slots_available = max(0, max_out_connection_attempts - count_peers_with_status(outconnecting) - count_peers_with_status(outhandshaking)) warning: usize is unsigned !
n_slots_available = min(missing_out_connections, connection_slots_available)
=> choose up to n_slots_available candidates based on the following criteria:
1 - most recent success (None = most ancient) 
2 - most ancient failure (None = most ancient)

Repair CI

In GitLab by @damip

For CI to pass, run cargo fmt on the whole project

Ip address validation

In GitLab by @flipchan

Verify that there is no invalid ip addresses in the file, so that someone can't "fool" the program to submit bad/invalid ips

Documentation in protocol/controller.rs

In GitLab by @AureliaDolo

fn_node_controller and fn_node_writer

In GitLab by @AureliaDolo

define

async fn fn_node_controller(/*  NodeId, reader, writer, node_message_rx, node_event_tx */) {
    /*
        TODO
        - define writer_tx, writer_rx = mpsc(1024)
        - define writer_evt_tx, writer_evt_rx = mpsc(1024)
        - spawn fn_node_writer(writer, writer_evt_tx, writer_rx)
        - set a regular timer to ask for peers


        - Loop events:
            * reader.next()  // incoming data from reader
                TODO process

            * node_message_rx  // incoming command from controller
                TODO process

            * writer_evt_rx:  // incoming event from writer
                TODO process

            * peer ask timer
                TODO use writer_tx to ask for peer list

        - stop writer
        - notify controller of closure (with reason) through node_event_tx
    */
}

and

async fn fn_node_writer(/* writer, writer_evt_tx, writer_rx */) {
    // TODO
    // wait for writer_rx event
    //   - writer.send(data).await, with timeout
    //   - if error, notify through writer_evt_tx and return
}

Timestamp in logs

In GitLab by @damip

Network integration test attempt

In GitLab by @damip

Use netsim, tempfile to define network and file tests

Review controller.rs

In GitLab by @damip

I have made cleanups and changes to controller.rs. Please review it and correct stuff here.

Support NAT-ed/firewalled/proxied connections

In GitLab by @damip

Modify the low level network controller in the following way:

• add a boolean to PeerInfo: is_reachable
• if a peer connects to us and we didn't know about it, define is_reachable=false
• when transmitting peer lists to peers, only send is_reachable=true peers, and own IP if reachable
• only attempt connections towards is_reachable=true peers
• when a is_reachable=false peer disconnects, it can be removed from peer list (except if it was banned)

this has consequences on peer database cleanup and node database cleanup/selection and load/dump functions

Reorganize project to make it mock-compatible

In GitLab by @damip

add cargo.logck to git

In GitLab by @damip

Peer load/dump alternative

In GitLab by @damip

An alternative, simpler way to do load/dump in the peer database

Simplify Gitlab CI

In GitLab by @damip

Add logging

In GitLab by @damip

Add debug! and info! calls everywhere

Replace strings by std::path::PathBuf when needed

In GitLab by @AureliaDolo

Prevent remote ID to be the same as local in handshake

In GitLab by @damip

Someone could be redirecting our own traffic to ourselves...

Prevent incoming connections from our own ip

In GitLab by @damip

Implement peer_database cleanup

In GitLab by @damip

Implement the cleanup() function in network/peer_database.rs

It needs to remove some nodes from the database if the db size exceeds config.max_known_nodes.

• it must not remove connected, connecting or handshaking peers
• it must limit the number of idle peers to max_idle_peers (todo create this config entry)
  • drop in priority those with the oldest successful connection (None = oldest)
• it must limit the number of banend peers to max
  • drop in priority the oldest failed connection (drop all None, even tho there shouldn't be any)

Define own routable IP

In GitLab by @damip

Add a config entry to define own routable IP address

Evaluatre security issues on Vec<_> deserialization

In GitLab by @damip

Attackers may send arbitrary big Vec<_>. Evaluate the risk and mitigation options.

Peer db: add new_nodes method

In GitLab by @damip

When we receive a list of peers from the network, we want to fuse it with our current peer database.

• add a "new_nodes" method to the peer database struct impl that takes a set of new IP addresses (HashSet) as a parameter
• this method needs to add peers from this new list to our current hashmap but in a smart way because some malicious peers may announce a large number of random (but valid) IP addresses
• the database length must not exceed config limits (to be defined) to prevent memory flooding. If we have all the peers we need, no need to add more.
• the database should keep existing peers that it already knows are working, and replace dead peers with new, untested ones

For all of that, only the fields within the PeerInfo struct may be used as a source of information

Warning: peers that are connecting, handshaking connected or banned must not be removed from the database by this function.

Network restructuration

In GitLab by @damip

• generic PeerID type with minimal traits: partialeq, hash
• handshake/connection

Increment flexbuffers version

In GitLab by @damip

Thanks to our feedback, flexbuffers moved to version 0.2.1. Update cargo.toml

Test Network Controller

In GitLab by @pdelrieu

Test the network controler alone.
Test connections in and out
Detect timeout error
Test connection close.

Create top level documentation for every module

In GitLab by @AureliaDolo

Create top level doc following modules :

• crypto
  • hash
  • signature
• network
  • config
  • connection_controller
  • peer_info_database
• protocol
  • binders
  • controller
  • messages