libp2p/rust-libp2p

The Rust Implementation of the libp2p networking stack.

Central repository for work on libp2p

This repository is the central place for Rust development of the libp2p spec.

Getting started

• Main documentation can be found on https://docs.rs/libp2p.

• The examples folder contains small binaries showcasing the many protocols in this repository.

• For security related issues please file a private security vulnerability report . Please do not file a public issue on GitHub.

• To report bugs, suggest improvements or request new features please open a GitHub issue on this repository.

• For rust-libp2p specific questions please use the GitHub Discussions forum https://github.com/libp2p/rust-libp2p/discussions.

• For discussions and questions related to multiple libp2p implementations please use the libp2p Discourse forum https://discuss.libp2p.io.

• For synchronous discussions join the open rust-libp2p maintainer calls or the biweekly libp2p community calls.

Repository Structure

The main components of this repository are structured as follows:

• core/: The implementation of libp2p-core with its Transport and StreamMuxer API on which almost all other crates depend.

• transports/: Implementations of transport protocols (e.g. TCP) and protocol upgrades (e.g. for authenticated encryption, compression, ...) based on the libp2p-core Transport API.

• muxers/: Implementations of the StreamMuxer interface of libp2p-core, e.g. (sub)stream multiplexing protocols on top of (typically TCP) connections. Multiplexing protocols are (mandatory) Transport upgrades.

• swarm/: The implementation of libp2p-swarm building on libp2p-core with the central interfaces NetworkBehaviour and ConnectionHandler used to implement application protocols (see protocols/).

• protocols/: Implementations of application protocols based on the libp2p-swarm APIs.

• misc/: Utility libraries.

• libp2p/examples/: Worked examples of built-in application protocols (see protocols/) with common Transport configurations.

Community Guidelines

The libp2p project operates under the IPFS Code of Conduct.

tl;dr

• Be respectful.
• We're here to help: [email protected]
• Abusive behavior is never tolerated.
• Violations of this code may result in swift and permanent expulsion from the IPFS [and libp2p] community.
• "Too long, didn't read" is not a valid excuse for not knowing what is in this document.

Maintainers

(In alphabetical order.)

• João Oliveira (@jxs)
• Thomas Eizinger (@thomaseizinger)

Notable users

(open a pull request if you want your project to be added here)

• COMIT - Bitcoin–Monero Cross-chain Atomic Swap.
• Forest - An implementation of Filecoin written in Rust.
• fuel-core - A Rust implementation of the Fuel protocol.
• HotShot - Decentralized sequencer in Rust developed by Espresso Systems.
• ipfs-embed - A small embeddable ipfs implementation used and maintained by Actyx.
• Homestar - An InterPlanetary Virtual Machine (IPVM) implementation used and maintained by Fission.
• beetle - Next-generation implementation of IPFS for Cloud & Mobile platforms.
• Lighthouse - Ethereum consensus client in Rust.
• Locutus - Global, observable, decentralized key-value store.
• OpenMina - In-browser Mina Rust implementation.
• rust-ipfs - IPFS implementation in Rust.
• Safe Network - Safe Network implementation in Rust.
• Starcoin - A smart contract blockchain network that scales by layering.
• Subspace - Subspace Network reference implementation
• Substrate - Framework for blockchain innovation, used by Polkadot.
• Taple - Sustainable DLT for asset and process traceability by OpenCanarias.

rust-libp2p's Issues

Add a bandwidth management library

Would wrap around a Transport and implement the Transport trait itself.

For uploads, one possible way of implementing this is having a buffer whose content is dispatched over time to the underlying transport (using tokio-timer for example). If the buffer is full when we want to send something, return WouldBlock. Don't forget to later notify the tasks for which we returned WouldBlock.

As for downloads, we can return WouldBlock if we detect that the rate would be too fast. We would only read a maximum number of bytes from the underlying transport in order to not go over the rate. If we return WouldBlock, then don't forget to notify the tasks later.

Create a README.md and tests for every crate

The top-level README should cover what packages/crates are in this repository and a high-level overview of what they do and maybe something similar to https://github.com/libp2p/go-libp2p.

Every package should also contain its own README with a clearer explanation of what the library is, how it's intended to be used and what the API is. Tests, examples and obviously rustdocs would also be required here.

Check off a package if you've added README, tests and rustdocs:

• datastore
• example
• libp2p-peerstore
• libp2p-ping
• libp2p-secio
• libp2p-swarm
• libp2p-tcp-transport
• multistream-select
• rw-stream-sink

Server stops with an error if a client doesn't support any of the protocols of the server

This problem is easy to fix, however it becomes much harder to fix if we want to report failing clients to the user of the library so that they can for example perform logging.

Implement Ping / KeepAlive

See: https://github.com/libp2p/go-libp2p-ping

Remove timeout parameter from kademlia

Check that the identity of the node matches its public key in secio

Since nodes self-report their identity, it cannot be trusted.

Discussion: composable dialing

Dialing hasn't really been sorted out yet. It's easy to do for simple multiaddrs, but for complex ones, particularly those with multiple hops, things get much fuzzier.

Imagine we have registered two forms of transport: TCP and p2p-circuit, which is used to relay connections.

One example which is difficult to make work is something like
ip4/1.2.3.4/tcp/8888/p2p-circuit/p2p/DestPeer

(circuit relay multiaddr format is <relay-peer>/p2p-circuit/<remote-peer>)

This address, when used for dialing, says "Connect to the peer DestPeer on any available address, through a relay node we will connect to via tcp on port 8888 over the ipv4 address 1.2.3.4"

ip4/4.5.6.7/tcp/30042/p2p-circuit/ip4/1.2.3.4/tcp/8888/p2p-circuit/p2p/DestPeer

Here's a double-hop relay:
Connect to the peer DestPeer through the relay on tcp://1.2.3.4:8888, which we will connect to through the relay on tcp://4.5.6.7:30042.

We'll need to require dialers to handle the whole address, and give them a closure or similar required to instantiate connections to different encapsulated multi-addresses.

So the logic for a circuit dial would (I think?) be

1. find the left-most mention of p2p-circuit.
2. split this multiaddr along that mention into the relay and remote addresses.
3. context.dial(relay).and_then(move |conn| RelayProtocol::dial_through(remote, conn))

This reads as: dial to the relay node, and then when we get a connection to it, attempt to dial the rest of the address through it.

One interesting thing to note is that our individual transport Conns to peers can actually be instances of a Socket being multiplexed over a different Conn to the same peer.

Even in the multi-hop case, we only open one logical connection, through TCP. this means that we're limited to basically one layer of virtual calls. I am not sure this holds true in general, but it depends how fine-grained we want to go.

As in, if we have the multi-addr ip4/1.1.1.1/tcp/20202/ws, we can either have the ws transport wrap a dialled tcp connection, or have the ws transport handle the tcp connection internally. The first option is ideal for "composability" because it means that the ws transport can handle any kind of wire beneath it, but then you can get into the situation of multiple levels of boxed trait objects, which are bad for performance in a multitude of ways.

It may not really be avoidable though, as long as we ensure the cost of connecting/communicating with a peer remains roughly proportional to how complicated the address for that peer is: simple peers will have fewer layers of indirection. More complex peer connections pay a cost for their complexity.

Code style: split `swarm/src/transport.rs` in multiple modules

I find this module messy and I think we should split it in multiple modules for better readability.

Why is there a custom multihash impl?

This should be using https://github.com/multiformats/rust-multihash and if that is not working as expected please PR needed changes there.

Record Store Interface

See: https://github.com/libp2p/interface-record-store

Grant access to the public key of the secio layer

Once we upgraded a connection through secio, the upper layer need to have somehow access to the public key that is used by the remote.

Required for a proper Kademlia implementation, otherwise we have no way to check the authenticity of nodes.

An easy solution would be to add a trait PublicKeyProvide with a get_public_key(&self) -> ? method, that would be implemented on the output of the secio middleware and on any stream that lies on top of secio. In my opinion it makes sense that this trait belongs to the core of the library (and not to libp2p-secio), in order to avoid a spaghetti of dependencies and also to enable alternative implementations of secio/security.

Multiplex stream implementation

Under the stream muxer interface.

See go implementation: https://github.com/whyrusleeping/go-smux-multiplex

Implement http2 multiplexing middleware

Although HTTP 2.0 is not a multiplexing protocol, we can use it in order to benefit from the existing ecosystem instead of having to create our own library.

Test Infrastructure

We want some way to easily create a large and distributed (over IP-space) simulated network that allows for easy testing of peer discovery, routing and other aspects.

From initial research, following tools seem suitable:

• https://gns3.com
• https://www.nsnam.org/ (NS-3)

Implement floodsub messages authentication and encryption

Compile issue error: paths in `use` groups are experimental (see issue #44494)

I have try to compile the lates master
I get the following issue

error: paths in use groups are experimental (see issue #44494)

Maybe it is a compiler version issue.
What is the compiler version used.

See the script dump belove including the compiler version and the OS version.

   Compiling tokio-tcp v0.1.0
   Compiling ring v0.12.1 (https://github.com/briansmith/ring?rev=3a14ef619559f7d4b69e2286d49c833409eef34a#3a14ef61)
   Compiling cid v0.2.3
   Compiling varint v0.1.0 (file:///home/cbr/work/hashwave/rust-libp2p/varint-rs)
warning: missing documentation for a variant
  --> varint-rs/src/lib.rs:46:5
   |
46 | /     error_chain! {
47 | |         errors {
48 | |             ParseError {
49 | |                 description("error parsing varint")
...  |
60 | |         }
61 | |     }
   | |_____^
   |
note: lint level defined here
  --> varint-rs/src/lib.rs:21:9
   |
21 | #![warn(missing_docs)]
   |         ^^^^^^^^^^^^
   = note: this error originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)

warning: missing documentation for a variant
  --> varint-rs/src/lib.rs:46:5
   |
46 | /     error_chain! {
47 | |         errors {
48 | |             ParseError {
49 | |                 description("error parsing varint")
...  |
60 | |         }
61 | |     }
   | |_____^
   |
   = note: this error originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)

warning: missing documentation for a method
   --> varint-rs/src/lib.rs:285:5
    |
285 |     pub fn source(&self) -> &T {
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^

warning: missing documentation for a struct
   --> varint-rs/src/lib.rs:408:1
    |
408 | pub struct VarintCodec<W> {
    | ^^^^^^^^^^^^^^^^^^^^^^^^^

   Compiling tokio v0.1.6
   Compiling multiaddr v0.3.0
   Compiling multistream-select v0.1.0 (file:///home/cbr/work/hashwave/rust-libp2p/multistream-select)
   Compiling libp2p-peerstore v0.1.0 (file:///home/cbr/work/hashwave/rust-libp2p/peerstore)
   Compiling libp2p-swarm v0.1.0 (file:///home/cbr/work/hashwave/rust-libp2p/swarm)
   Compiling aio-limited v0.1.0 (https://github.com/paritytech/aio-limited.git#c6f15ffb)
error: paths in `use` groups are experimental (see issue #44494)
  --> swarm/src/upgrade/plaintext.rs:24:17
   |
24 | use std::{iter, io::Error as IoError};
   |                 ^^^^^^^^^^^^^^^^^^^^

error: paths in `use` groups are experimental (see issue #44494)
  --> swarm/src/upgrade/simple.rs:25:17
   |
25 | use std::{iter, io::Error as IoError, sync::Arc};
   |                 ^^^^^^^^^^^^^^^^^^^^

error: paths in `use` groups are experimental (see issue #44494)
  --> swarm/src/upgrade/simple.rs:25:39
   |
25 | use std::{iter, io::Error as IoError, sync::Arc};
   |                                       ^^^^^^^^^

error: aborting due to 3 previous errors

error: Could not compile `libp2p-swarm`.
warning: build failed, waiting for other jobs to finish...
error: build failed
rumskib:~/work/hashwave/rust-libp2p> 

rumskib:~/work/hashwave/rust-libp2p> git log | head -40

commit 37881d511ee731a3ebf532adfec90b45279b7952
Author: Pierre Krieger <[email protected]>
Date:   Tue May 8 22:07:07 2018 +0200

    Pass the host name with websockets (#177)

commit 64278244defc91b0470b582d402c3ec1b0a7e2be
Author: Toralf Wittner <[email protected]>
Date:   Tue May 8 20:43:00 2018 +0200

    Add log statements to multistream-select. (#180)
    
    Addresses issue #126.

commit d9e1e437b8fa64aea23f7789ac6de7dd01594a10
Author: Pierre Krieger <[email protected]>
Date:   Tue May 8 20:09:41 2018 +0200

    Try fix circleci again (#181)

commit 773cf6c26db31c56828de4ba528730fa16890e68
Author: Toralf Wittner <[email protected]>
Date:   Tue May 8 19:20:57 2018 +0200

    Add log statements to libp2p-mplex. (#179)
    
    * Add log statements to libp2p-mplex.
    
    Addresses issue #126.
    
    * Run `cargo fmt`.
    
    * Wording.
    
    * Adjust log levels.

commit fd0d75f61eb8dbd24de9de5dbc9220e7bb38e497
Author: Pierre Krieger <[email protected]>
Date:   Sun May 6 17:41:06 2018 +0200

rumskib:~/work/hashwave/rust-libp2p> 

rumskib:~/work/hashwave/rust-libp2p> rustc --version

rustc 1.24.1
rumskib:~/work/hashwave/rust-libp2p> cargo --version

cargo 0.25.0
rumskib:~/work/hashwave/rust-libp2p> lsbv���

lsb_release lsblk       

rumskib:~/work/hashwave/rust-libp2p> lsb_release -a

No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 17.10
Release:	17.10
Codename:	artful

Add a transport capability system

A transport protocol like UDP doesn't guarantee reliability or ordering, but some upper layer protocols may require that the transport supports these characteristics.

Therefore it wouldn't be a bad idea to add methods to the Transport trait so that upper layers can check for these capabilities.

Feature requests

https://github.com/libp2p/specs/ doesn't mention rust libp2p much. Could you open up a project tracker for this repo, documenting what has been done, what is in progress and what needs to be done?

I am looking at using rust-libp2p for Ethereum sharding with Drops of Diamond.

The following features don't show up for rust-libp2p or in search results, so I suggest adding them to issues and the project tracker:

• interface-transport
• Circuit Relay
• https://github.com/libp2p/interface-connection
• https://github.com/multiformats/multiaddr#implementations (listed as a beta for Rust)
• https://github.com/libp2p/interface-stream-muxer
• https://github.com/multiformats/multicodec#implementations
• gossipsub, see https://github.com/libp2p/go-floodsub/issues/77 for an introduction
• random-walk
• NAT traversal, e.g. libnice, libwebrtc, or natty. This actually [appears to be implemented](https://github.com/libp2p/rust-libp2p/search?utf8=%E2%9C%93&q=nat_traversal&type=0, it is just not too easy to find, so this is more of a docs issue.

There are Rust implementations of all the major formats that IPFS is using:

• rust-multiaddr
• rust-multihash
• rust-mutlibase
• rust-cid

— https://github.com/libp2p/libp2p/issues/33#issuecomment-392007707

What's already done in rust-libp2p:

This repository includes a facade crate named libp2p, which reexports the rest of the repository.

General overview of the architecture

Architecture of the other crates of this repository:

• datastore: Utility library whose API provides a key-value storage with multiple possible backends. Used by peerstore.
• example: Example usages of this library.
• libp2p-identify: Protocol implementation that allows a node A to query another node B what information B knows about A. Implements the ConnectionUpgrade trait of libp2p-core.
• libp2p-peerstore: Generic storage for information about remote peers (their multiaddresses and their public key), with multiple possible backends. Each multiaddress also has a time-to-live. Used by libp2p-core.
• libp2p-ping: Implementation of the ping protocol (the exact protocol is specific to libp2p). Implements the ConnectionUpgrade trait of libp2p-core.
• libp2p-secio: Implementation of the secio protocol. Encrypts communications. Implements the ConnectionUpgrade trait of libp2p-core.
• libp2p-core: Core library that contains all the traits of libp2p and plugs things together.
• libp2p-tcp-transport: Implementation of the Transport trait of libp2p-core for TCP/IP.
• libp2p-websocket: Implementation of the Transport trait of libp2p-core for Websockets.
• multistream-select: Implementation of the multistream-select protocol, which is used to negotiate a protocol over a newly-established connection with a peer, or after a connection upgrade.
• rw-stream-sink: Utility library that makes it possible to wrap around a tokio Stream + Sink of bytes and implements AsyncRead + AsyncWrite.

Additional transports

• IPC
• Bluetooth
• NFC

These are all interesting because they have their own discovery mechanisms that don't require Kademlia and in fact discovered addresses should not be propagated over "globally addressable" transports like IP since they would be almost entirely false positives. We could propagate bluetooth addresses to other bluetooth devices (to prevent an identify call) but I don't think that's necessary since we already narrow down the number of devices that we need to connect to by having the concept of "pairing", where the user selects which devices to trust.

I think that a good API would be for these to each have a Controller struct that has a fn scan taking a Fn(impl Stream<Item = Client>) -> impl Stream<Item = Client>, where Client is a struct containing information on a discovered client and the output is a list of clients to attempt connection to. For a chat app, this function could hand off control to a UI dialog and then return the clients that the user has chosen.

Implement crypto channels / secio

See: https://github.com/libp2p/go-libp2p-secio

Use bs58 instead of base58

I wrote base58, but the implementation is far from perfect. It's better to use bs58

Peer Routing Interface

See: https://github.com/libp2p/interface-peer-routing

libp2p-ratelimit shouldn't depend on tokio

I don't know for sure if we need a timer, but if one is needed, then it should depend on tokio-timer 0.2 and not on tokio.
Otherwise, tokio depends on tokio-core which doesn't compile for emscripten.

Secio incompatible with ring-0.12.1

Compilation of secio/src/lib.rs fails with

error[E0308]: mismatched types
   --> /home/my_user/.cargo/git/checkouts/rust-libp2p-98135dbcf5b63918/7d6ad23/secio/src/lib.rs:163:36
    |
163 |             RSAKeyPair::from_pkcs8(Input::from(&private[..])).map_err(|err| Box::new(err))?;
    |                                    ^^^^^^^^^^^^^^^^^^^^^^^^^ expected struct `untrusted::Input`, found a different struct `untrusted::Input`
    |
    = note: expected type `untrusted::Input<'_>` (struct `untrusted::Input`)
               found type `untrusted::Input<'_>` (struct `untrusted::Input`)
note: Perhaps two different versions of crate `untrusted` are being used?
   --> /home/my_user/.cargo/git/checkouts/rust-libp2p-98135dbcf5b63918/7d6ad23/secio/src/lib.rs:163:36
    |
163 |             RSAKeyPair::from_pkcs8(Input::from(&private[..])).map_err(|err| Box::new(err))?;
    |                                    ^^^^^^^^^^^^^^^^^^^^^^^^^

error[E0599]: no method named `digest_algorithm` found for type `ring::hmac::VerificationKey` in the current scope

and similar errors.
secio seems to be using untrusted-0.6.0, while ring-0.12.1 expects untrusted::Input from untrusted-0.5.1.
Likewise, secio calls digest_algorithm on ring::hmac::VerificationKey, a method that exists now, but did not in release 0.12.1.

Small nitpick, could you rename to `rust-libp2p`

Just to follow all the name convention used by all the other rust pieces we have :)

Implement QUIC

Create a QuicConfig<T> struct.

Since we require the underlying transport to be UDP and UDP operates on individual packets of data and not a stream, T::Output should implement Stream<Bytes> + Sink<Bytes> (or something similar to Bytes) instead of AsyncRead + AsyncWrite.

QuicConfig itself should implement Transport and its Output should implement StreamMuxer.

Use the protobuf-codegen library to generate the protobuf at compile-time

This wasn't done before because this requires installing protoc on the machine that compiles libp2p.
However a new crate https://crates.io/crates/protobuf-codegen was released, and is pure Rust and can thus be used.

Allow putting multiple middlewares on top of one another

Right now the API doesn't easily allow for example putting multiplex on top of multiplex on top of multiplex. The number of layers is hard-coded at compile-time, and cannot be easily changed dynamically at runtime.
Another example problem is that it is not possible to negotiate directly multiplex without going through plaintext or secio.

Peer Store

See: https://github.com/libp2p/go-libp2p-peerstore

Log important events with the `log` crate

Once #117 and #120 are merged, to avoid conflicts.

• examples (enable logging)
• identify
• peerstore
• ping
• secio
• swarm
• tcp-transport
• websocket
• multiplex
• multistream-select
• kademlia

Implement Multistream-select

See: https://github.com/multiformats/multistream

Add an example of rust-libp2p running in the browser

With asmjs and/or wasm.

Is that an objective as well? It should be easy to achieve.

Implement Kademlia record store

It is currently unimplemented.

Handle pings and pongs in the websocket transport

Follow-up of #84

Right now pinging a websocket opened by this library will trigger an error and close the stream.

Change nat_traversal to take ownership of one of the `Multiaddress`

So that the implementation can modify the address in place instead of building a new one.

However this requires adding some methods on the multiaddress struct for this to be feasible.

Switch to upstream rust-multiaddr

multiformats/rust-multiaddr#28 has been merged!

Once the current spaghetti of pull requests has been merged, we should remove rust-multiaddr from the local repo and switch to https://github.com/multiformats/rust-multiaddr/

It is possible to create an invalid Multiaddr through AddrComponent

An AddrComponent is not necessarily valid, while a Multiaddr always is.
This should be fixed either by checking the validity of the AddrComponent when building a Multiaddr, or by changing the API of AddrComponent.

Implement Kademlia pinging oldest nodes to empty the k-buckets

This is currently implemented with TODOs in the libp2p-kad crate.

Rename the libraries so that the naming scheme is correct

Directories should simply be identify, kad, and so on, instead of libp2p-identify, libp2p-kad, etc.
The crate names, however, should have the libp2p- prefix.

Implement PubSub/FloodSub

See: https://github.com/libp2p/go-floodsub

Use of `FuturesUnordered`may prevent `SwarmFuture` from making progress.

PR #163 replaced various uses of vectors of futures with FuturesUnordereds and PR #169 wanted to revert those changes because the change resulted in problems, namely that SwarmFuture could stall. One example is starting the echo-server example, connecting to it and disconnecting again, without sending valid data:

TRACE 2018-05-09T15:46:10Z: mio::poll: registering with poller
TRACE 2018-05-09T15:46:10Z: tokio_threadpool::builder: build; num-workers=8
DEBUG 2018-05-09T15:46:10Z: tokio_reactor::background: starting background reactor
TRACE 2018-05-09T15:46:10Z: mio::poll: registering with poller
DEBUG 2018-05-09T15:46:10Z: libp2p_tcp_transport: Now listening on /ip4/0.0.0.0/tcp/10333
TRACE 2018-05-09T15:46:10Z: libp2p_swarm::swarm: Swarm listening on /ip4/0.0.0.0/tcp/10333
Now listening on "/ip4/0.0.0.0/tcp/10333"
DEBUG 2018-05-09T15:46:10Z: tokio_core::reactor: consuming notification queue
DEBUG 2018-05-09T15:46:10Z: tokio_core::reactor: loop poll - Duration { secs: 0, nanos: 46334 }
DEBUG 2018-05-09T15:46:10Z: tokio_core::reactor: loop time - Instant { tv_sec: 26326, tv_nsec: 680689193 }
DEBUG 2018-05-09T15:46:10Z: tokio_core::reactor: loop process, Duration { secs: 0, nanos: 69532 }
# first `echo "" | nc localhost 10333` from a separate console
TRACE 2018-05-09T15:46:31Z: tokio_reactor: event Readable Token(0)
DEBUG 2018-05-09T15:46:31Z: tokio_reactor: loop process - 1 events, 0.000s
DEBUG 2018-05-09T15:46:31Z: tokio_core::reactor: loop poll - Duration { secs: 20, nanos: 709840848 }
DEBUG 2018-05-09T15:46:31Z: tokio_core::reactor: loop time - Instant { tv_sec: 26347, tv_nsec: 390631598 }
DEBUG 2018-05-09T15:46:31Z: tokio_core::reactor: loop process, Duration { secs: 0, nanos: 92083 }
DEBUG 2018-05-09T15:46:31Z: libp2p_tcp_transport: Incoming connection from /ip4/127.0.0.1/tcp/54306
DEBUG 2018-05-09T15:46:31Z: libp2p_swarm::transport::upgrade: Starting protocol negotiation (listener)
TRACE 2018-05-09T15:46:31Z: mio::poll: registering with poller
TRACE 2018-05-09T15:46:31Z: tokio_reactor: event Readable | Writable Token(4194305)
DEBUG 2018-05-09T15:46:31Z: tokio_reactor: loop process - 1 events, 0.000s
DEBUG 2018-05-09T15:46:31Z: tokio_core::reactor: loop poll - Duration { secs: 0, nanos: 106871 }
DEBUG 2018-05-09T15:46:31Z: tokio_core::reactor: loop time - Instant { tv_sec: 26347, tv_nsec: 391297053 }
DEBUG 2018-05-09T15:46:31Z: tokio_core::reactor: loop process, Duration { secs: 0, nanos: 74241 }
# CTRL-C the connection
TRACE 2018-05-09T15:46:46Z: tokio_reactor: event Readable | Writable | Hup Token(4194305)
DEBUG 2018-05-09T15:46:46Z: tokio_reactor: loop process - 1 events, 0.000s
DEBUG 2018-05-09T15:46:46Z: tokio_core::reactor: loop poll - Duration { secs: 15, nanos: 152174476 }
DEBUG 2018-05-09T15:46:46Z: tokio_core::reactor: loop time - Instant { tv_sec: 26362, tv_nsec: 543692607 }
DEBUG 2018-05-09T15:46:46Z: tokio_core::reactor: loop process, Duration { secs: 0, nanos: 192847 }
TRACE 2018-05-09T15:46:46Z: mio::poll: deregistering handle with poller
DEBUG 2018-05-09T15:46:46Z: tokio_reactor: dropping I/O source: 1
DEBUG 2018-05-09T15:46:46Z: libp2p_swarm::transport::upgrade: Error while negotiated protocol upgrade: Custom { kind: Other, error: MultistreamSelectError(IoError(Custom { kind: BrokenPipe, error: StringError("unexpected eof") })) }
DEBUG 2018-05-09T15:46:46Z: libp2p_swarm::connection_reuse: error while upgrading listener connection: Custom { kind: Other, error: MultistreamSelectError(IoError(Custom { kind: BrokenPipe, error: StringError("unexpected eof") })) }
TRACE 2018-05-09T15:46:46Z: libp2p_swarm::swarm: Swarm received new connection on listener socket
 WARN 2018-05-09T15:46:46Z: libp2p_swarm::swarm: Error in listener upgrade: Custom { kind: Other, error: MultistreamSelectError(IoError(Custom { kind: BrokenPipe, error: StringError("unexpected eof") })) }
# second `echo "" | nc localhost 10333`from a separate console
TRACE 2018-05-09T15:47:11Z: tokio_reactor: event Readable Token(0)
DEBUG 2018-05-09T15:47:11Z: tokio_reactor: loop process - 1 events, 0.000s
# second CTRL-C
# third `echo "" | nc localhost 10333` from a separate console
TRACE 2018-05-09T15:47:35Z: tokio_reactor: event Readable Token(0)
DEBUG 2018-05-09T15:47:35Z: tokio_reactor: loop process - 1 events, 0.000s
# third CTRL-C
# and so on ...

AFAICT this is because notifications of tasks of Futures that have been added to FuturesUnordered will not be propagated until FuturesUnordered::poll is called. Presumably this problem does not always show up on the happy path because more notifications trigger SwarmFuture::poll which in turn polls FuturesUnordered.

Stream Muxer Interface

See: https://github.com/libp2p/interface-stream-muxer

Implement top-level Node or "Host"

This is what brings it all together into a nice usable format.

See: https://github.com/libp2p/go-libp2p/tree/master/examples/libp2p-host

multiaddr_to_socketaddr test failing on master

https://github.com/paritytech/rust-libp2p/blob/710dec31108385e43e871aab50c385d1abdd0e24/libp2p-tcp-transport/src/lib.rs#L155-L161

This test has been there for a long time and it has suddenly started failing, probably after a cargo update.

Move SimpleProtocol to the facade crate

Made "difficult" because libp2p-relay uses it.

Remove the PeerStore from the API of identify and kademlia

Instead of using a Peerstore trait, use a trait or an API more specific to identify or kademlia.

Is that ready to use & when?

Add support for /p2p/<hash> in addition to /ipfs/<hash>

IPFS nodes seem to ignore our nodes

When we connect to an IPFS Go node, the remote opens three substreams but doesn't use them at all (doesn't send any data on them).

PR #120 allows one to make Kademlia requests to other nodes. I would expect that if we connect to a node through Kademlia, this node would want to get information about us.

Note that if two Rust nodes connect to one another, they will successfully identify each other through a substream. It's only when connecting to a Go node that things don't work.

I am not sure whether this behaviour from Go nodes is expected, or if it's a bug in our code.

cc @diasdavid @whyrusleeping

Peer Discovery Interface

See: https://github.com/libp2p/interface-peer-discovery