MetaMask Light Client Development.
In one line: Ethereum on libp2p.
MetaMask Light Client Development
License: MIT License
MetaMask Light Client Development.
In one line: Ethereum on libp2p.
Mustekala Roadmap
As a first step, we want to have a few slices that we subscribe to and get the accounts balances and token balances.
Mustekala is the name of the Ethereum Light Client project of Metamask.
The current network devp2p is broken
The following 4-layer architecture comprises the project:
A more thoroughly discussion is in this document
The concept is straightforward: Get the data from an Ethereum node, dump it into IPFS, consume it in a browser peer.
More of this point in this document
A plethora of scaling challenges and use cases emerge from the MVP, including (but not limited to)
The comprehensive reference in this file
List of Sprints: Dates and Goals
Short sprint. Team retreat next week.
Consume and Visualize Data
section of #8Microservice: consume from parity to IPFS client
Improve kitsunet peer discovery over rendezvous
A list in memory where the new trie hashes (only the hashes) are written between blocks (+ its RPC)
slice(state_root, slice_root, depth)
walks from an state or storage trie node relative to the state root, until complete the given depth
.Implement a bridge returning for the RPC GetSlice()
method as done in go-ethereum
Linked to MetaMask/mesh-testing#53
DO NOT MERGE YET - experimental branch to work out stability issues in libp2p.
For the last few days I've been troubleshooting some stability issues with libp2p, I'll describe what those are and what the possible fixes for them are as well bellow:
The current issues with connection management in libp2p prevent it from being able to connect to more than ~10 simultaneous peers, I've observed that after we reach that threshold things become very unstable and we're no longer able to send messages over pubsub reliably, as well as maintain those connections open for any significant period of time (connection drops). This is due to several issues:
Physical (non muxed) connections don't get reused correctly, they instead get thrown away on each dial, which under certain situations can lead to connections piling up, which eventually backups the connection queue (connection backlog). The particular line where this might be happening is this - https://github.com/libp2p/js-libp2p-switch/blob/master/src/dial.js#L184.
Too many concurrent dials and low connection timeout. I'm not entirely sure whether this is a real issue just yet, but I've seen improvements lowering the number of concurrent dials done by libp2p-switch (https://github.com/libp2p/js-libp2p-switch/blob/master/src/transport.js#L11) to ~2 as well as increasing the timeout to ~2mins (https://github.com/libp2p/js-libp2p-switch/blob/master/src/transport.js#L15), in some cases. This is not conclusive and might be affected by other factors, but definitely something to keep an eye on.
Wrong way of closing connections. We don't seem to be closing connections properly in libp2p when the connection manager signals a disconnect, we do close muxed connection, but the physical connection is left dangling in some cases, leading to connection pile up. This should not happen, as destroying the muxed should in theory destroy the connection but I've seen a number of cases where this doesn't happen. In any case, I believe we should have an explicit Connection.destroy
in the interface-connection
code to ensure that the connection closes properly in all cases.
No way of detecting stale connections. Libp2p supports a variety of transports, and not all of those transports have a way of detecting weather the connection has gone stale (other side died/dropped), for this I believe we need a heartbeat mechanism that would detect and disconnect stale/dangling connections and perform the required/correct steps needed to properly clean them up.
Connection.destroy()
in interface-connection
One thing that puzzled me while troubleshooting this, was that our deployed mesh doesn't seem to be running into this issues and we're able to maintain a stable mesh with well over ~100 simultaneous peers, but once I tried adding a circuit-relay node to the mesh everything would start falling apart. The reason is that we're keeping the amount of concurrent connections in libp2p-wrtc-star to 4, this due to wrtc specific limitations in the browser. This effectively mitigates the issues listed above when using wrtc, but shows up as soon as another transport is used, in this case websocket, which is used to connect to the relay node.
Disclaimer: It's a bit difficult to pinpoint this issues when so many things are happening at the same time, hence some of my observations/conclusions might be off, but I believe I've identified at least some of the issues. The action items above should increase stability considerably.
Build a stable libp2p based kitsunet mesh.
We should have a method to batch multiple slice calls into one request.
v1.8.17
So we have a slice of the ethereum state valid for a given root (and subsequently a block), this exercise points to get the best algorithm to determine the difference of the same slice for a different rule. This way, we can schedule the trie-node requests to update our slice. Get it?
.
Yet another roadmap...
Moving from MetaMask/eth-ipfs-browser-client#1
Divided in five big areas
(go-ipfs)
(js-ipfs)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.