raiden-network/raiden-pathfinding-service

Repo for the Raiden Pathfinding Service

raiden-pathfinding-service's Issues

Catch connection errors

Problem Definition

Currently the PFS shows a big stacktrace when it cannot connect to the Ethereum node. We should catch this and show a helpful error message instead.

Actual Outcome

12-17 10:54:40 - pathfinder.cli - INFO - Starting Web3 client for node at http://parity.mainnet.ethnodes.brainbot.com:8545
Traceback (most recent call last):
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/site-packages/urllib3/connection.py", line 159, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw)
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/site-packages/urllib3/util/connection.py", line 57, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/site-packages/gevent/_socketcommon.py", line 208, in getaddrinfo
    return get_hub().resolver.getaddrinfo(host, port, family, type, proto, flags)
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/site-packages/gevent/resolver/thread.py", line 65, in getaddrinfo
    return self.pool.apply(_socket.getaddrinfo, args, kwargs)
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/site-packages/gevent/pool.py", line 159, in apply
    return self.spawn(func, *args, **kwds).get()
  File "src/gevent/event.py", line 381, in gevent._event.AsyncResult.get
  File "src/gevent/event.py", line 409, in gevent._event.AsyncResult.get
  File "src/gevent/event.py", line 399, in gevent._event.AsyncResult.get
  File "src/gevent/event.py", line 379, in gevent._event.AsyncResult._raise_exception
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/site-packages/gevent/_compat.py", line 47, in reraise
    raise value.with_traceback(tb)
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/site-packages/gevent/threadpool.py", line 281, in _worker
    value = func(*args, **kwargs)
socket.gaierror: [Errno 8] nodename nor servname provided, or not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/site-packages/urllib3/connectionpool.py", line 354, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/site-packages/urllib3/connection.py", line 181, in connect
    conn = self._new_conn()
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/site-packages/urllib3/connection.py", line 168, in _new_conn
    self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x110441cc0>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/site-packages/urllib3/connectionpool.py", line 638, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/site-packages/urllib3/util/retry.py", line 398, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='parity.mainnet.ethnodes.brainbot.com', port=8545): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x110441cc0>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/paul/.pyenv/versions/3.6.5/bin/pathfinder", line 11, in <module>
    load_entry_point('pathfinder', 'console_scripts', 'pathfinder')()
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/Users/paul/Work/raiden-pathfinding-service/pathfinder/cli.py", line 90, in main
    contract_data = get_contracts_deployed(int(web3.net.version), contracts_version)
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/site-packages/web3/net.py", line 21, in version
    return self.web3.manager.request_blocking("net_version", [])
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/site-packages/web3/manager.py", line 109, in request_blocking
    response = self._make_request(method, params)
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/site-packages/web3/manager.py", line 92, in _make_request
    return request_func(method, params)
  File "cytoolz/functoolz.pyx", line 232, in cytoolz.functoolz.curry.__call__
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/site-packages/web3/middleware/formatting.py", line 50, in apply_formatters
    response = make_request(method, params)
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/site-packages/web3/middleware/gas_price_strategy.py", line 18, in middleware
    return make_request(method, params)
  File "cytoolz/functoolz.pyx", line 232, in cytoolz.functoolz.curry.__call__
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/site-packages/web3/middleware/formatting.py", line 50, in apply_formatters
    response = make_request(method, params)
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/site-packages/web3/middleware/attrdict.py", line 18, in middleware
    response = make_request(method, params)
  File "cytoolz/functoolz.pyx", line 232, in cytoolz.functoolz.curry.__call__
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/site-packages/web3/middleware/formatting.py", line 50, in apply_formatters
    response = make_request(method, params)
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/site-packages/web3/middleware/normalize_errors.py", line 9, in middleware
    result = make_request(method, params)
  File "cytoolz/functoolz.pyx", line 232, in cytoolz.functoolz.curry.__call__
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/site-packages/web3/middleware/formatting.py", line 50, in apply_formatters
    response = make_request(method, params)
  File "cytoolz/functoolz.pyx", line 232, in cytoolz.functoolz.curry.__call__
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/site-packages/web3/middleware/formatting.py", line 50, in apply_formatters
    response = make_request(method, params)
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/site-packages/web3/middleware/exception_retry_request.py", line 80, in middleware
    return make_request(method, params)
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/site-packages/web3/providers/rpc.py", line 68, in make_request
    **self.get_request_kwargs()
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/site-packages/web3/utils/request.py", line 26, in make_post_request
    response = session.post(endpoint_uri, data=data, *args, **kwargs)
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/site-packages/requests/sessions.py", line 581, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/site-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/site-packages/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/Users/paul/.pyenv/versions/3.6.5/lib/python3.6/site-packages/requests/adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='parity.mainnet.ethnodes.brainbot.com', port=8545): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x110441cc0>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known',))

Expected Outcome

Should not show the stacktrace, but a nice error message.

Matrix listener

Add a matrix listener to get balance proofs and/or fee updates.

• Add matrix listener, connect it to PathfindingService
• Define BalanceProof message that includes all Locks
  • BalanceProof now includes locked_amount
• Define FeeUpdate message
• Define PathsRequest message #50
• Integrate message verification once the message format is specified
  • FeeInfo
  • BalanceProof
  • PathsRequest
• Remove model/BalanceProof and model/Lock classes.

API access to routing service

• Implement PathsResource.get()
• Test API interface

Make REST API optional

In #6 we agreed that a REST API is useful for testing but might be unnecessary once the matrix transport works. It should then be made optional and disabled by default.

Remove hard-coded contracts file

Currently the CLI still uses a hardcoded file to get the contracts ABIs and bytecode.

raiden-pathfinding-service/pathfinder/cli.py

Line 108 in 35306f6

contracts_path = os.path.join(module_dir, 'contract', 'contracts_12032018.json')

The file has been removed some commits ago.

• Update to use the raiden-contracts provided ContractManager
• Add more tests for the CLI

Go through all ToDos and fix or create issues

Check the codebase for open TODOs or FIXMEs and fix them or create issues.

pathfinding should return fee estimate

Currently the get_paths function only returns the sequence of nodes for routing the payment. It should also include the estimated fee for this payment.

See https://raiden-network-specification.readthedocs.io/en/latest/pathfinding_service.html#id6

• Add fee estimation to output
• Adapt tests
• Test Rest Api

PFS provides disjunctive paths

Description

As Josh I want to be able to

get disjunctive paths from the PFS

So that I can

better my chances of my payment going through.

Acceptance Criteria

• One can set "disjunctive path" as a argument at request with the PFS API Avoid overlaps to a certain degree by default
• The PFS returns at least two disjunctive paths as JSON objects

Related story / issue

Reviewer

build Request_Paths message for Raiden client -> PFS

Integration-Test with currently available cross-project objects

• Network popularization via API in network_service.py
• small scale, edge case networks sim (or verify that current sim is hard enough)

Number of routes returned by PFS can be configured

Description

As Josh I want to be able to

set the routes that should be returned by the PFS

So that I can

get an optimized answer in case I want this.

Acceptance Criteria

• There is an argument one can pass with the request to the PFS that sets, how many routes the PFS provides
• The argument has a default
• The PFS server would react according to the configured argument and send the requested number routes (or a number in case the routes are not sent yet)

Story points

2

Tasks

• Check implementation
• Define default number of routes

[Spec] Public REST endpoints or pure matrix communication

Once we know more about the Matrix transport we have to decide whether we will provide a public REST API or just accept communication over Matrix.

Unit test coverage for current pathfinding algorithm

I'm thinking of a simple populate_networks-like fixture that creates a token network of, say, 8 nodes and simple channels between them.

This would include

• Testing proper fee prioritization
• Testing the custom disjoint heuristic (basically "disjoint paths when possible")
• Testing hop bias (low bias = fee-oriented routing, high bias = hop-oriented routing)
• Testing failure of isolated nodes or partitions
• etc

Build data base for future token-network analytics

Address already used in REST API tests

In the tests I see some errors like:

127.0.0.1 - - [2018-03-12 18:15:52] "PUT /api/1/balance HTTP/1.1" 400 200 0.000960
.Traceback (most recent call last):
  File "/Users/paul/Work/raiden-pathfinding-service/venv/lib/python3.6/site-packages/gevent/greenlet.py", line 536, in run
    result = self._run(*self.args, **self.kwargs)
  File "/Users/paul/Work/raiden-pathfinding-service/venv/lib/python3.6/site-packages/gevent/baseserver.py", line 360, in serve_forever
    self.start()
  File "/Users/paul/Work/raiden-pathfinding-service/venv/lib/python3.6/site-packages/gevent/baseserver.py", line 304, in start
    self.init_socket()
  File "/Users/paul/Work/raiden-pathfinding-service/venv/lib/python3.6/site-packages/gevent/pywsgi.py", line 1450, in init_socket
    StreamServer.init_socket(self)
  File "/Users/paul/Work/raiden-pathfinding-service/venv/lib/python3.6/site-packages/gevent/server.py", line 127, in init_socket
    self.socket = self.get_listener(self.address, self.backlog, self.family)
  File "/Users/paul/Work/raiden-pathfinding-service/venv/lib/python3.6/site-packages/gevent/server.py", line 138, in get_listener
    return _tcp_listener(address, backlog=backlog, reuse_addr=cls.reuse_addr, family=family)
  File "/Users/paul/Work/raiden-pathfinding-service/venv/lib/python3.6/site-packages/gevent/server.py", line 229, in _tcp_listener
    sock.bind(address)
OSError: [Errno 48] Address already in use: ('localhost', 5002)
Mon Mar 12 18:15:53 2018 <Greenlet at 0x1096baaf8: <bound method BaseServer.serve_forever of <WSGIServer at 0x1097482e8 address=localhost:5002>>> failed with OSError

API access to token network fee updates

• Implement ChannelFeeResource.put() and connect it to the TokenNetwork logic
• Test API interface

[Spec] Client address for payments

Question here is if it can be assumed that a client has the same address in the token network it searches a route in as in the RDN network which is used to pay for the services of the PFS.

web3 request retry with backoff

When a web3 request fails for some reason we should try to reconnect a couple of times before crashing the PFS.

Continuous deployment

A goal of issue #82 was to create a continuous deplyoment pipeline. This turned out to be more difficult as expected, therefore this issue will progress on this.

The current setup is at https://github.com/palango/services-dev

Tasks

• Setup deployment pipeline

Add tests for cli.py

cli.py has a low test coverage (see https://codecov.io/gh/raiden-network/raiden-pathfinding-service/src/master/pathfinder/cli.py). Add more tests to increase coverage of the cli interface.

• increase coverage of cli.py
• verify that cli parameters reach the PathfindingService constructor

Add replay protection for RequestPaths

While building RequestPaths #50, the question arose if PFS payment would be done agnostically. If so, replay protection is needed.

Handle common errors better

Currently errors like an invalid password for matrix are displayed as:

INFO:raiden_libs.blockchain:Starting blockchain polling (interval 2s)
CRITICAL:pathfinder.pathfinding_service:Unhandled exception. Terminating the program
Traceback (most recent call last):
  File "/Users/paul/Work/raiden-pathfinding-service/venv-test/lib/python3.6/site-packages/gevent/greenlet.py", line 536, in run
    result = self._run(*self.args, **self.kwargs)
  File "/Users/paul/Work/raiden-pathfinding-service/venv-test/lib/python3.6/site-packages/raiden_libs/transport/matrix.py", line 82, in _run
    self.connect()
  File "/Users/paul/Work/raiden-pathfinding-service/venv-test/lib/python3.6/site-packages/raiden_libs/transport/matrix.py", line 43, in connect
    self.room = self.client.join_room(self.room_name)
  File "/Users/paul/Work/raiden-pathfinding-service/venv-test/lib/python3.6/site-packages/matrix_client/client.py", line 287, in join_room
    response = self.api.join_room(room_id_or_alias)
  File "/Users/paul/Work/raiden-pathfinding-service/venv-test/lib/python3.6/site-packages/matrix_client/api.py", line 192, in join_room
    return self._send("POST", path)
  File "/Users/paul/Work/raiden-pathfinding-service/venv-test/lib/python3.6/site-packages/matrix_client/api.py", line 685, in _send
    code=response.status_code, content=response.text
matrix_client.errors.MatrixRequestError: 404: {"errcode":"M_NOT_FOUND","error":"Room alias #test:transport01.raiden.network not found"}

Common cases should be handled and display nice error messages.

• Matrix errors
• Ethereum node errors

[tmp] PFS Planning

Copied from raiden-network/raiden#2704

PFS planning

To plan the integration of the PFS I want to get a discussion started here. Below is a four step plan that marks useful milestones for the PFS and its integration and provides time for parallel development of requirements for stages (e.g. matrix room for sharing balance proofs (which will be used for the MS as well), decision on fees in the client).

Before specing this out further I'd like to get some feedback.

Stage 1 - Feature parity

PFS

• Index events, recreate topology from them
• Provide endpoint to get routes

Raiden

• Remove internal topology state
• Request route information during payment initialisation
• Fixtures for spawning the PFS during tests

Stage 2 - Balance Updates

PFS

• Include endpoint to provide balance updates
• Include balances in route calculations

Raiden

• Send balance updates to PFS

Stage 3 - Integrate Fees

PFS

• Include endpoint to provide fee updates
• Include fees in route calculations

Raiden

• Requirement: have fees implemented
• Send fee updates to PFS

Stage 4 - Payed PFS

PFS

• Decide on payment method.

[Spec] Should PFS provide suitable partners to connect to for new nodes

An open question is how new nodes in a token network find suitable partners.

One idea is to use the PFS to return the a list of suitable partners, possible configurable with the desired deposit and the desired number of channels.

Basic request to PFS server

Description

As Josh I want to be able to

know that the PFS I selected works

So that I can

be sure that I will get routes later if required.

Acceptance Criteria

• PFS answers on general info route
• PFS returns settings/infos (versionsnummer, IP, betreiber)
• PFS can be run locally and remotely
• PFS can be updated easily

Related story / issue

• #80

Story points

Tasks

• Implement endpoint
• Test endpoint
• Setup meeting with Ulo
• Define requirements for deployment
• Setup deployment pipeline

Automatic deployment

As part of #82 we want to setup automatic deployment from the master branch.

The current setup is at https://github.com/palango/services-dev

User defines PFS

Description

As Josh I want to be able to

set the PFS in the Raiden CLI

So that I can

use it later to get routes.

Acceptance Criteria

• PFS address can be set using the Raiden CLI
• PFS address is stored inside the Raiden state to be used later

Related story / issue

Reviewer

Setup code climate

We want to test if codeclimate.com provides useful tips to improve our code.

We need to set it up so we automatically get reports on PRs.

Pathfinding algorithms

Research and implement suitable pathfinding algorithms on different network topologies and scenarios.

Requirements are:

• only return routes that have capacity >= transfer amount
• minimize/maximize some route property (might be fees, hops, some combination, ...)
• return multiple routes, especially (partially) non-overlapping

Check network existence when following specified networks

When following specified networks the code must make sure that these networks exist and show a warning to the user otherwise.

Cleanup

Some short term tasks to clean the repo and adjust the code style to Raiden conventions.

• Setup linter like raiden (pylint, flake8)
• Setup isort
• Update dependencies
• Check if no_ssl_verification wrapper is still necessary
• Check for unused dependencies
  • Remove numpy dependency
• Pin dependencies

PFS provides the shortest path

Description

As Josh I want to be able to

ask the PFS for a path between two nodes

So that I can

see the route between two nodes.

Acceptance Criteria

• API endpoint for path requests via HTTP
• Path is returned as JSON Object
• Interface is accepted by the client team
• List with addresses which represent the route (A -> B -> C)
• Shortest route provided
• Tested via HTTP endpoint
• Unit tests for routing

Story Points

13

Tasks

• Define interface
• Clarify interface with Lefteris
• Start Http interface with PathfindingService
• Set default fee to 0
• Check coverage of existing tests, fix if necessary

Related story / issue

Spec Open Questions

These were removed from the spec.

• How do clients open channels? Additional service offered by the pathfinding server?
• Is it OK to assume that clients address in the RDN token network is the same as in the (possibly) different network it asks the pathfinding service for a path?
• Are the updating endpoints publicly available or just for the matrix channel listener?

@palango Please address them one by one and incorporate the correct answers into the spec and eventually into the code.

Implement replay protection nonce for update_fee

• Starting in token_network.py, l. 170, work through the test pipeline
• Update pathfinding_service.py and validate the messages

PFS listens to Blockchain Events

Description

As Josh I want the PFS to

show me the network topology

So that it can

see, if it matches with reality.

Acceptance Criteria

• TokenNetworkCreated events are observed and create new networks inside the PFS
• ChannelOpen opens new channels in the network
• ChannelClose closes channels in network
• ChannelNewDeposit events are observed and increase the capacity for the respective channel
• Topology can be shown to the user
• Data is not stored
• Data is pulled from the blockchain everytime PFS gets started
• All tests are updated and the BlockchainListener tested

Story Points

13

Tasks

• Move BlockchainListener from explorer: https://github.com/raiden-network/explorer/blob/master/backend/metrics_backend/utils/blockchain_listener.py
• Update blockchain listener tests from libs to PFS
• Update Tests (if necessary) https://github.com/raiden-network/raiden-pathfinding-service/tree/master/pathfinder/tests
• Check if MockListener needs updates to new API
• Use networkx export for topology

Related story / issue

• #73

Extract channel_descriptions into fixture

We should extract the channel_descriptions for the commonly used network(s) into fixtures.

Blockchain listener

Integrate a blockchain listener to react on new events.

• Initial integration
• Testing

Events pulled out of order

The BlockchainListener currently queries the different events one after another. This caused some problems in the backend of the explorer, as it can lead to an inconsistent view of the network (channel can be opened and closed multiple times in the period that is queried).

It can be fixed by querying all events together.

Network image creation seems to have duplicate nodes

Problem Definition

When I look at the graps generated from the networks they seem to have a lot of common nodes, it seems like the canvas is not cleared before drawing the next graph.

Fee representation convention

Decide on how to represent fees, i.e. use percent (with float), use basis points, come up with an own convention.

User defines PFS

Description

As Josh I want to be able to

set the PFS in the Raiden CLI

So that I can

use it to find routes later.

Acceptance Criteria

• PFS selection can be set in the CLI
• PFS selection is stored somewhere in the Raiden client for later usage

Related story / issue

Reviewer

PFS provides multiple paths

Description

As Josh I want to be able to

get multiple options for routing a payment

So that I can

try another route when one fails.

Acceptance Criteria

• PFS endpoints provides multiple routes for a given payment
• List of routes as JSON
• Same endpoint as #83
• Routes sorted by the number of hops increasing

Story Points

5

Related story / issue

• #83

Tasks

• Define interface
• Check interface with Lefteris
• Check coverage, add tests if necessary
• Test proper sorting of routes