etclabscore/jade-service-runner

👟 Jade Service Runner

Jade Service Runner - A JSON-RPC Service Manager

Jade Service Runner

An awesome JSON-RPC service manager!
View Demo · Report Bug · Request Feature

Table of Contents

• About The Project
  • Better dApp Development
  • Supported Services
• Getting Started
  • Prerequisites
  • Installation
• Usage
  • JSON-RPC API Documentation
  • Extending services
• Contributing
• Resources

About The Project

Service Runner is an opinionated JSON-RPC service manager, that provides daemonization, installation, and discovery for JSON-RPC based services.

Service Runner helps developers gain access to a user's locally run services. It provides users with a management and installation tool, that allows developers to discover services and reliably request access to those services.

Better dApp Development

Service Runner improves the dApp development cycle, by reducing the number of steps required for running services that are local to the user, in addition to associated with relying on locally running JSON-RPC services. To do this effectively, Jade Service Runner supports the following:

• Allows dApp developers to specify what services they'd like to use
• Provides defaults for the services to run
• Provides users with an easy installation path
• Provides reliable discovery of pre-existing services run by the service runner
• Provides OpenRPC interface to the Service Runner functionality, as well as the underlying services
• Allows dApp developers the ability to retrieve reliable JSON-RPC connection information from the service
• Provides typed interfaces to develop applications against

Supported Services

Currently it supports core-geth with the following environments:

• mainnet (ETC)
• kotti
• ethereum
• goerli
• rinkeby

Getting Started

Prerequisites

• node v12.10.0 or greater.
• npm v6.10.3 or greater.

Installation

Install via npm package

npm install -g @etclabscore/jade-service-runner

Install the JavaScript client:

npm install @etclabscore/jade-service-runner-client

Then require it into any module.

const { ServiceRunner } = require("@etclabscore/jade-service-runner-client");
const ERPC = require("@etclabscore/ethereum-json-rpc");
const serviceRunner = new ServiceRunner({ transport: { type: "http", port: 8002, host: "localhost" } });
const serviceName = "core-geth";
const successful = await serviceRunner.installService(serviceName, "1.11.2");
if (successful === false) throw new Error("Service not installed")
const serviceConfig = serviceRunner.start(serviceName, "1.11.2", "kotti");
const erpc = new ERPC(serviceConfig);
erpc.getBalance("0x0DEADBEEF");

To run the jade-service-runner:

jade-service-runner

Usage

JSON-RPC API Documentation

You can view the API documentation here.

Extending services

You can extend jade-service-runner with your own configuration via the command line interface

jade-service-runner -c extended-jade-service-runner-config.json

Roadmap

See the open issues for a list of proposed features (and known issues).

Contributing

How to contribute, build and release are outlined in CONTRIBUTING.md, BUILDING.md and RELEASING.md respectively. Commits in this repository follow the CONVENTIONAL_COMMITS.md specification.

License

Apache License 2.0

Resources

• Jade.Builders
• Jade Specification

jade-service-runner's Issues

JSON-RPC request stays pending

Describe the bug
JSON-RPC requests stays pending, then timeout.

To Reproduce
Steps to reproduce the behavior:

1. Start Service Runner
2. Go to Postman
3. create request
4. See error

Expected behavior
Expect response from service runner

Screenshots

Desktop (please complete the following information):

• OS: OSX
• Browser: Postman

environments typo in openrpc.json for listServices(filter)

jade-service-runner/openrpc.json

Line 62 in a9f537c

"enviornments": {

fix broken link service-runner.jade.builders

Describe the bug
service-runner.jade.builders should link to gh pages. Currently, the deploy does not include a CNAME file.

Expected behavior
When building the typedocs, we should echo "service-runner.jade.builders" > docs/CNAME.
example of this here

passing config errors

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

 jade-service-runner -c extended-jade-service-runner-config.json
(node:78367) UnhandledPromiseRejectionWarning: Error: ENOENT: no such file or directory, open './build/src/service-runner-config.json'
(node:78367) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:78367) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Expected behavior
should see no error and jade-service-runner should run

Resourcces

Heres what extended-jade-service-runner-config.json looks like

{
  "services": [
    {
      "name": "multi-geth-dev",
      "rpcPort": "${DYNAMIC_TCP_PORT_1}",
      "version": "1.9.0",
      "environments": [
        {
          "name": "dev",
          "args": {
            "start": [
              "--dev",
              "--datadir",
              "${SERVICE_DIR}/multi-geth/1.9.0/dev",
              "--rpc",
              "--rpcport",
              "${DYNAMIC_TCP_PORT_1}",
              "--rpccorsdomain",
              "*"
            ],
            "stop": [],
            "teardown": []
          }
        }
      ],
      "os": {
        "osx": {
          "commands": {
            "setup": [
              {
                "cmd": "chmod",
                "args": [
                  "+x",
                  "${SERVICE_DIR}/geth"
                ]
              }
            ],
            "start": "${SERVICE_DIR}/geth",
            "stop": "",
            "teardown": ""
          },
          "assets": [
            "https://github.com/multi-geth/multi-geth/releases/download/v1.9.0/multi-geth-osx.zip"
          ]
        },
        "linux": {
          "commands": {
            "setup": [],
            "start": "geth",
            "stop": "",
            "teardown": ""
          },
          "assets": [
            "https://github.com/multi-geth/multi-geth/releases/download/v1.9.0/multi-geth-linux.zip"
          ]
        }
      }
    }
  ]
}

requests not working from browser

Describe the bug
From the browser, requests do not complete.

To Reproduce

1. use pristine-cli to create a new pristine-typescript-react project.
2. npm install @etclabscore/jade-service-runner-client

const client = new JadeServiceRunnerClient({
  transport: {
    port: 8002,
    host: "localhost"
  }
));
await client.installService("multi-geth", "1.9.0");

4. see request does not complete
5. see screenshots

Expected behavior
Request should complete in the browser

Screenshots

Create a issue template for adding a new service to the config

ethereum network not working

Describe the bug
erroring that --ethereum argument doesnt exist

Solution
remove --ethereum from the configuration for the ethereum environment

Feat: Discussion about Jade Service Runner multiple version shared resources

Jade Service Runner should support multiple version resource reuse. The issue is that many types of services/api/programs might need to share the same resources or sets of resources between versions. Currently there is no way to accomplish this, outside of a hard reset for services. A tangential problem that we encounter is interdependent services. I'm not saying we should support this, but it is a topic for discussion. So I'll try and address the concerns below.

Problem 1:
We have a resource environment called a service. Within that environment, you can have many different versions. The versions might need to share resources, how do you enable the versions to share. How do you also restrict sharing to some but not all versions/env?

Problem 2:
Services are their own environment. How do you depict a dependency between services?

Examples
Real Life: Multi-geth 1.9.2 is different than 1.9.0 , but would like to share the data dir.

Real Life: Jade Signer relies on the same data dir as multi-geth x.x.x

Describe the solution you'd like
I believe a solution to Problem 1, might take the shape of dynamic shared resource allocation where possible. The following I believe might hold true.

1. Define at the top of a service config a set of shared resource descriptions/sub_paths for resource
2. Define a valid version interval for that service version to be a part of a shared config
3. Use service runner to resolve and define paths within the service manifest file such that
   service runner can have virtual paths that map versions to directories.
4. If a service is cleaned up, check to see if any other versions of that service are relying on the shared asset before removing all references to the shared asset.

Cavet
The above approach makes a few assumptions.

1. That the resource under question is available to be remapped to a autogenerated shared resource directory.
2. The directory is not generated at some point during runtime and needs to then be dynamically added to a shared config. It assumes the root path is known

The problems I see with the above is that it gets kind of messy, resolving paths for shared resources. If you could restrict the class of shared resources to being arguments for services, this would make things easier. However, I don't believe that it's the best experience for a developer, due to the need to potentially share resources due to dynamic generation of said resources.

Curious what people think. Below is a gist of proposed schema changes to make the above work. It's all very tenative and not super high priority relative to the other stuff, but worth collecting our thoughts for later.

https://gist.github.com/zcstarr/00755ac35fc6649d76c40c5bdd3ce0a9

invalid args (noops) are ignored, not errored on

Describe the bug
Invalid arguments provided to the command are ignored, instead of causing an error.

To Reproduce

$ jade-service-runner wow such args
info: Service Runner port starting on [object Object] {"service":"ServiceRunner","context":"startServiceRunner","path":"/"}
info: Service Runner started on [object Object] {"service":"ServiceRunner","context":"startServiceRunner","path":"/"}

Expected behavior
Error, invalid use.

Desktop (please complete the following information):

• OS: linux

Additional context

$ npm -v && node -v && jade-service-runner --version
6.12.1
v13.1.0
1.7.4

multiple versions in config errors

Describe the bug
after adding another version of multi-geth to the config, it errors

$ jade-service-runner 
error: [object Object] {"service":"ServiceRunner","context":"Config"}
error: Could not start service runner. {"service":"ServiceRunner","context":"CLI"}

add a note about supported services

Describe the bug
A clear and concise description of what services service-runner supports out of the box

Solution
Currently it supports multi-geth

with the following environments

kotti
mainnet
ethereum
rinkeby
goerli

Update to latest core-geth

Describe the bug
Need latest core geth 1.11.15

add windows binary for multigeth

Describe the bug
As a User, I'd like to run this on windows but the binary is missing in the configuration.

Config is required, but treated as optional

jade-service-runner/src/cli/index.ts

Line 12 in 9953399

"-c, --config <configFile>",

+

jade-service-runner/src/cli/commands.ts

Line 20 in 9953399

if (prog.config) { extendedConfig = await fs.readJSON(prog.config); }

Feat: Service Runner needs to support resolving bad Manifest files

Currently discrepancies between the manifest file and services can cause listing services to break. More importantly service runner needs a safe way for users to uninstall packages, and flag configuration mismatch packages.

The use case is installing a package, updating config so the package no longer exists, then attempting to show that there is a mismatch that is problematic. This would allow service runner ui implementors to expose a way for clients to rectify the issue, by uninstalling or finding a suitable configuration.

This will become more important when the specification for a decentralized service registry is created.

Expected behavior is as follows.
A programatic uninstall api
A programatic possibly RPC available configuration mismatch api
listService ignores missing service configurations

add Jenkinsfile

Describe the solution you'd like
Add Jenkinsfile to:

• build on each OS and produce an nexe build.
• run tests
• run release (circle)

ECONNREFUSED when -d=/external/media/

Describe the bug
jade-service-runner fails to begin managing services when data dir (-d) is configured at an external path, like a USB flash drive.

To Reproduce
Something equivalent to:

$ JADE_DEBUG=1 jade-service-runner -d /media/ia/SANDISK128/jade-services &> ~/dev/meowsbits/stargate-sg1/jade-service-runner.debug.log

Expected behavior
Program should run without ECONNREFUSED errors.

Just for reference and comparison, the following works ok for me:

$ JADE_DEBUG=1 jade-service-runner -d ~/jade-services &> ~/dev/meowsbits/stargate-sg1/jade-service-runner.debug.log

Screenshots

debug: EXIT -  processing event:exit  service: multi-geth env: goerli {"service":"ServiceRunner","context":"ServiceProcessManager","path":"/"}                                               
info: {} {"service":"ServiceRunner","context":"ServiceProcessManager","path":"/"}                                                                        
debug: EXIT -  processing event:exit  service: multi-geth env: kotti {"service":"ServiceRunner","context":"ServiceProcessManager","path":"/"}                                                
info: {} {"service":"ServiceRunner","context":"ServiceProcessManager","path":"/"}                                                                        
debug: connect ECONNREFUSED 127.0.0.1:42205 {"errno":-111,"code":"ECONNREFUSED","syscall":"connect","address":"127.0.0.1","port":42205,"service":"ServiceRunner","context":"Util","path":"/"}             
debug: initial health check in 1117.369418732048ms 1572305111313 < 1572305011325 {"service":"ServiceRunner","context":"ServiceProcessManager","path":"/"}                                                     
debug: connect ECONNREFUSED 127.0.0.1:38907 {"errno":-111,"code":"ECONNREFUSED","syscall":"connect","address":"127.0.0.1","port":38907,"service":"ServiceRunner","context":"Util","path":"/"}
debug: initial health check in 1407.8949842745567ms 1572305111323 < 1572305011325 {"service":"ServiceRunner","context":"ServiceProcessManager","path":"/"}
debug: connect ECONNREFUSED 127.0.0.1:42205 {"errno":-111,"code":"ECONNREFUSED","syscall":"connect","address":"127.0.0.1","port":42205,"service":"ServiceRunner","context":"Util","path":"/"}            
debug: initial health check in 1645.8415776538045ms 1572305111313 < 1572305012444 {"service":"ServiceRunner","context":"ServiceProcessManager","path":"/"}                                                   
debug: connect ECONNREFUSED 127.0.0.1:38907 {"errno":-111,"code":"ECONNREFUSED","syscall":"connect","address":"127.0.0.1","port":38907,"service":"ServiceRunner","context":"Util","path":"/"}
debug: initial health check in 1764.3346258361723ms 1572305111323 < 1572305012744 {"service":"ServiceRunner","context":"ServiceProcessManager","path":"/"}                                   
debug: connect ECONNREFUSED 127.0.0.1:42205 {"errno":-111,"code":"ECONNREFUSED","syscall":"connect","address":"127.0.0.1","port":42205,"service":"ServiceRunner","context":"Util","path":"/"}
debug: initial health check in 2668.503618736497ms 1572305111313 < 1572305014091 {"service":"ServiceRunner","context":"ServiceProcessManager","path":"/"} 
debug: connect ECONNREFUSED 127.0.0.1:38907 {"errno":-111,"code":"ECONNREFUSED","syscall":"connect","address":"127.0.0.1","port":38907,"service":"ServiceRunner","context":"Util","path":"/"}
debug: initial health check in 2854.2840118644617ms 1572305111323 < 1572305014511 {"service":"ServiceRunner","context":"ServiceProcessManager","path":"/"}
debug: connect ECONNREFUSED 127.0.0.1:42205 {"errno":-111,"code":"ECONNREFUSED","syscall":"connect","address":"127.0.0.1","port":42205,"service":"ServiceRunner","context":"Util","path":"/"}
debug: initial health check in 4046.9650614914553ms 1572305111313 < 1572305016761 {"service":"ServiceRunner","context":"ServiceProcessManager","path":"/"}
debug: connect ECONNREFUSED 127.0.0.1:38907 {"errno":-111,"code":"ECONNREFUSED","syscall":"connect","address":"127.0.0.1","port":38907,"service":"ServiceRunner","context":"Util","path":"/"}
debug: initial health check in 4590.837390145283ms 1572305111323 < 1572305017368 {"service":"ServiceRunner","context":"ServiceProcessManager","path":"/"} 
debug: connect ECONNREFUSED 127.0.0.1:42205 {"errno":-111,"code":"ECONNREFUSED","syscall":"connect","address":"127.0.0.1","port":42205,"service":"ServiceRunner","context":"Util","path":"/"}
debug: initial health check in 8227.690294874923ms 1572305111313 < 1572305020814 {"service":"ServiceRunner","context":"ServiceProcessManager","path":"/"}                                    
debug: connect ECONNREFUSED 127.0.0.1:38907 {"errno":-111,"code":"ECONNREFUSED","syscall":"connect","address":"127.0.0.1","port":38907,"service":"ServiceRunner","context":"Util","path":"/"}
debug: initial health check in 8525.544782872623ms 1572305111323 < 1572305021962 {"service":"ServiceRunner","context":"ServiceProcessManager","path":"/"} 
debug: connect ECONNREFUSED 127.0.0.1:42205 {"errno":-111,"code":"ECONNREFUSED","syscall":"connect","address":"127.0.0.1","port":42205,"service":"ServiceRunner","context":"Util","path":"/"}
debug: initial health check in 16807.744794546932ms 1572305111313 < 1572305029045 {"service":"ServiceRunner","context":"ServiceProcessManager","path":"/"}                                         
debug: connect ECONNREFUSED 127.0.0.1:38907 {"errno":-111,"code":"ECONNREFUSED","syscall":"connect","address":"127.0.0.1","port":38907,"service":"ServiceRunner","context":"Util","path":"/"}          
debug: initial health check in 16696.652171241407ms 1572305111323 < 1572305030499 {"service":"ServiceRunner","context":"ServiceProcessManager","path":"/"}
debug: HEALTH -  processing event:health  service: multi-geth env: goerli {"service":"ServiceRunner","context":"ServiceProcessManager","path":"/"}                                           
debug: connect ECONNREFUSED 127.0.0.1:42205 {"errno":-111,"code":"ECONNREFUSED","syscall":"connect","address":"127.0.0.1","port":42205,"service":"ServiceRunner","context":"Util","path":"/"}
debug: PENDING -  processing event:pending  service: multi-geth env: goerli {"service":"ServiceRunner","context":"ServiceProcessManager","path":"/"}                                         
debug: rendering service config for multi-geth {"service":"ServiceRunner","context":"RenderServiceTemplate","path":"/"}                                  
debug: rendered service config for multi-geth {"service":"ServiceRunner","context":"RenderServiceTemplate","path":"/"}                                                                       
debug: LAUNCH -  processing event:launch  service: multi-geth env: goerli {"service":"ServiceRunner","context":"ServiceProcessManager","path":"/"}       
debug: PENDING     pid: -1 status: pending hash:multi-geth_1.9.2_goerli {"service":"ServiceRunner","context":"ServiceProcessManager","path":"/"}                                             
debug: initial health check in 0ms 1572305131323 < 1572305031323 {"service":"ServiceRunner","context":"ServiceProcessManager","path":"/"}                
debug: EXIT -  processing event:exit  service: multi-geth env: goerli {"service":"ServiceRunner","context":"ServiceProcessManager","path":"/"}                                                            
debug: RUNNING     pid: undefined status: running hash:multi-geth_1.9.2_goerli {"service":"ServiceRunner","context":"ServiceProcessManager","path":"/"} 

Desktop (please complete the following information):

$ uname -a
Linux ubp52 5.2.9-050209-generic #201908160940 SMP Fri Aug 16 09:42:58 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
$ jade-service-runner -v
1.7.4
$ node -v && npm -v
v13.0.1
6.12.0

Additional context
I have a hunch this might be caused by or related to the client's (eg. multi-geth) handling IPC paths. Sometimes IPC paths on external drives aren't happy. Sometimes the path names are too long.

add note about configuring service directory in readme

you can configure the directory with -d option:

jade-service-runner -d ~/.services

Error handling in jade-service-runner `serviceManager`

Describe the bug
When making request to jade-service-runner to start new service. Upon an error returns
{ "id": 1, "jsonrpc": "2.0", "result": { "error": { "code": 6969, "message": "unknown error" } } }

To Reproduce
Steps to reproduce the behavior:
Make call to start service that isn't installed
{ "jsonrpc": "2.0", "method": "startService", "params": ["multi-geth", "1.9.2", "kotti"], "id": 1 }

Feat: Discussion about Jade Service Runner Behavior

Creating this issue to start a conversation about what the service runner should be solving, so
we can scope the initial version of the project properly, while keeping the end developer in mind.
The aim is to use this document to help write the readme and generate architecture docs, and figure out what v0 needs to do.

Jade Service Runner

Abstract high level goals

Jade Service Runner is a utility that makes it easy for dApp developers to allow end users to run and connect to their own services. In order to achieve this the service runner should make it easy to complete the following tasks. Either by hand or by code automation from the dApp.

User potential high level abstract goal

• find a service
• download the service
• install service
• support dApp configuration for a service ( user doesn't have to be too technical)
• run or connect to an existing dApp configuration for a service
• maintain service liveness

dApp developer potential high level abstract goals

• request access to open-rpc supported services
• use whatever shared open-rpc services are configured properly for the dapp
• specify service requirements
• provide easy to understand setup for users without heavy lifting
• have IDE useable discovery of RPC functionality (types)
• use generic or specific open-rpc services i.e. support mgeth ∩ geth ∩ parity vs parity ∪ mgeth ∪ geth
• easy to configure support for mobile(light client) or remote services

Where are we headed??

Let's try to imagine what an dApp developer would have to do to launch a shopping dApp.

Assumed Developer Concerns

1. Pick a or many services + version + env + args (limit security access) //dApps
2. Figure out where to get a service client
3. Decide how to run service (assume local/remote - (ethernode)/embed)
4. Assuming local for what follows
5. Decide to launch service if its not running or assume its running
6. If its not running and doesn't exist suggestion installation to a user
7. Decide how to figure out its running the right version + env (config) //think multi-geth/gorli vs main
8. Verify the service is up once its running // liveness (rpc.discover)
9. Suggest how to reboot the service if its crashed or gone missing
10. Develop the rest of the app
11. Security concerns

Jade Service Runner Addresses

1. With config? what are the params we should use?
2. Is there a list of support services or registry?
3. local is the aim but what about mobile support?
4. x
5. Who launches the service the dApp wants to run?
6. In an ideal world do we allow the dApp dev to specify installation instructions?
7. This is tricky, becaue forks of clients exist and if services are "non-blessed" how do we know what version ( Does this require a registry)?
8. This should be easy enough to do/track with service runner
9. The service runner daemon should take care of rebooting
10. Hopefully using the open-rpc generator for the services its requesting generates easy to use clients that can connect to the services requested easily.
11. Running many services and different versions manifest file has been suggested for specifying open-rpc files to use for generating clients, as well as potentially generating a generic

publish nexe binary builds for each os on release

Is your feature request related to a problem? Please describe.
I'd like a binary produced from the build so that I can distribute the binary through brew, apt-get, chocolatey and github release page.

Describe the solution you'd like
https://github.com/nexe/nexe provides a nice way to generate these binarys on each OS with our jenkins setup

Resources
Similar issue: open-rpc/generator#132

ม.ร.ว.ณัชพล วงษ์เปลี่ยนผัน นานณัชพล วงษ์เปลี่ยนผัน

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [e.g. iOS]
• Browser [e.g. chrome, safari]
• Version [e.g. 22]

Smartphone (please complete the following information):

• Device: [e.g. iPhone6]
• OS: [e.g. iOS8.1]
• Browser [e.g. stock browser, safari]
• Version [e.g. 22]

Additional context
Add any other context about the problem here.

installService example in README is missing version param

Describe the bug
installService example in README is missing version param

Screenshots

result has error in it

Describe the bug
Getting errors ends up in the result key for jsonrpc

To Reproduce
Steps to reproduce the behavior:

1. https://playground.open-rpc.org/?schemaUrl=https://services.jade.builders/
2. Click on listServices
3. Scroll down to the inspector
4. enter "foo" into params: []
5. See error

Expected behavior
I should see error at the root level:

{
   "jsonrpc": "2.0",
   "error": {}
}

Screenshots

Implement service/latest/ when you don't care about version

Is your feature request related to a problem? Please describe.
I often times dont care what version of the service is running, and want to leave it to the user to ensure that they have the versions that they want, giving warnings when there is missing or broken funcitonality.

Describe the solution you'd like
A way to hit a service irrespective of its version. ie: "give me any geth" or "give me any signer"

Describe alternatives you've considered
jade-package-manager??

remove semantic release deps

move them to circle ci only.

Feat: Jade Service Runner supports user defined RPC restrictions

Currently, the jade service runner is open for all request. We need a system that supports restricting access to RPC services on the users behalf.

The requirements from both sides are as follows.

1. Users can easily/ add remove restrictions
2. Service Runner Server - Apps can programmatically remove/add restrictions
3. It requires no change on the dApp developers end support hitting a service runner endpoint
   or non service runner endpoint.

It seems like the most straight forward way to achieve the above, is to create domain/ request origin based restrictions on access. This requires 0 changes to expectations that developers have about accessing a service. The most problematic issue here is the eventual issue of non domain originating request, which might lend itself to a separate method of authentication.

To have complete coverage of all cases, and support for peering might require, more work. It might make sense, in order to support p2p use cases that we use a capabilities security model , where a dApp might register with a users service runner, and be able to delegate that other service runners have the capability to receive requests from a 3rd party user.

The use case extends from the following.

User A is using a dApp that creates a client called SimpleMath, that communicates over a p2p transport. The client searches the service runner network for available SimpleMath service peers, retrieves a list of peers and makes request to the list of available peers. In order to support making request to these 3rd party SimpleMath service peers, you would be required to as member of the p2p network, add this user, user A to a whitelist as a peer service provider.

The options if you don't as a peer just open the service to all request, is to
A. whitelist all incoming request ip individually
B. open the service to all request / or only certain services endpoints to all request
C. use a capabilities scheme and request the user prove they have the request capabilities.

In option C, loosely the dApp would give the user the capability to make a request, and with the peering request would provide the peers with proof of the capability to make a request with regards to the dApp . I.e. the people/machines that are serving as peers are then accepting request to a service originating from the service, but not a specific origin domain. I imagine a scheme that is probably in line with OCAP-LD. and takes some inspiration from web3 Login EIP.

Short Term vs Long Term.
I feel as though its very unclear that when we get to peering, that people will not simply want to restrict bandwidth, and blacklist services entirely from services that might expose them to leaking personal data or that they don't want to make through the network.

The way forward seems.
Implement simple origin domain whitelisting/blacklisting that can be toggled per service, and see slowly how use cases drive future development in this area of restricted RPC calls.

Jenkins PR-merge job fails

Describe the bug
Every PR is having checks fail because the jenkins build is borked.

To Reproduce
Steps to reproduce the behavior:

1. Change readme
2. Make a pr

Expected behavior
all green

Screenshots

Update Multi-Geth version

With the Atlantis hard-fork coming up, master should ideally have Multi-Geth 1.9.2-stable or later which is the recommended minimum viable version for the Atlantis hard-fork. The current version is 1.9.0.

https://github.com/etclabscore/jade-service-runner/blob/master/client.ts

The automated release is failing 🚨

🚨 The automated release from the master branch failed. 🚨

I recommend you give this issue a high priority, so other packages depending on you could benefit from your bug fixes and new features.

You can find below the list of errors reported by semantic-release. Each one of them has to be resolved in order to automatically publish your package. I’m sure you can resolve this 💪.

Errors are usually caused by a misconfiguration or an authentication problem. With each error reported below you will find explanation and guidance to help you to resolve it.

Once all the errors are resolved, semantic-release will release your package the next time you push a commit to the master branch. You can also manually restart the failed CI job that runs semantic-release.

If you are not sure how to resolve this, here is some links that can help you:

• Usage documentation
• Frequently Asked Questions
• Support channels

If those don’t help, or if this issue is reporting something you think isn’t right, you can always ask the humans behind semantic-release.

---

Not verified

Something went wrong and the openrpc.json was not able to be verified.

---

Good luck with your project ✨

Your semantic-release bot 📦🚀

configurable and separate DIRs for service concerns

I want to be able to separate DIR concerns for default services, separating, say, executables from their/an associated data storage location (also transport, like IPC, paths).

For example: https://github.com/etclabscore/jade-service-runner/blob/master/src/service-runner-config.json#L9-L36

      "environments": [{
          "name": "mainnet",
          "summary": "Ethereum Classic Mainnet",
          "args": {
            "start": [
              "--classic",
              "--datadir",
              "${SERVICE_DIR}/multi-geth/1.9.2/classic",
              "--ipcpath",
              "${SERVICE_DIR}/multi-geth/1.9.2/classic/geth.ipc",
              "--port",
              "${DYNAMIC_TCP_PORT_2}",
              "--rpc",
              "--rpcport",
              "${DYNAMIC_TCP_PORT_1}",
              "--rpccorsdomain", 
              "*"
            ],
            "stop": [],
            "teardown": []
          },
          "health": {
           "interval": 20000, 
           "retries": 5,
           "port": "${DYNAMIC_TCP_PORT_1}",
           "protocol": "tcp"
           }
        },

I ran into this in a weird thing with a USB drive that wouldn't let me make anything executable in it, though I wanted to use it as my services dir. So I wound up extending the config with, eg. kotti2 and hand-setting some custom datadirs for new custom services (only custom by virtue of their DIR values).

But I'm wondering if we shouldn't think twice about defaultily keeping the executable and storage dir as neighbors; in "production" cases it's easily imaginable that I'd want to separate my storage system from my executable file system. At least to think twice about having them share configuration values by default.

So maybe something like a configuration where

          "args": {
            "start": [
              "--classic",
              "--datadir",
              "${SERVICE_DATA_DIR}/multi-geth/1.9.2/classic",
              "--ipcpath",
              "${SERVICE_TRANSPORT_DIR}/multi-geth/1.9.2/classic/geth.ipc",
              "--port",

with those SERVICE*DIR variables also being made available through a CLI flag and/or env var.

The automated release is failing 🚨

🚨 The automated release from the master branch failed. 🚨

I recommend you give this issue a high priority, so other packages depending on you could benefit from your bug fixes and new features.

You can find below the list of errors reported by semantic-release. Each one of them has to be resolved in order to automatically publish your package. I’m sure you can resolve this 💪.

Errors are usually caused by a misconfiguration or an authentication problem. With each error reported below you will find explanation and guidance to help you to resolve it.

Once all the errors are resolved, semantic-release will release your package the next time you push a commit to the master branch. You can also manually restart the failed CI job that runs semantic-release.

If you are not sure how to resolve this, here is some links that can help you:

• Usage documentation
• Frequently Asked Questions
• Support channels

If those don’t help, or if this issue is reporting something you think isn’t right, you can always ask the humans behind semantic-release.

---

Invalid npm token.

The npm token configured in the NPM_TOKEN environment variable must be a valid token allowing to publish to the registry https://registry.npmjs.org/.

If you are using Two-Factor Authentication, make configure the auth-only level is supported. semantic-release cannot publish with the default auth-and-writes level.

Please make sure to set the NPM_TOKEN environment variable in your CI with the exact value of the npm token.

---

Good luck with your project ✨

Your semantic-release bot 📦🚀

get running on domain

Is your feature request related to a problem? Please describe.
get jade-service-runner running on services.jade.builders domain

Additional context
AWS Deploy, pristine has the EB deploy, it might need some work to get this to work with https and certs the way we need. Maybe we can script this as infra-as-code and get it back into pristine.

something wrong with SERVICE_DIR caching (?)

Describe the bug
Using jade-service-runner -d causes unexpected behavior. Using this flag with different values at different times gives seemingly-inconsistent results.

To Reproduce

ubp52 /home/ia/myproject/
$ jade-service-runner
# > Cool!
# > Now want to move bulky 'services' dir external drive. 
ubp52 /home/ia/myproject/
$ mv ./services /media/ia/SANDISK128/jade-services-runner-services
ubp52 /home/ia/myproject/
$ jade-service-runner -d /media/ia/SANDISK128/jade-services-runner-services
# > Hmm... funny stuff.
$ mv  /media/ia/SANDISK128/jade-services-runner-services / /media/ia/SANDISK128/services
$ cd /media/ia/SANDISK128/
ubp52 /media/ia/SANDISK128
$ ls -lshat
total 160K
drwxr-xr-x 2 ia ia  32K Oct 25 09:02 .fseventsd/
drwxr-xr-x 4 ia ia  32K Oct 26 05:36 services/
drwxr-xr-x 4 ia ia  32K Apr  1  2018 .Spotlight-V100/
drwxr-xr-x 3 ia ia  32K Apr  1  2018 .Trashes/
-rw-r--r-- 1 ia ia 4.0K Apr  1  2018 ._.Trashes
ubp52 /media/ia/SANDISK128
$ jade-service-runner # <-- This should now use './services' as default '-d' services dir.
info: Service Runner port starting on [object Object] {"service":"ServiceRunner","context":"startServiceRunner","path":"/"}
info: Service Runner started on [object Object] {"service":"ServiceRunner","context":"startServiceRunner","path":"/"}
info: installing multi-geth 1.9.2 {"service":"ServiceRunner","context":"Routes","path":"/"}
info: Installing: multi-geth - 1.9.2 {"service":"ServiceRunner","context":"Installer","path":"/"}
info: installed multi-geth 1.9.2 {"service":"ServiceRunner","context":"Routes","path":"/"}
info: starting service multi-geth {"service":"ServiceRunner","context":"Routes","path":"/"}
info: starting service multi-geth {"service":"ServiceRunner","context":"Routes","path":"/"}
error: multi-geth: child process exited with err Error: spawn /media/ia/SANDISK128/jade-services-runner-services/multi-geth/1.9.2/geth ENOENT {"service":"multi-geth","context":"Child Service Process","path":"/renderService/goerli/1.9.2"}
error: multi-geth: child process exited {"service":"multi-geth","context":"Child Service Process","path":"/renderService/goerli/1.9.2"}
error: multi-geth: child process exited with err Error: spawn /media/ia/SANDISK128/jade-services-runner-services/multi-geth/1.9.2/geth ENOENT {"service":"multi-geth","context":"Child Service Process","path":"/renderService/kotti/1.9.2"}
error: multi-geth: child process exited {"service":"multi-geth","context":"Child Service Process","path":"/renderService/kotti/1.9.2"}
info: {} {"service":"ServiceRunner","context":"ServiceProcessManager","path":"/"}
info: {} {"service":"ServiceRunner","context":"ServiceProcessManager","path":"/"}
error: connect ECONNREFUSED 127.0.0.1:45697 {"service":"ServiceRunner","context":"Util","path":"/"}
error: connect ECONNREFUSED 127.0.0.1:41313 {"service":"ServiceRunner","context":"Util","path":"/"}
error: connect ECONNREFUSED 127.0.0.1:41313 {"service":"ServiceRunner","context":"Util","path":"/"}
error: connect ECONNREFUSED 127.0.0.1:45697 {"service":"ServiceRunner","context":"Util","path":"/"}
error: connect ECONNREFUSED 127.0.0.1:45697 {"service":"ServiceRunner","context":"Util","path":"/"}
error: connect ECONNREFUSED 127.0.0.1:41313 {"service":"ServiceRunner","context":"Util","path":"/"}
error: connect ECONNREFUSED 127.0.0.1:41313 {"service":"ServiceRunner","context":"Util","path":"/"}
error: connect ECONNREFUSED 127.0.0.1:45697 {"service":"ServiceRunner","context":"Util","path":"/"}
^C
ubp52 /media/ia/SANDISK128
$ jade-service-runner -d ./services
info: Service Runner port starting on [object Object] {"service":"ServiceRunner","context":"startServiceRunner","path":"/"}
info: Service Runner started on [object Object] {"service":"ServiceRunner","context":"startServiceRunner","path":"/"}
info: installing multi-geth 1.9.2 {"service":"ServiceRunner","context":"Routes","path":"/"}
info: Installing: multi-geth - 1.9.2 {"service":"ServiceRunner","context":"Installer","path":"/"}
info: installed multi-geth 1.9.2 {"service":"ServiceRunner","context":"Routes","path":"/"}
info: starting service multi-geth {"service":"ServiceRunner","context":"Routes","path":"/"}
info: starting service multi-geth {"service":"ServiceRunner","context":"Routes","path":"/"}
error: multi-geth: child process exited with err Error: spawn /media/ia/SANDISK128/jade-services-runner-services/multi-geth/1.9.2/geth ENOENT {"service":"multi-geth","context":"Child Service Process","path":"/renderService/goerli/1.9.2"}
error: multi-geth: child process exited {"service":"multi-geth","context":"Child Service Process","path":"/renderService/goerli/1.9.2"}
error: multi-geth: child process exited with err Error: spawn /media/ia/SANDISK128/jade-services-runner-services/multi-geth/1.9.2/geth ENOENT {"service":"multi-geth","context":"Child Service Process","path":"/renderService/kotti/1.9.2"}
error: multi-geth: child process exited {"service":"multi-geth","context":"Child Service Process","path":"/renderService/kotti/1.9.2"}
info: {} {"service":"ServiceRunner","context":"ServiceProcessManager","path":"/"}
info: {} {"service":"ServiceRunner","context":"ServiceProcessManager","path":"/"}
error: connect ECONNREFUSED 127.0.0.1:41589 {"service":"ServiceRunner","context":"Util","path":"/"}
error: connect ECONNREFUSED 127.0.0.1:37713 {"service":"ServiceRunner","context":"Util","path":"/"}
error: connect ECONNREFUSED 127.0.0.1:41589 {"service":"ServiceRunner","context":"Util","path":"/"}
error: connect ECONNREFUSED 127.0.0.1:37713 {"service":"ServiceRunner","context":"Util","path":"/"}

Expected behavior
Program uses a SERVICES_DIR in the place where I tell it to.

OS env

$ uname -a
Linux ubp52 5.2.9-050209-generic #201908160940 SMP Fri Aug 16 09:42:58 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
$ jade-service-runner -v
1.7.4
$ node -v && npm -v
v13.0.1
6.12.0

Add OpenETC and Hyperledger Besu node support

How can i run jade-service-runner on https

I want to run jade-service-runner on https instead of http

i've been looking for a way to do that in vain, can anyone please shed some light ?

thank you

calling list*Service before installService results in error

Describe the bug
calling listService before startService results in error
To Reproduce
Steps to reproduce the behavior:

1. in a clean directory run jade-service-runner
2. call listRunningServices

curl -X GET \
  http://localhost:8002 \
  -H 'Content-Type: application/json' \
  -d '{
  "jsonrpc": "2.0",
  "method": "listRunningServices",
  "params": [
  ],
  "id": 1
}'

3. call installService

curl -X GET \
  http://localhost:8002 \
  -H 'Content-Type: application/json' \
  -d '{
  "jsonrpc": "2.0",
  "method": "installService",
  "params": [
  	"multi-geth",
  	"1.9.0"
  ],
  "id": 1
}'

4. See error

Expected behavior
Should not error, should install service.

More information

➜  service-scratch jade-service-runner
(node:17185) ExperimentalWarning: The http2 module is an experimental API.
info: Service Runner starting on 8002 {"service":"ServiceRunner","context":"Commands"}
info: Service Runner started on 8002 {"service":"ServiceRunner","context":"Commands"}
info: installing multi-geth 1.9.0 {"service":"ServiceRunner","context":"Routes"}
info: Installing: multi-geth - 1.9.0 {"service":"ServiceRunner","context":"Installer"}
error: Could not install multi-geth Error: EEXIST: file already exists, mkdir './services' {"service":"ServiceRunner","context":"Routes"}

Desktop (please complete the following information):

• OS: Mac
• Version 1.0.10

listing services with multiple names different versions errors

Describe the bug
listing services with multiple names different versions errors

To Reproduce
Steps to reproduce the behavior:

1. Boot service runner with a new version of multi-geth
2. Try to list services
3. See screenshots

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots

Additional context

➜  jade-service-runner git:(fix/bump-multi-geth-1.9.7) npm start

> @etclabscore/[email protected] start /Users/shanejonas/openrpc/jade-service-runner
> npm run build:code && node build/src/cli/index.js -d ~/.services/


> @etclabscore/[email protected] build:code /Users/shanejonas/openrpc/jade-service-runner
> tsc

info: Service Runner port starting on [object Object] {"service":"ServiceRunner","context":"startServiceRunner","path":"/"}
debug: Service Runner internal port starting on 61555 {"service":"ServiceRunner","context":"startServiceRunner","path":"/"}
debug: added service / with 61555 http to routing table {"service":"ServiceRunner","context":"Router","path":"/"}
debug: setting up connections {"service":"ServiceRunner","context":"ProxyServer","path":"/"}
debug: listening on 8002 {"service":"ServiceRunner","context":"httpFrontend","path":"/"}
debug: set up connections {"service":"ServiceRunner","context":"ProxyServer","path":"/"}
info: Service Runner started on [object Object] {"service":"ServiceRunner","context":"startServiceRunner","path":"/"}
debug: Requesting connection be established {"service":"ServiceRunner","context":"httpFrontend","path":"/"}
debug: established http connection {"service":"ServiceRunner","context":"httpFrontend","path":"/"}
debug: forwarding request [object Object] POST {"jsonrpc":"2.0","id":"1","method":"listServices","params":["all"]} {"service":"ServiceRunner","context":"httpFrontend","path":"/"}
debug: making request to backend on port 61555 with {"jsonrpc":"2.0","id":"1","method":"listServices","params":["all"]} {"service":"ServiceRunner","context":"HttpBackend","path":"/"}
debug: listing services {"service":"ServiceRunner","context":"Routes","path":"/"}
debug: listing available services {"service":"ServiceRunner","context":"Routes","path":"/"}
debug: got services and returning {"service":"ServiceRunner","context":"Routes","path":"/"}
error: Could not find service multi-geth with osx {"service":"ServiceRunner","context":"Config","path":"/"}
debug: returning a response {"service":"ServiceRunner","context":"HttpBackend","path":"/"}
debug: received response: {
  "headers": {
    "access-control-allow-origin": "*",
    "content-type": "application/json",
    "date": "Tue, 31 Dec 2019 19:59:28 GMT",
    "connection": "keep-alive",
    "content-length": "85"
  },
  "statusCode": 200,
  "reason": "OK",
  "payload": "{\"id\":\"1\",\"jsonrpc\":\"2.0\",\"result\":{\"error\":{\"code\":6969,\"message\":\"unknown error\"}}}"
} {"service":"ServiceRunner","context":"httpFrontend","path":"/"}

Add more information about custom RPC services

I wonder how to use jade-service-runner for a custom rpc service i.e. http://111.112.113.114:2345/my-eth.

list all supported services

Is your feature request related to a problem? Please describe.
As a user, I'd like to know all the services.

Describe the solution you'd like
List out the services in the configuration.

Additional context & Use Case
I'd like to list out all supported services to be able to have a ui button to install each one.

request times out... see UnhandledPromiseRejectionWarning: "listener" argument must be a function

Describe the bug
jade-service-runner times out on my request.

To Reproduce

$ cat /etc/systemd/system/jade_service_runner.service
[Unit]
Description=Jade Service Runner

[Service]
Type=simple
User=ia
Restart=always
RestartSec=3
TimeoutStopSec=10min
ExecStart=/bin/sh -c 'JADE_DEBUG=1 /home/ia/.nvm/versions/node/v13.1.0/bin/jade-service-runner -d /home/ia/jade-services -c ~/dev/meowsbits/stargate-sg1/jade-extended-config.json'

[Install]
WantedBy=default.target

$ systemctl [re]start jade_service_runner.service

$ http --verbose --timeout 5 --json POST localhost:8002 id=0 method=listServices params:=[]

http: error: Request timed out (5.0s).

(The less fancy curl - as below - gives same result.)

$ curl -X POST -d '{"method": "listServices", params:[], "id": "0"}' localhost:8002
curl: (52) Empty reply from server

$ journalctl -u jade_service_runner.service -f
Hint: You are currently not seeing messages from other users and the system.
      Users in groups 'adm', 'systemd-journal' can see all messages.
      Pass -q to turn off this notice.
-- Logs begin at Fri 2019-10-18 17:07:27 EDT. --
Nov 12 14:57:21 ubp52 sh[21592]: (node:21593) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 9)
Nov 12 15:02:34 ubp52 sh[24339]: (node:24344) ExperimentalWarning: The http2 module is an experimental API.
Nov 12 15:02:34 ubp52 sh[24339]: (node:24344) Warning: N-API is an experimental feature and could change at any time.
Nov 12 15:02:34 ubp52 sh[24339]: info: Service Runner port starting on [object Object] {"service":"ServiceRunner","context":"startServiceRunner","path":"/"}
Nov 12 15:02:34 ubp52 sh[24339]: debug: Service Runner internal port starting on 37155 {"service":"ServiceRunner","context":"startServiceRunner","path":"/"}
Nov 12 15:02:34 ubp52 sh[24339]: debug: added service / with 37155 http to routing table {"service":"ServiceRunner","context":"Router","path":"/"}
Nov 12 15:02:34 ubp52 sh[24339]: debug: setting up connections {"service":"ServiceRunner","context":"ProxyServer","path":"/"}
Nov 12 15:02:34 ubp52 sh[24339]: debug: listening on 8002 {"service":"ServiceRunner","context":"httpFrontend","path":"/"}
Nov 12 15:02:34 ubp52 sh[24339]: debug: set up connections {"service":"ServiceRunner","context":"ProxyServer","path":"/"}
Nov 12 15:02:34 ubp52 sh[24339]: info: Service Runner started on [object Object] {"service":"ServiceRunner","context":"startServiceRunner","path":"/"}
Nov 12 15:02:40 ubp52 sh[24339]: debug: Requesting connection be established {"service":"ServiceRunner","context":"httpFrontend","path":"/"}
Nov 12 15:02:40 ubp52 sh[24339]: debug: established http connection {"service":"ServiceRunner","context":"httpFrontend","path":"/"}
Nov 12 15:02:40 ubp52 sh[24339]: debug: forwarding request [object Object] POST {"id":"0","method":"listServices","params":[]} {"service":"ServiceRunner","context":"httpFrontend","path":"/"}
Nov 12 15:02:40 ubp52 sh[24339]: debug: making request to backend on port 37155 with {"id":"0","method":"listServices","params":[]} {"service":"ServiceRunner","context":"HttpBackend","path":"/"}
Nov 12 15:02:40 ubp52 sh[24339]: (node:24344) UnhandledPromiseRejectionWarning: TypeError: "listener" argument must be a function
Nov 12 15:02:40 ubp52 sh[24339]:     at ClientRequest.once (events.js:340:11)
Nov 12 15:02:40 ubp52 sh[24339]:     at new ClientRequest (_http_client.js:164:10)
Nov 12 15:02:40 ubp52 sh[24339]:     at Object.request (http.js:38:10)
Nov 12 15:02:40 ubp52 sh[24339]:     at Promise (/home/ia/.nvm/versions/node/v13.1.0/lib/node_modules/@etclabscore/jade-service-runner/build/src/lib/backends/httpBackend.js:31:44)
Nov 12 15:02:40 ubp52 sh[24339]:     at new Promise (<anonymous>)
Nov 12 15:02:40 ubp52 sh[24339]:     at Object.send (/home/ia/.nvm/versions/node/v13.1.0/lib/node_modules/@etclabscore/jade-service-runner/build/src/lib/backends/httpBackend.js:27:16)
Nov 12 15:02:40 ubp52 sh[24339]:     at ConnectionManager.forwardRequest (/home/ia/.nvm/versions/node/v13.1.0/lib/node_modules/@etclabscore/jade-service-runner/build/src/lib/connectionManager.js:61:53)
Nov 12 15:02:40 ubp52 sh[24339]:     at EventEmitter.connectionBus.on (/home/ia/.nvm/versions/node/v13.1.0/lib/node_modules/@etclabscore/jade-service-runner/build/src/lib/connectionManager.js:108:18)
Nov 12 15:02:40 ubp52 sh[24339]:     at emitOne (events.js:116:13)
Nov 12 15:02:40 ubp52 sh[24339]:     at EventEmitter.emit (events.js:211:7)
Nov 12 15:02:40 ubp52 sh[24339]: (node:24344) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 3)
Nov 12 15:02:40 ubp52 sh[24339]: (node:24344) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Expected behavior
Not to silently time out; at least an error message.

Desktop (please complete the following information):

$ jade-service-runner -v                                                                                                                │
1.7.4

$ node -v && npm -v && uname -a
v13.1.0
6.12.1
Linux ubp52 5.2.9-050209-generic #201908160940 SMP Fri Aug 16 09:42:58 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux