e2log/configrpc

A Distributed Configuration Service Based on gRPC

ConfigRPC

A Distributed Configuration Service Based on gRPC.

Each comfiguration is composed of a key/value pair, where the key can be any string and value is a JSON document so that nested structured configurations can be stored.

As an use case example, we at e2log.com, use a similar service to store feature toggles and parameters for each module of our B2B SaaS platform. All feature toggles and parameters for all modules are stored in a single JSON document with the customer identifier as the key.

This is the ConfigRPC server component, it provides a gRpc API for retrieving partial or total configuration objects from the main value using JSON Path expressions.

JSON Path gRPC API.

service ConfigrpcService {
    rpc findString( FindStringRequest ) returns ( FindStringResponse ) {}
}

message FindStringRequest{
    string key = 1;
    google.protobuf.StringValue jsonPath = 2;
}

message FindStringResponse{
    KeyString kv = 1;
}

message KeyString {
    string key = 1;
    string value = 2;
}

Examples

Given the following configuration data pair

key: /service/data

value:

{
  "environments" : [
    {
      "environment": "TEST", 
      "database": {"host": "10.0.0.79", "user": "sa", "password_key": "test-db-pwd"}, 
      "admin": "[email protected]"
     },
    {
      "environment": "STAGING",
      "database": {"host": "10.1.0.237", "user": "sa", "password_key": "staging-db-pwd"},
      "admin": "[email protected]"
    }
  ]
}

Query 1: All database properties of 1st environment

key : `/service/data` 

jsonPath : `$.environments[0].database`

Returns: {host=10.0.0.79, user=sa, password_key=test-db-pwd}

Query 2: The host property of the database of 1st environment

key : `/service/data` 

jsonPath : `$.environments[0].database.host`

Returns: 10.0.0.79

Query 3: The property admin of all the database configurations

key : `/service/data` 

jsonPath : `$..admin`

Returns: ["[email protected]","[email protected]"]

Using gRPC-UI

All admins JsonPath query:

All admins result:

How do I get set up?

• java 11 or newer

Dependencies

This server component depends on the interface submodule configrpc-api.jar

Install the dependency configrpc-api.jar locally, see project for install details

Run

./mvnw spring-boot:run

Note: this project uses Google protocol buffers

configrpc's Issues

Create sample Java client

A client sample is useful for demonstrating the value proposition and helps beginners to get started.

Add Postgres support

Current implementations is based on H2 in memory only which was created for quick start and demos.
Postgres implementation should use a jsonb column for the configuration value.

Publish a docker image on docker hub

Create sample non Java client (Python or NodeJs)

A non Java client sample is useful for demonstrating the multi language suport and helps beginners to get started.

Add Protobuf client API

How can client provide their own protos ?

• Add a proto store mechanism, would the package manager protop work for this use case?

Add write API

As a client, I want to be able to store new k/v configurations so that configurations can be stored from the client application.