jovany-wang/dousi

Native Multiple Languages RPC Framework.

dousi's Issues

Support protobuf

Use SWIG to wrapper native code for other languages.

Feature Request: ChaosProxy that helps users mock any networking issues.

Is it necessary to support actor model?

Use macro to avoid many `RegisterMethod`s.

Support arrow IPC protocol

Update to replace log4j with log4j2

• Update to replace log4j with log4j2

Add event based framework for distributed networking access.

We have 3 P2P nodes, and every node has its code:

class Node implements EventBased {

  void f1() {
      // change my state
  }
  void f2() {
     // change my state as well
  }

}

Server config items setting

• TCP no delay option.
• buffer size option.

Do not support concurrently call on the client side.

Calls are missing when benchmarking.

After the benchmark client(BenchmarkMultiThreadsClient) stoped, result shows that it missed around 70% requests.
Current qps is 0, sum= 3278165.

The dynamic asserting.

Since there is no approach to assert the static errors such as protobufs or other IDLs, we should support the dynamic asserting:

If the method signification doesn't match the real argument list, the server will return a exception to indicate that, and then the client will throw it in getting.

Remote method doesn't support void return.

Reproducation code

service:

class VoidReturnService : public dousi::AbstractService {
public:
    void Perform(const std::string &str) {
        data_ = str;
    }

    std::string Get() {
        return data_;
    }

private:
    std::string data_ = "123";
};

server:

        // VoidReturnService
        auto void_return = dousi::executor::CreateService<VoidReturnService>();
        void_return.RegisterMethod(dousi::Remote(&VoidReturnService::Perform));
        void_return.RegisterMethod(dousi::Remote(&VoidReturnService::Get));

The server could not pass the compiling with these words complaint:

Support P2P IPC.

In many distributed systems, such as Spark Flink, Ray, and other like YARN based systems, a local tenancy often boardcast to other ends like notifications. It's valuable that we support the P2P IPC to make the things simpler.

Usage example

// remote methods definitions
void Heartbeat(int termId) {
  // handle.
}

void AppendLogs(int termId, int logId, buffer log_data) {
  // handle
}

// processor1
dousi::Grid grid = dousi::Grid::GetOrCreate(master_addr="127.0.0.1:10001", grid_name="my_grid_1");
grid.SubProcessorsChanged([](ProcessorsChangedType type, ProcessorsInfo before, ProcessorsInfo after) {
    // Handle the processors changed.
});
grid.Sub(/*method_to_be_subscribed=*/dousi::Remote(&Heartbeat));
grid.Sub(/*method_to_be_subscribed=*/dousi::Remote(&AppendLogs));

// processor2
dousi::Grid grid = dousi::Grid::GetOrCreate(master_addr="127.0.0.1:10001", grid_name="my_grid_1");
grid.SubProcessorsChanged([](ProcessorsChangedType type, ProcessorsInfo before, ProcessorsInfo after) {
    // Handle the processors changed.
});
grid.Sub(/*method_to_be_subscribed=*/dousi::Remote(&Heartbeat));
grid.Sub(/*method_to_be_subscribed=*/dousi::Remote(&AppendLogs));

// processor3
dousi::Grid grid = dousi::Grid::GetOrCreate(master_addr="127.0.0.1:10001", grid_name="my_grid_1");
grid.SubProcessorsChanged([](ProcessorsChangedType type, ProcessorsInfo before, ProcessorsInfo after) {
    // Handle the processors changed.
});
grid.Pub(/*method_to_be_subscribed=*/dousi::Remote(&Heartbeat), /*termId=*/100);
grid.Pub(/*method_to_be_subscribed=*/dousi::Remote(&AppendLogs， 100， 200， “this log”));

It should noted that the Grid is used to be the identifier to the local tenancy for processors discovery and networking.

For implementation

Each processor holds an executor and a submitter for methods invoking and submitting.

Support bytes as argument type in Java.

Failed to call twice with no `Get()` on the first one.

Reprod code:

echoer.Call(dousi::Remote(&Echoer::echo), "A");  // Do not Get this future.
echoer.Call(dousi::Remote(&Echoer::echo), "B");

Above code will fail since the current codebase is not send the parts of one message together.
This is the server error message:

Performance optimizing plan

This issue traces the performance optimizing tasks.

• Add WaitAndPopBatch for StdLockedQueue to avoid locking time.
• Use emplaced buffer instead of new int, new string, etc.
• Avoid unpack twice. #35

Benchmarks

• Benchmark on Linux VS epoll.

Support co/do?

Enhancements

• Make dousi::Remote annotation around a method not only a method pointer: dousi::Remote(Echoer::echo)
• Init() accepts the listening port or the connecting target address. #16
• Pass custom-defined class as argument.
• Code style formater.
• Refine traits and TraitCalls.
• Remove class AbstractService.
• Return a DousiFuture<bool> for void return method on client side.
• Support arrow protocol to exchange large relation-data.
• Use codec as plugable so that we can use msgpack or other codec like protobuf.

Pass service instance by pointer instead of closure-object.

https://github.com/jovany-wang/dousi/blob/master/core/tests/basic_call_test.cc#L38
This line is not correct now, because in the server side, we pass the service instance by closure-object so the value will not be changed any more once binding.

Failed to read the buffer in channelread.

    @Test
    public void testMultiCalls() throws ParserAddrException, InterruptedException {
        DousiRpcClient client = new NettyRpcClient("127.0.0.1:10001");
        AdderService service = client.newStub(AdderService.class);
        int result1 = service.add(3, 4);
        int result2 = service.add(1, 2);
        Assert.assertEquals(7, result1);
        Assert.assertEquals(3, result1);
        client.shutdown();
    }

The above test will be failed since we do not cache the read buffer if there is not enough a package in channelread.

Implement eventbase

Test enhancements

Cpp RPC

• User-defined classes as arguments and return type.

Java

• User-defined classes as arguments and return type.
• Full arguments test.
• Void return.
• Add CI for Java Client.

TODOs for Cross-Languaged RPC

• Handle the overloading of methods.
• Auto-handle the schema of a method(args and return type, etc).
• Custom class arguments and return type for both Cpp and Java.

Current TODO tasks

• Encapsulate asio::socket as a class with AsnycRead(), AsyncWrite. #6
• Refine APIs.
• Use spdlog instead of mack log. #23
• Add more basic and complex tests.
• Add basic benchmark.
• Support exporting installed headers and libs.
• Use io threads. #38
• master uses stream lib #22
• Refactor service instance performing process with one by one remote call.
• Use condition variable instead of while true getting object.

TODO Features List.

• Java RPC Server
• Protobuf protocol

Dousi Design Overview

Dousi Server Side Design Overview

1. Motivations

The purpose of Dousi RPC framework is to build a highly performance, multiple languages server and easy to use RPC framework for the distributed domain especially for modern distributed end-to-end system, such as online recommender system, search engine system, etc.

[Highly Performance] For internal remote method invocations, the highly performance should prefer being the low latency and high throughput to high concurrency. This is the biggest difference between Dousi motivation and the motivations of other RPC frameworks like brpc. The reason for this point is internal remote invocations has no high concurrency requirement like the RPC server which should provide the services to a big number of public clients.

[Multiple Languages Server] For a big end-to-end system, it is almost built in several languages, And it's usual that upstream uses language A and the downstream uses language B. In the traditional solution, It almost uses one protocol to be communicated between the upstream and downstream but uses different RPC framework in different sides. Dousi aims to provides multiple languages server and client to solve this issue, so that users shouldn't know the protocol details between different RPC frameworks in one large system.

[Easy to Use Abstraction] For this motivation, we would like to try our best to provide a very easy to use abstraction. We
should not only provide the abstraction in the same language of server and client, but also provide the easy to use abstraction in the different languages of server and client.

2. Overview of the abstrations

The bottom layer is the core layer which provides the basic abilities. The middle layer is runtime, and it provides the runtimes of Dousi. And the top layer is unified remote invocations API.

3. Overview of the threading model

For the motivations mentioned above, we are designing the Dousi RPC framework to achieve the following goals:

• Trade off the requests order and the CPU usage.
• Pull result of remote invocation lazily by default.
• Do not copy data when shipping the buffers in a process.

There are some key points from the picture:

1. A buffer pool to manage all buffers generated by both receiving requests and preparing the results. The buffers we ship in the server have no copy operation since the buffer objects are managed by the pool.
2. We separated the IO threads and the work threads to let users have the ability to set a reasonable proportion for them to achieve a best CPU usage. Limited by the C++ implementation, it's hard to do serialize and deserialize in IO threads. Because we use the template and function traits for the remote method signification, the work threads have more information for the remote method signification.
3. For one service, we expect that it performs its methods one by one, while the different services performing their methods concurrently. So we build 2 queues for every service instance: requests queue and results queue. The work threads fetch the requests from that service queue and push the result to the result queue.
4. Back pressure is controlled by a manage, which is a monitor to monitor the services queues.

Add handy as another eventloop.

https://github.com/yedf/handy is a great library to help us build the highly performance RPC framework.

Add an abstraction to support the eventloop as an abstracted layer, so that we can switch them.