smallnest/rpcx

Best microservices framework in Go, like alibaba Dubbo, but with more features, Scale easily. Try it. Test it. If you feel it's better, use it! 𝐉𝐚𝐯𝐚有𝐝𝐮𝐛𝐛𝐨, 𝐆𝐨𝐥𝐚𝐧𝐠有𝐫𝐩𝐜𝐱! build for cloud!

• stable branch: v1.7.x
• development branch: master

Official site: http://rpcx.io

Notice: etcd

since rpcx 1.7.6, some plugins have been moved to the independent project:

• etcd plugin has been moved to rpcx-etcd
• zookeeper plugin has been moved to rpcx-zookeeper
• consul plugin has been moved to rpcx-consul
• redis plugin has been moved to rpcx-redis
• influxdb plugin has been moved to rpcx-plugins
• opentelemetry plugin has been moved to rpcx-plugins

Announce

A tcpdump-like tool added: rpcxdump。 You can use it to debug communications between rpcx services and clients.

Cross-Languages

you can use other programming languages besides Go to access rpcx services.

• rpcx-gateway: You can write clients in any programming languages to call rpcx services via rpcx-gateway
• http invoke: you can use the same http requests to access rpcx gateway
• Java Services/Clients: You can use rpcx-java to implement/access rpcx services via raw protocol.
• rust rpcx: You can write rpcx services in rust by rpcx-rs

If you can write Go methods, you can also write rpc services. It is so easy to write rpc applications with rpcx.

Installation

install the basic features:

go get -v github.com/smallnest/rpcx/...

If you want to use quic、kcp registry, use those tags to go get 、 go build or go run. For example, if you want to use all features, you can:

go get -v -tags "quic kcp" github.com/smallnest/rpcx/...

tags:

• quic: support quic transport
• kcp: support kcp transport

Which companies are using rpcx?

Features

rpcx is a RPC framework like Alibaba Dubbo and Weibo Motan.

rpcx is created for targets:

1. Simple: easy to learn, easy to develop, easy to integrate and easy to deploy
2. Performance: high performance (>= grpc-go)
3. Cross-platform: support raw slice of bytes, JSON, Protobuf and MessagePack. Theoretically it can be used with java, php, python, c/c++, node.js, c# and other platforms
4. Service discovery and service governance: support zookeeper, etcd and consul.

It contains below features

• Support raw Go functions. There's no need to define proto files.
• Pluggable. Features can be extended such as service discovery, tracing.
• Support TCP, HTTP, QUIC and KCP
• Support multiple codecs such as JSON, Protobuf, MessagePack and raw bytes.
• Service discovery. Support peer2peer, configured peers, zookeeper, etcd, consul and mDNS.
• Fault tolerance：Failover, Failfast, Failtry.
• Load banlancing：support Random, RoundRobin, Consistent hashing, Weighted, network quality and Geography.
• Support Compression.
• Support passing metadata.
• Support Authorization.
• Support heartbeat and one-way request.
• Other features: metrics, log, timeout, alias, circuit breaker.
• Support bidirectional communication.
• Support access via HTTP so you can write clients in any programming languages.
• Support API gateway.
• Support backup request, forking and broadcast.

rpcx uses a binary protocol and platform-independent, which means you can develop services in other languages such as Java, python, nodejs, and you can use other prorgramming languages to invoke services developed in Go.

There is a UI manager: rpcx-ui.

Performance

Test results show rpcx has better performance than other rpc framework except standard rpc lib.

The benchmark code is at rpcx-benchmark.

Listen to others, but test by yourself.

Test Environment

• CPU: Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz, 32 cores
• Memory: 32G
• Go: 1.9.0
• OS: CentOS 7 / 3.10.0-229.el7.x86_64

Use

• protobuf
• the client and the server on the same server
• 581 bytes payload
• 500/2000/5000 concurrent clients
• mock processing time: 0ms, 10ms and 30ms

Test Result

mock 0ms process time

Throughputs	Mean Latency	P99 Latency

mock 10ms process time

Throughputs	Mean Latency	P99 Latency

mock 30ms process time

Throughputs	Mean Latency	P99 Latency

Examples

You can find all examples at rpcxio/rpcx-examples.

The below is a simple example.

Server

    // define example.Arith
    ……

    s := server.NewServer()
	s.RegisterName("Arith", new(example.Arith), "")
	s.Serve("tcp", addr)

Client

    // prepare requests
    ……

    d, err := client.NewPeer2PeerDiscovery("tcp@"+addr, "")
	xclient := client.NewXClient("Arith", client.Failtry, client.RandomSelect, d, client.DefaultOption)
	defer xclient.Close()
	err = xclient.Call(context.Background(), "Mul", args, reply, nil)

Contributors

Contribute

see contributors.

Welcome to contribute:

• submit issues or requirements
• send PRs
• write projects to use rpcx
• write tutorials or articles to introduce rpcx

License

Apache License, Version 2.0

rpcx's Issues

ServeByHTTP support customized mux

golang/go#13395

Add forking mode for clients

Forking mode: one client can send a requests to multiple servers and it only handles the fast response, other responses will be ignored.

It fits low latency case but it will take more resources.

benchmark code has some issue

some requests are not OK

close rpcx client cause crash

I create a rpcx client to send rpc to remote server with Call API, and close it with defer. If the remote server is not running, the rpc call failed. But the whole program crashed with "Handler crashed with error runtime error: invalid memory address or nil pointer dereference".

It is clearly that the crash is caused by the Close API. Because it didn't crash if I removed the Close API.

How can i fix it?
Thanks

关于rpc的几个疑问

1.etcd注册部分目前使用的是定时拉取，其实可以使用watch.
2.client没有加锁，也就是client无法保证线程安全？
3.go原生rpc里的call没有超时，且上层无法感知连接的可用状态，之前我们遇到过网络环境不好时，连接已经不可用，call没有返回，导致大量线程阻塞，大概10多分钟后所有call集体返回超时．针对这种情况该rpc框架也是基于go原生rpc搭建，针对这个问题是否会有所考虑？

add doc for all codes

add doc/comments for all codes

support compression for requests and responses

common zip compression and snappy, etc.

TLS support

add TLS support for TCP and HTTP

服务集群的权重问题

刚看了之前提的问题, 解决的很棒!

目前框架内分配内服务的分配策略有随机,轮训,以及一致性hash.其实这里还缺少按权重分配策略,如加权轮训,加权随机,以及加权一致性hash.权重可通过服务定期向zookeeper等上报,client获取到权重后进行加权分配.
这样解决相同服务部署多个在不同服务器上时,主机硬件配置不一致,主机之间负载不均衡等情况下的负载均衡问题.

check services' health and update its weight

If the connection between a service and a client has some issues, but the service is OK and it can report OK to registries, for example, only network between the service an the client is slow. We should handle this case and reduce weight of this service.

Reducing weight only take effect for WeightedRoundrobin, for other select modes such as random, round robin, TBD.

support let's encrypt tls

use the lib:
https://github.com/rsc/letsencrypt

connection pool support

support QUIC protocol

QUIC, a multiplexed stream transport over UDP
QUIC is a new transport which reduces latency compared to that of TCP. On the surface, QUIC is very similar to TCP+TLS+HTTP/2 implemented on UDP. Because TCP is implemented in operating system kernels, and middlebox firmware, making significant changes to TCP is next to impossible. However, since QUIC is built on top of UDP, it suffers from no such limitations.

Key features of QUIC over existing TCP+TLS+HTTP2 include
Dramatically reduced connection establishment time
Improved congestion control
Multiplexing without head of line blocking
Forward error correction
Connection migration

implement a manger web-ui

implement a manger web-ui to manage services:

• list services
• deactivate/activate services
• route?
• and more?

add etcd watch to replace pulling implementation

just like zookeeper

support customized service name

etcd_selector bug

run _examples/etcd_registry/client.go

panic: map is nil

code: clientselector/etcd_selector.go
line: 175 s.metadata[key] = n.Value

fix: see pr

add Authentication feature for RPC services

Requirement:
Authentication can be used to allow only authorized clients to call RPC services. If clients have bot been authorized, RPC services should return errors

Investigation:
Current rpcx defines IPostConnAcceptPlugin, which can be used to control permits of clients based on IP blacklist or whitelist. But we need a more powerful ability to control permission based on authentication.

Design
Add a authentication plugin to authorize clients. An Authentication header has been added before Requests that is like Oauth2, for example,

Authorization: Bearer 0b79bab50daca910b000d4f1a2b675d604257e42

rpcx doesn't care the value of Authorization header and how to verify this value. It provides a plugable authorization mechanism. Developers can use Oauth/Oauth2, session, databases, cache and others to handle this value of Authorization header.

An simple authorization should be provided.

is there any plans to support other languages?

alias name of services don't work

If we register a service with an alias name, for example,

server.RegisterName("NewName", new(Arith), "weight=1&m=devops")

client can't invoke this service by this name "NewName"

有个问题想请教哈

如果采用轮询、最小调用次数统计等策略，以达到负载均衡的目的。那是不是已建立好的rpc客户端与rpc服务端连接，都需要断开老连接、建立新连接？

add tls example

add an example for tls configuration

compile error

➜  clientpool git:(master) go run client.go
# github.com/smallnest/rpcx/clientselector
../../clientselector/ping_utils.go:15: p.Network undefined (type *fastping.Pinger has no field or method Network)

implement FlatBuffers codec

references:

• https://github.com/google/flatbuffers
• https://open.dgraph.io/post/rpc-vs-grpc/
• https://rwinslow.com/posts/use-flatbuffers-in-golang/

Currently flatbuffers is not like other codecs. It is difficult to implement ServerCodec and ClientCodec. One possible approach is creating generator that can create a generic interface for all flatbuffers but it is out of scope of this project.

add geo routing to select closest rpc services for clients

we can get geo location based on IP addresses of services and clients. So clients can select the closest services by comparing geo locations.

It is useful because clients always select services in the same datacenter (city).

implement multicast registry

什么时候支持多种语言，尤其是对于ios/android

什么时候支持多种语言，尤其是对于ios/android ?

文档里有个中文

粗现中文了.
support many codec. for example, Gob、Json、MessagePack、gencode、ProtoBuf等

implement colfer codec

https://github.com/pascaldekloe/colfer

Both faster and smaller than: Protocol Buffers, FlatBuffers and MessagePack

可以考虑加到这个列表里

https://github.com/mfornos/awesome-microservices

例子循环10000次出错

multi_server，将client_consistenthash.go中10改成10000后测试出现错误：
error for Arith: 7*8, dial tcp 127.0.0.1:8973: bind: An operation on a socket could not be performed because the system lacked sufficient buffer space or because a queue was full.

win7 64位系统, go version go1.6 windows/amd64

add routing based on PING result

according to results of PING, we can set weight of each services.
Shorter ping time of services can get more chances to be selected.

add broadcast mode for clients

broadcast mode: one client sends a request to all servers and the client will handle it as success only all servers handle this request successfully.

It fits broadcast cache or configuration, etc.

Fixed: ipv4/ipv6 redeclared errors

go version: go1.7.1 linux/amd64

when you compile " go build ping_utils.go" or " go build fastping.go " , you may attack below errors.

the solution: you need to "go get -u golang.org/x/net/icmp" .

golang.org/x/net/ipv6

../../../../golang.org/x/net/ipv6/sockopt_asmreq_unix.go:15: setsockoptIPMreq redeclared in this block
previous declaration at ../../../../golang.org/x/net/ipv6/sockopt_asmreq_posix.go:15
../../../../golang.org/x/net/ipv6/sockopt_unix.go:15: getInt redeclared in this block
previous declaration at ../../../../golang.org/x/net/ipv6/sockopt_posix.go:15
../../../../golang.org/x/net/ipv6/sockopt_unix.go:27: setInt redeclared in this block
previous declaration at ../../../../golang.org/x/net/ipv6/sockopt_posix.go:27
../../../../golang.org/x/net/ipv6/sockopt_unix.go:35: getInterface redeclared in this block
previous declaration at ../../../../golang.org/x/net/ipv6/sockopt_posix.go:35
../../../../golang.org/x/net/ipv6/sockopt_unix.go:54: setInterface redeclared in this block
previous declaration at ../../../../golang.org/x/net/ipv6/sockopt_posix.go:54
../../../../golang.org/x/net/ipv6/sockopt_unix.go:65: getICMPFilter redeclared in this block
previous declaration at ../../../../golang.org/x/net/ipv6/sockopt_posix.go:65
../../../../golang.org/x/net/ipv6/sockopt_unix.go:77: setICMPFilter redeclared in this block
previous declaration at ../../../../golang.org/x/net/ipv6/sockopt_posix.go:77
../../../../golang.org/x/net/ipv6/sockopt_unix.go:84: getMTUInfo redeclared in this block
previous declaration at ../../../../golang.org/x/net/ipv6/sockopt_posix.go:84
../../../../golang.org/x/net/ipv6/sockopt_unix.go:103: setGroup redeclared in this block
previous declaration at ../../../../golang.org/x/net/ipv6/sockopt_posix.go:103
../../../../golang.org/x/net/ipv6/sockopt_unix.go:117: setSourceGroup redeclared in this block
previous declaration at ../../../../golang.org/x/net/ipv6/sockopt_posix.go:117
../../../../golang.org/x/net/ipv6/sockopt_unix.go:117: too many errors

golang.org/x/net/ipv4

../../../../golang.org/x/net/ipv4/sockopt_posix.go:29: cannot use s (type uintptr) as type int in argument to getsockopt
../../../../golang.org/x/net/ipv4/sockopt_posix.go:51: cannot use s (type uintptr) as type int in argument to setsockopt
../../../../golang.org/x/net/ipv4/sockopt_unix.go:17: getInt redeclared in this block
previous declaration at ../../../../golang.org/x/net/ipv4/sockopt_posix.go:17
../../../../golang.org/x/net/ipv4/sockopt_unix.go:38: setInt redeclared in this block
previous declaration at ../../../../golang.org/x/net/ipv4/sockopt_posix.go:38
../../../../golang.org/x/net/ipv4/sockopt_unix.go:54: getInterface redeclared in this block
previous declaration at ../../../../golang.org/x/net/ipv4/sockopt_posix.go:54
../../../../golang.org/x/net/ipv4/sockopt_unix.go:68: setInterface redeclared in this block
previous declaration at ../../../../golang.org/x/net/ipv4/sockopt_posix.go:68
../../../../golang.org/x/net/ipv4/sockopt_unix.go:82: getICMPFilter redeclared in this block

add timeout for client rpc call

add timeout for client rpc call so users can cancel some rpc calls

rpcx performance become very low

it is caused by rpcx.Client.

For DirectRPCClient, its Select method always returns a new connection. It is not what I expected.

🐴主有没有各种大小和连接数下的性能测试共参考啊？

如题。

add connection status

add methods to check status of underlying net.Conn

investigate ratelimit libraries

Current rpcx uses http://github.com/juju/ratelimit as the ratelimiting.
I have found uber has released their ratelimit library. Maybe I need compare those two libraries and select the more simple and better performance library.

https://github.com/uber-go/ratelimit

add "group" concept for clients and services

although we can set different zookeeper base path or etcd node for different groups, it is still convenient to set group tag for services which has same path/node, then clients only use services that has the same group.

For example, we can set some services with test group for testing, test clients only call those test services. After testing, we can move the test group tag and deploy those services.

请问如何获取客户端的IP地址及端口？

有没有获取当前客户端IP及端口的方法呢？
找了半天还是没有发现相应的函数。

服务之间通信

有时候要实现服务之间参数同步，不重启修改参数。所以需要服务分组广播接口。

Possible deprecation of net/rpc

Go proposal
Can deprecation of net/rpc break rpcx?

请问现在能否被用于生产环境？

support deactivate a service temporarily

If we want to upgrade a service, we want to deactivate this service temporarily and this service becomes maintained. After upgraded or maintained, we want to activate this service again.

rpcx should add this function to update status of services.
For first stage servers，clients and registries should support this function.
For second stage, we should a rpcx-manager web ui to check and manage services

支持临时地暂停一个服务。有时候我们需要暂停一个服务，以便升级或者维护，rpcx必须提供相应的接口，往registry上更新这个服务的状态， client监听到这个状态后应该移除(暂停)这个服务。

将来实现rpcx-manager web UI也可以方便的在UI上操作

运行_examples/zookeeper_registry报error for 127.0.0.1:2181: 7*8, No available service

1，zookeeper已安装
2，已运行server.go, go run server.go
3，运行client报错

vagrant@vagrant-ubuntu-trusty-64:/go/src/github.com/smallnest/rpcx/_examples/zookeeper_registry$ go run client.go
2016/10/25 17:20:58 Connected to 127.0.0.1:2181
2016/10/25 17:20:58 Authenticated: id=96821782470918149, timeout=40000
error for 127.0.0.1:2181: 7_8, No available service
error for 127.0.0.1:2181: 7_8, No available service
error for 127.0.0.1:2181: 7_8, No available service
error for 127.0.0.1:2181: 7_8, No available service
error for 127.0.0.1:2181: 7_8, No available service
error for 127.0.0.1:2181: 7_8, No available service
error for 127.0.0.1:2181: 7_8, No available service
error for 127.0.0.1:2181: 7_8, No available service
error for 127.0.0.1:2181: 7_8, No available service
error for 127.0.0.1:2181: 7_8, No available service

add benchmark

add benchmark for rpcx, net/rpc and net/rpc/jsonrpc

add etcd registry for service discovery

implement JSON-RPC2

unify errors

unify errors.
investigate existed error frameworks such as github.com/pkg/errors or invent a wheel