Giter Site home page Giter Site logo

apache / dubbo-go-pixiu Goto Github PK

View Code? Open in Web Editor NEW
466.0 62.0 145.0 49.79 MB

Based on the proxy gateway service of dubbo-go, it solves the problem that the external protocol calls the internal Dubbo cluster. At present, it supports HTTP and gRPC[developing].

Home Page: https://dubbogo.github.io/dubbo-go-proxy/

License: Apache License 2.0

Go 98.27% Shell 0.97% Makefile 0.57% Dockerfile 0.03% Smarty 0.06% Python 0.09% Java 0.01% Ruby 0.01%
dubbo proxy dubbo-cluster upstream-services dubbogo dubbo-go microservice microservices gateway http-proxies

dubbo-go-pixiu's Introduction

Pixiu Logo

Language Build Status

English | 中文

Introduction

Dubbo-Go-Pixiu(official site: https://dubbo.apache.org/zh/docs3-v2/dubbo-go-pixiu/) is a high-performance API gateway and multi-language solution Sidecar in the Dubbo ecosystem

It is an open source Dubbo ecosystem API gateway, and also a sidecar to let other compute language program access the dubbo clusters by HTTP/gRPC protocol. As an API gateway, Pixiu can receive external network requests, convert them into dubbo and other protocol requests, and forward them to the back cluster; as a sidecar, Pixiu expects to register to the Dubbo cluster instead of the proxy service, allowing multilingual services to access the Dubbo cluster to provide faster solution

Quick Start

Requirment

  1. go 1.17 or higher
  2. docker or docker-desktop

you can find out all demo in https://github.com/apache/dubbo-go-pixiu-samples. download it and operate as below.

git clone https://github.com/apache/dubbo-go-pixiu-samples.git

update pixiu to latest version

go get github.com/apache/[email protected]

cd samples dir

cd dubbogo/simple

we can use start.sh to run samples quickly. for more info, execute command as below for more help

./start.sh [action] [project]
./start.sh help

we run [direct] samples step by step as follows.

prepare config file and docker

'prepare' command will prepare dubbo-server and pixiu config file firstly, and then start docker container.

./start.sh prepare direct

if prepare config file manually, notice:

  • modify $PROJECT_DIR in conf.yaml to absolute path

start dubbo or http server

./start.sh startServer direct

start pixiu

./start.sh startPixiu direct

if run pixiu manually in pixiu project, use command as below.

 go run cmd/pixiu/*.go gateway start -c /[absolute-path]/dubbo-go-pixiu/samples/dubbogo/simple/direct/pixiu/conf.yaml

if run pixiu manually in pixiu project and wasm feature, use command as below.

build pixiu project use command operate

go build -tags wasm -o pixiu cmd/pixiu/*.go

run pixiu app binary

go build cmd/pixiu/*.go
./pixiu gateway start -c /[absolute-path]/dubbo-go-pixiu/samples/dubbogo/simple/direct/pixiu/conf.yaml

Try a request

use curl to try or use unit test

curl http://localhost:8883/UserService/com.dubbogo.pixiu.UserService/GetUserByCode \
-H "x-dubbo-http1.1-dubbo-version:1.0.0" -H "x-dubbo-service-protocol:dubbo" \
-H "x-dubbo-service-version:1.0.0" -H "x-dubbo-service-group:test" \
-H "Content-Type:application/json" \
 -d '[1]'
curl http://localhost:8883/UserService/com.dubbogo.pixiu.UserService/UpdateUserByName  \
-H "x-dubbo-http1.1-dubbo-version:1.0.0" -H "x-dubbo-service-protocol:dubbo" \
-H "x-dubbo-service-version:1.0.0" -H "x-dubbo-service-group:test" \
-H "Content-Type:application/json" \
 -d '["tc",{"id":"0002","code":1,"name":"tc","age":15}]'
curl http://localhost:8883/UserService/com.dubbogo.pixiu.UserService/GetUserByCode \
-H "x-dubbo-http1.1-dubbo-version:1.0.0" -H "x-dubbo-service-protocol:dubbo" \
-H "x-dubbo-service-version:1.0.0" -H "x-dubbo-service-group:test" \
-H "Content-Type:application/json" \
 -d '[1]'
./start.sh startTest body

Clean

./start.sh clean direct

Start Docker

docker run --name pixiu-gateway -p 8888:8888 dubbogopixiu/dubbo-go-pixiu:latest
docker run --name pixiu-gateway -p 8888:8888 \
    -v /yourpath/conf.yaml:/etc/pixiu/conf.yaml \
    -v /yourpath/log.yml:/etc/pixiu/log.yml \
    dubbogopixiu/dubbo-go-pixiu:latest

Features

  • Multi-protocol support: Currently, Http, Dubbo2, Triple, gRPC protocol proxy and conversion are supported, and other protocols are being continuously integrated.
  • Safety certificate: Support HTTPS, JWT Token verification and other security authentication measures.
  • Registry integration: Support to obtain service metadata from Dubbo or Spring Cloud cluster, support ZK, Nacos registry.
  • Traffic management: Integrate with sentinel, support multiple protocols for rate limiting.
  • Observability: Integrate with opentelemetry and jaeger for distributed tracing.
  • Admin and visual interface: Have pixiu-admin for remote administration and visualization

Control Plane

The pixiu control plane is forked from istio v1.14.3. Offers a variety of capabilities, including service discovery, traffic management, security management.

Contact Us

The project is under intensively iteration, you are more than welcome to use, suggest and contribute codes.

Community

DingDing Group (31203920):

flowchart

We welcome the friends who can give us constructing suggestions instead of known-nothing.

License

Apache License, Version 2.0

dubbo-go-pixiu's People

Contributors

albumenj avatar alchemy-lee avatar alexstocks avatar baerwang avatar bobtthp avatar cityiron avatar dependabot[bot] avatar dongzl avatar ironcity avatar kezhenxu94 avatar laurencelizhixin avatar ma642 avatar mark4z avatar masterkenway avatar maxingg avatar nicktming avatar oaoit avatar pantianying avatar philyue avatar tydhot avatar williamfeng323 avatar wongoo avatar xianlezheng avatar xiaoliu10 avatar yamicro avatar yqxu avatar zgq25302111 avatar zhaokaiwei avatar zouyx avatar ztelur avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

dubbo-go-pixiu's Issues

API gateway config should have version

What would you like to be added:
As the proxy user, I want to config the versions of my API such that I can manage different versions of my APIs.
Why is this needed:

Add access_log

What would you like to be added:
Add access_log filter

Why is this needed:
For problem orientation, when a request is not response success, user need know what happens

merge paramTypes into mappingParams

What would you like to be added:

  1. remove the paramTypes, move it to the mappingParam configuration.
  2. the paramTypes can only have either mapTo or opt. So that the opt fields will not be allocated to the generic invoke call.

Why is this needed:
simplify the configuration on API config.

Make filter configurable

What would you like to be added:
Make filter configurable
Why is this needed:
Filters should have standard configuration injection

URI mapping

What would you like to be added:
retrieve value from URI. For example, giving API config test.com/user/:id, when calling test.com/user/12345, we should be able to retrieve 12345 from id by configuring mapping name uri.id
Why is this needed:

Improved remove dubbogo config file

What would you like to be added:
Remove dubbogo's client.yaml and log.yaml configuration.
Use proxy configuration to adapter the dubbogo consumer.

Why is this needed:
Dubbogo Proxy is project not juet for dubbogo, it need low coupling.

replace new pixiu mascot

What would you like to be added:

Why is this needed:
now, we have a new mascot pic and replace the old one

resolve .yaml config for http forword.

resolve .yaml config for http forward. and put the information into API.httpBackendConfig struct.

http forward need more information to make the behavior clear. such as proxy rule, rewrite rule, head replace, head append and so on.

Add api load

What would you like to be added:
Api config can load from e.g: api_config.yaml, or config center like nacos.

Why is this needed:
Gateway must can control api, so api config load from remote or local.

add more integrate test

What would you like to be added:
now, we have one integrate test example, add more

Why is this needed:

lost the origin URL path in filter.

What happened:
There is a bug in replacepath filter. In the filter, it will replace the original URL path with the configured target path. It makes the URI parameters value missing. So before this bug fixed, the URI analysis will not be able to use(for HTTP-to-http proxy).
What you expected to happen:
The origin URL should be retained.
How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

Add auto service discovery

What would you like to be added:
When config register info, will auto load from remote, change the service to a api(up/down) from rule.

Why is this needed:
When we not use dubbogo generic invoke, we need load invoke obj for self.

Add rate limit filter

What would you like to be added:
A filter can control the request rate, sentinel-go may be a good idea.

Why is this needed:
Rate limit avoid heavy traffic make back-end service not available.

Router to route the requests

What would you like to be added:
As a developer, I want to init a router to manage the requests that send to the proxy. The router should be able to find the target API, add a new target API, update and remove an existing API.
Why is this needed:

Add cluster deployment

What would you like to be added:
Proxy can run more than one instance for a cluster, consider data synchronization, load balancing.

Why is this needed:
Cluster deployment is very important for really use online.

Add white/black list

What would you like to be added:
Filter for white/black, can be ip address, backend endpoint.

Why is this needed:
Control if ip can request to backend or backend can be call by proxy

Param mapping for HTTP-2-HTTP request

What would you like to be added:
As a user, I want to be able to configure the mapping relationship between the integration parameters. If no set, then the parameters from the inbound request will directly send to the integration server.
Why is this needed:

user can add customized filter

What would you like to be added:
allowing user add their customized filter
Why is this needed:
works of filters embedded in dubbo-go-proxy may not satisfied.So,adding customized filter is useful very much

Add http request forwarding

What would you like to be added:
Support http request forwarding.

Why is this needed:
HTTP forwarding are gateway basic functionality.

Add metric

What would you like to be added:
Metric,support prometheus first

Why is this needed:
Export prometheus format info by HTTP URL

dubbogo 1.5.1 up to 1.5.5

What would you like to be added:
Now proxy is based on 1.5.1, but laster is 1.5.5, We should upgrade

Why is this needed:

API Gateway Configuration

What would you like to be added:
As a user, I want to be able to config my own API gateway. I should be able to config the exposed API's parameters(query, path, body), path, headers, and status. and I should be able to config the backend service, it could be Dubbo or HTTP, and I should be able to load the config from a local YAML file.
Why is this needed:

Add high extensible filter and plugin

What would you like to be added:
Design filter plugin, first load plugins from the specified path. (dynamic loading can open a new issue).

Why is this needed:
Plugin can add by customer for self, no changes to the project code are required.

Add call multi cluster

What would you like to be added:
Support backend have more than one register center(multi cluster), may be nacos cluster, zk cluster etc

Why is this needed:
When proxy use as a gateway, need call one than one Independent cluster.

api_config.yaml中使用通用形式(dubbo request universality)无法调用dubbo

What happened:

参照sample/dubbogo/proxy/api_config.yaml里的配置和docs/sample/dubbo-universality.md里的方式,并没有返回期望的POST结果

What you expected to happen:

  • Request
curl host:port/api/v1/test-dubbo/UserProvider/com.ic.user.UserProvider?method=GetUserByName&group=test&version=1.0.0 -X POST -d '{"types":["java.lang.String"],"values":"tc"}' --header "Content-Type: application/json"
  • Response
{
    "age": 18,
    "code": 1,
    "iD": "0001",
    "name": "tc"
}

How to reproduce it (as minimally and precisely as possible):

  1. 使用2020/12/22最新的develop代码32fda76a 编译 sample/dubbogo/server/appcmd/proxy/
  2. 使用sample/dubbogo/server/config/的配置启动server
  3. 使用sample/dubbogo/proxy/的配置启动proxy
  4. 所有配置中仅仅修改了zk的真实地址
  5. 使用Get请求发送到127.0.0.1:8888/api/v1/test-dubbo/user?name=tc返回期望结果
  6. 使用POST请求发送common请求,返回404

Anything else we need to know?:

  1. 上述server和proxy运行在golang的pod中
  2. 使用的镜像为docker hub上的golang:1.14-stretch, 即: go version go1.14.6 linux/amd64
  3. server与proxy均使用该pod完成编译,再分别运行二进制文件
  4. 运行结果如下:
# cat /etc/os-release 
PRETTY_NAME="Debian GNU/Linux 9 (stretch)"
NAME="Debian GNU/Linux"
VERSION_ID="9"
VERSION="9 (stretch)"
VERSION_CODENAME=stretch
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

# uname -a
Linux my-hostname-8c9f6b5b6-t8jwx 3.10.0-957.el7.x86_64 #1 SMP Thu Oct 4 20:48:51 UTC 2018 x86_64 GNU/Linux

# ps -ef
UID         PID   PPID  C STIME TTY          TIME CMD
root          1      0  0 Dec15 ?        00:00:00 tail -f /etc/hosts
root       7098      0  0 09:44 ?        00:00:00 bash
root       7918   7098  0 09:49 ?        00:00:00 bash server-start.sh
root       7919   7918  0 09:49 ?        00:00:07 /go/bin/dubbo-proxy/dubbo-go-server-proxy
root       7936   7098  0 09:49 ?        00:00:00 bash proxy.sh
root       7937   7936  0 09:49 ?        00:00:06 /go/bin/dubbo-proxy/dubbo-go-proxy start -c /go/src/dubbo-proxy/sample/dubbogo/proxy/conf.yaml -a /go/src/dubbo-proxy/sample/dubbogo/proxy/api_config.yaml

# curl -i  http://127.0.0.1:8888/api/v1/test-dubbo/user?name=tc
HTTP/1.1 200 OK
Content-Type: application/json;charset=UTF-8
Date: Tue, 22 Dec 2020 01:58:27 GMT
Content-Length: 43

{"age":18,"code":1,"iD":"0001","name":"tc"}

# curl -i localhost:8888/api/v1/test-dubbo/UserProvider/com.ic.user.UserProvider?method=GetUserByName&group=test&version=1.0.0 -X POST -d '{"types":["java.lang.String"],"values":"tc"}' --header "Content-Type: application/json"
[1] 7987
[2] 7988
bash: -X: command not found
[2]+  Done                    group=test
root@golang-ecp-ewell-8c9f6b5b6-t8jwx:/go# HTTP/1.1 404 Not Found
Date: Tue, 22 Dec 2020 01:59:30 GMT
Content-Length: 18
Content-Type: text/plain; charset=utf-8

404 page not found
[1]+  Done                    curl -i localhost:8888/api/v1/test-dubbo/UserProvider/com.ic.user.UserProvider?method=GetUserByName

Code clean-up

What would you like to be added:
Clean up the unnecessary codes
Why is this needed:

Incorrect version field

What happened:
image
pixiu is now v0.2.0 version, but the version field is still 0.1.0

What you expected to happen:

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

Etcd test cases sometimes don't work properly

What happened:
test case in pkg/remoting/etcd3/client_test.go line 285
sometimes it can pass, sometimes can't

What you expected to happen:

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

Mapping the inbound parameters to integration parameters

What would you like to be added:
As a user, I want to be able to configure the mapping relationship between the integration parameters. If no set, then the parameters from the inbound request will directly send to the integration server.

Why is this needed:

Resources level filter

What would you like to be added:
Currently, only the filters configured in the method level will be extracted and process. We should allow resources level filter configuration
Why is this needed:

Add admin server

What would you like to be added:
A admin server, can accept http request or grpc request for dynamically update some configuration at runtime.

Why is this needed:
Proxy not need restart when running.

API to manage API configuration

What would you like to be added:
As a user, I want to be able to manage the API configuration via restful API.
Why is this needed:

An embarrassing issue

Sorry for my disturbing. I'm a student, and I'm interested in dubbo-go-pixiu. Besides, I'm also planning to apply for the ospp program instructed by @xiaoliu10 but I have some troubles. I've sent an email but didn't receive after three days. So I'll really appreciate it if I could get in touch with the mentor @xiaoliu10 . Sorry for my bothering again!

Automatically retrieve paramTypes from metadata center

What would you like to be added:

  1. When no configuration for param types, it should be able to retrieve the paramType list from the metadata center if exists.
  2. The manual paramType configuration overwrites the paramType list from the metadata center.
  3. Invoke generic call base on the metadata paramTypes.

*** The metadata center will not automatically update if the service updates its parameter types. It would trigger an unsuccessful invoke in the proxy due to the incorrect parameter type.

Why is this needed:
Reduce the configuration effort. Reduce the barrier to start using the Pixiu.

dubbo-go-proxy user list

Welcome to use dubbo-go-proxy. To know who is using it, pls append your org info as follow,

  • Organization:Alibaba (Required)
  • Location: Hangzhou, Zhejiang, China(Required)
  • Contact: [email protected] (Optional)
  • Purpose:gateway for microservice lib (Required)
  • Version: v1.0.0(Required)
  • Icon:(Optional)
    Thanks again for your participation!

欢迎使用dubbogo,首先感谢你的使用,其次您可以参考下面的样例来提供您的信息以收集下使用场景:

  • 组织:阿里巴巴(Required)
  • 地点:**杭州(Required)
  • 联系方式:[email protected](Optional)
  • 场景:服务化后接口网关(Required)
  • 版本:v1.0.0(Required)
  • 公司logo:(Optional)
    再次感谢你的参与!!!

pprof config less double quote

now:
PprofConf PprofConf yaml:"pprofConf json:"pprofConf" mapstructure:"pprofConf"
change to :
PprofConf PprofConf yaml:"pprofConf" json:"pprofConf" mapstructure:"pprofConf"``

yaml:"pprofConf this code is missing a double quote

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.