apache/servicecomb-mesher

A high performance service mesh implementation written in go

Mesher

A service mesh implementation based on go chassis.

Why use mesher

• any infrastructure: if you use ServiceComb as control plane, you can run on any infrastructure(docker, kubernetes,VM, bare metal).
• service mesh and frameworks: you can develop micro services with java chassis or go chassis frameworks and use mesher to make other service join to the same system.
• flexible: you can develop and customize your own service mesh
• OS: support both linux and windows OS, which means you can govern your services writen in .net with java, go etc.
• API gateway: mesher is able to run as a independent edge service and manage ingress traffic.

Features

• Build on top of go micro service framework: so that mesher has all of features of go chassis,a high flexible go micro service framework. you can custom your own service mesh and API gateway by extending lots of components.
• Admin API：Listen on an isolated port, expose useful runtime information and metrics.
• support protocols: http and grpc
• No IP tables forwarding: Mesher leverage http_proxy and grpc proxy dialer, that makes better performance than using ip tables
• local health check: as a sidecar, mesher is able to check local service health by policy and dynamically remove it from service registry if service is unavailable.

Get started

Refer to mesher-examples

How to build

Build from scratch

1. Install ServiceComb service-center

2. build and run, use go mod

export GOPROXY=https://goproxy.io #if you are facing network issue
cd mesher
GO111MODULE=on go mod download
#optional
GO111MODULE=on go mod vendor
cd cmd/mesher
go build mesher.go

Build by script

cd build
export GOPATH=/path/to/gopath
export GOPROXY=https://goproxy.io #if you are facing network issue
./build_proxy.sh

it will build binary and docker image

• tar file: release/mesher-latest-linux-amd64.tar
• docker image name: servicecomb/mesher-sidecar:latest

Build mesher edge service

sudo docker build -t servicecomb/mesher-edge -f docker/edge/Dockerfile .

How to run

edge service

sudo docker run -e PAAS_CSE_SC_ENDPOINT="" \
-e PAAS_CSE_CC_ENDPOINT="" \
-p 80:30101 \
--name mesher-edge servicecomb/mesher-edge

Documentations

You can see more documentations in here, this online doc is for latest version of mesher, if you want to see your version's doc, follow here to generate it in local

RoadMap

1.7.0

• mesher ingress: mesher as a edge service to manage ingress traffic
• integrate with servicecomb kie: user can change mesher-sidecar and mesher-ingress in same config server, don't need to change local files

in future

• support skywalking

servicecomb-mesher's Issues

mesher如何从配置中心读取数据

我写的servicemesh是连接了配置中心了，但如何从配置中心获取下发的配置信息呐？

Mesher无法支持动态判断服务端是否开启SSL

Mesher不支持动态判断服务端是否开启SSL，只能在配置文件中写死协议类型，也就是说全系统必须统一为HTTP或HTTPS。

具体信息参考此issue
#382

通过mesher接入，调用失败，日志报错

问题描述：c应用和d应用都通过各自的mesher接入本地注册中心，c应用通过d应用的微服务名调用d的服务失败。

c应用->mesherC->mesherD->d应用

实际截图：
注册中心显示c和d都注册成功，d里面可以看到契约。

c应用中调用的代码：

mesherD的日志显示如下错误：

有遇到过的帮忙看看这是啥原因？

微服务可以通过mesher访问外部http服务吗？

mesher对接的微服务如何访问外部服务呢？

访问内部服务时，url中直接填服务名就行，mesher的负载均衡模块会将服务名映射为ip:port。

那如果是外部服务呢，如果url中填的是外部的域名(http://github.com/......)，此时能通过mesher代理请求吗？

servicecomb-mesher对接pilot，缺乏pilotv2 router的plugin

按照官网（https://docs.go-chassis.com/istio-guides.html）指引配置mesher和istio控制面pilot集成，在配置服务发现没问题，服务可以正常互通。但是想进一步配置路由规则，当按照官网教程配置router.yaml为pilotv2平台后，pod生成会出错误，显示没有安装pilotv2这个路由插件。已和社区owner Xiaoliang.tian沟通过，缺乏pilot v2 router的plugin，希望大家贡献！以下是问题贴图：

能用来向华为公有云CSE注册微服务吗？

我想通过servicecomb-mesher向华为公有云注册微服务，配置好ak、sk及服务注册地址后，出现如下错误：
2020-11-10 10:13:00.964 +08:00 WARN servicecenter/servicecenter.go:36 service [hellompython:0.0.2:proxy_pythonhttp] not exists in registry, register it
2020-11-10 10:13:01.024 +08:00 ERROR servicecenter/servicecenter.go:39 register service [hellompython:0.0.2:proxy_pythonhttp] failed, err: RegisterService failed, MicroServiceName/responseStatusCode/responsebody: hellompython/401/{"errorCode":"401002","errorMessage":"Request unauthorized","detail":"invalid request, header is invalid, ak sk or project is empty"}

2020-11-10 10:13:01.024 +08:00 ERROR registry/bootstrap.go:95 Register [hellompython] failed: RegisterService failed, MicroServiceName/responseStatusCode/responsebody: hellompython/401/{"errorCode":"401002","errorMessage":"Request unauthorized","detail":"invalid request, header is invalid, ak sk or project is empty"}

2020-11-10 10:13:01.024 +08:00 ERROR registry/registrator.go:78 start backoff for register microservice: RegisterService failed, MicroServiceName/responseStatusCode/responsebody: hellompython/401/{"errorCode":"401002","errorMessage":"Request unauthorized","detail":"invalid request, header is invalid, ak sk or project is empty"}
我有一个别人编译好的windows下的mesher可以向华为云，版本为mesher-1.4.12-windows-amd64，能否告知原因？

当前mesher是否支持扩应用调用

服务名调用问题

servicemesh服务名调用只能小写字母。即对方服务的服务名只能写成obpservice，不能写成OBPservice

在MAC上无法运行

Mesher在转发请求时将header中的Connection移除了，这个是为什么

servicecomb-mesher/proxy/protocol/http/sidecar.go

Line 324 in 6d1d269

req.Header.Del("Connection")

mesher支持使用grpc协议吗？

How to use gateway in mesher?

I haven't found a tutorial for using the gateway in mesher. Can you provide some examples or tutorials?

运行mesher.exe报错提示

failed: Get "http:///v4/default/registry/existence?type=microservice\u0026appId=defau
lt\u0026serviceName=calculator\u0026version=1.1.1": http: no Host in request URL"}

根据快速入门操作，运行mesher.exe，注册中心没有对应服务。

mesher是否支持按照api级进行流量控制？

servicecomb-mesher何时更新

servicecomb-mesher 最新 tag 为 v1.6.3
使用的go-chassis 版本为 1.6.1 什么时候会更新到 go-chassis v2.X 版本。 本来还想用go-chassis呢，一看servicecomb-mesher要当作api网关呢， 一看两者版本差距这么大。会有影响吗？

servicecomb-edge调用mesher代理的服务失败

看了文档，也运行了bmi的实例，基本上理解mesher的工作原理以及使用方法，但我疑惑的是servicecomb体系所有的通信都是围绕“契约”的，如果说我现在的新系统是基于servicecomb的，所有新的微服务也是使用 java-chasis 编写，但有一个老的服务B使用的是nodejs可以直接复用，那我使用mesher将该服务集成进新的系统。
假设现在使用一个基于java-chasis开发的微服务A调用B
1.是否B需要手动编写“契约”文件呢？因为A调用B的API时会从SC拉取契约，如果没有契约的话，调用应该会失败
2.假设B需要提供契约文件，为什么我每次启动mesher（1.6.3）之后，我写的契约都会删除，然后mesher自己创建一个新的契约文件
目前测试的结果使用 request -> edge (java) -> mesher -> nodejs，edge会报错

mesher version unknown，init classis fail

C:\Users\wwx1002335\AppData\Local\Temp\VERSION not found, mesher version unknown
init chassis fail: [C:\Users\wwx1002335\AppData\Local\Temp\conf\chassis.yaml] file not exist

运行就报上述错误

go build Mesher error when add zipkin tracing

i add zipkin tracer by :
1、Import _ "github.com/go-chassis/go-chassis-plugins/tracing/zipkin" in mesher.go.
2、Add chassisHandler.TracingConsumer for consumerChain in file bootstrap.go.
3、Add chassisHandler.TracingProvider for providerChain in file bootstrap.go.
then go build mesher.go, and get an error:
go/pkg/mod/github.com/go-chassis/[email protected]/tracing/zipkin/zipkin.go:75:3: undefined: zipkintracer.WithPrefixTracerState

mesher的升级计划

@tianxiaoliang hi, 麻烦问下
mesher当前社区版本最新1.6.3是19年9月发布，超两年没有更新，已达到生命周期，影响下期选型
帮忙确认下，后续社区是否升级计划？

无法正确获取client ip

场景：使用mesher的服务A与同样使用mesher的服务B

个人分析结论：
A调用B服务时，在B服务处无法获取到A服务的client ip。很大的可能是在mesher B的代理过程中，将这个IP丢弃。

正常的情况下，通过代理（如nginx、apache httpd）后，会提供写入x-forward-for请求头的能力。
是否mesher需要增加类似的配置能力？

采用容器部署的方式，一个pod，两个容器,业务容器还没启动成功，mesher容器已经启动成功，已经注册到service-center.

有什么机制可以规避这种问题。

Admin APIs not working when running as Container

How to register schema in mesher to make intercalls from other microservices.

I have java chassis SDK based serviceA talks to mesher acting as sidecar proxy for the python based serviceB.
When I call serviceB operations via mesher with restclient like this

		 RestTemplate restTemplate = RestTemplateBuilder.create();
	    String result =
	            restTemplate.getForObject("cse://testApp:serviceB/shippingservice/bmi?height={height}&weight={weight}"
	            		, null, String.class,request.getHeight(),request.getWeight());

I am always ended up in Message Not Found Error.

{
    "message": "Not Found"
}

Relevant logs

2023-03-02 13:12:31.428  INFO 1206 --- [oup0-1-thread-1] o.a.s.registry.swagger.SwaggerLoader     : try to load schema locally, appId=[testApp], serviceName=[serviceB], schemaId=[shippingservice]
2023-03-02 13:12:31.444  WARN 1206 --- [oup0-1-thread-1] o.a.s.registry.swagger.SwaggerLoader     : no schema in local, and can not get schema from service center, appId=Elavon, microserviceName=shippingservice, version=1.1.1, serviceId=387916b7066a225101afb347c009988e1a78ea67, schemaId=shippingservice.

Then I realised that mesher would not know about the schema/operations supported by the sidecar main serviceB which it proxies.

So the question is how to register the schema in mesher with the operations supported in the microservice in serviceB
I have tried creating schema yaml file and put it under mesher/conf folder. But it does not work. The schemas are not getting uploaded or registered in the service center.

Please suggest how to make intercommunication calls from serviceA to serviceB through mesher?

Wiki Page missing for Roadmap of Mesher

mesher支持文件上传的功能吗

注意到java-chassis里有upload的功能，不知道mesher支不支持

Description missing in https://hub.docker.com/r/gomesh/mesher/

小bug?

https://github.com/apache/servicecomb-mesher/blob/master/proxy/cmd/cmd.go

func parseConfigFromCmd(args []string) (err error) {
app := cli.NewApp()
app.HideVersion = true
app.Usage = "a service mesh that governance your service traffic."
app.Flags = []cli.Flag{
cli.StringFlag{
Name: "config",
Usage: "mesher config file, example: --config=mesher.yaml",
Destination: &Configs.ConfigFile,
},
cli.StringFlag{
Name: "mode",
Value: common.RoleSidecar,
Usage: fmt.Sprintf("mesher role [ %s|%s|%s ]",
common.RolePerHost, common.RoleSidecar, common.RoleEdge),
Destination: &Configs.Role,
},
cli.StringFlag{
Name: "service-ports",
EnvVar: common.EnvServicePorts,
Usage: fmt.Sprintf("service protocol and port,examples: --service-ports=http:3000,grpc:8000"),
Destination: &Configs.LocalServicePorts,
},
}
app.Action = func(c *cli.Context) error {
return nil
}

err = app.Run(args)
return

}

**这句代码 EnvVar: common.EnvServicePorts, 应该 是 EnvVar: os.Getenv(common.EnvServicePorts), 吧？ 求证 ？ **

where to configure the localHealthCheck

在哪配置 localHealthCheck

servicescomb-mesher不能使用go-chassis-cloud来提供公有云接口？

Where is the code for mesher side car injector?

I see the links in the mesher documentation points to dead links.

Can you please share the side car injector links?

Support third-party system authentication

Now is doing the authorization system, if someone had suggestion or interested of this. We can discuss of this detailed.

runtime error: invalid memory address or nil pointer dereference

resp.Body为nil的话 这里抛出异常了
io.Copy(w, resp.Body)
resp.Body.Close()
路径： /github.com/go-mesh/mesher/protocol/http/reverse_proxy.go:214
func copyChassisResp2HttpResp(w http.ResponseWriter, resp *http.Response) {
postProcessResponse(resp)
copyHeader(w.Header(), resp.Header)
w.WriteHeader(resp.StatusCode)
if resp == nil {
openlogging.GetLogger().Warn("response is nil because of unknown reason, plz report issue")
return
}
io.Copy(w, resp.Body)
resp.Body.Close()
}

异常：
goroutine 7435 [running]:
net/http.(*conn).serve.func1(0xc4212375e0)
/opt/go/src/net/http/server.go:1697 +0xd0
panic(0x13eaaa0, 0x1cb7eb0)
/opt/go/src/runtime/panic.go:491 +0x283
io.copyBuffer(0x1ccf9c0, 0xc420bae0d0, 0x0, 0x0, 0xc4214a8000, 0x8000, 0x8000, 0xc420bae0d0, 0xc420bae0d0, 0x1429fe0)
/opt/go/src/io/io.go:392 +0x102
io.CopyBuffer(0x1ccf9c0, 0xc420bae0d0, 0x0, 0x0, 0xc4214a8000, 0x8000, 0x8000, 0xc420cba9c0, 0x3, 0xc42001f300)
/opt/go/src/io/io.go:373 +0x82
net/http.(*response).ReadFrom(0xc420c42700, 0x0, 0x0, 0x0, 0x0, 0x0)
/opt/go/src/net/http/server.go:564 +0x557
io.copyBuffer(0x1cccb80, 0xc420c42700, 0x0, 0x0, 0x0, 0x0, 0x0, 0x14e1540, 0x0, 0x1cccb80)
/opt/go/src/io/io.go:386 +0x2bb
io.Copy(0x1cccb80, 0xc420c42700, 0x0, 0x0, 0x0, 0x0, 0x0)
/opt/go/src/io/io.go:362 +0x68
github.com/go-mesh/mesher/protocol/http.copyChassisResp2HttpResp(0x1cd6d00, 0xc420c42700, 0xc421315050)
/opt/build/workspace/mesher/br_fst2.3/compile/src/github.com/go-mesh/mesher/protocol/http/reverse_proxy.go:214 +0x11a
github.com/go-mesh/mesher/protocol/http.handleRequest(0x1cd6d00, 0xc420c42700, 0xc4213f66c0, 0xc420e485d0, 0xbc56139881, 0x1d27c40, 0x0)
/opt/build/workspace/mesher/br_fst2.3/compile/src/github.com/go-mesh/mesher/protocol/http/reverse_proxy.go:239 +0x168
github.com/go-mesh/mesher/protocol/http.LocalRequestHandler(0x1cd6d00, 0xc420c42700, 0xc4214a0000)
/opt/build/workspace/mesher/br_fst2.3/compile/src/github.com/go-mesh/mesher/protocol/http/reverse_proxy.go:136 +0x49d
net/http.HandlerFunc.ServeHTTP(0x1568788, 0x1cd6d00, 0xc420c42700, 0xc4214a0000)
/opt/go/src/net/http/server.go:1918 +0x44
net/http.serverHandler.ServeHTTP(0xc4201ae000, 0x1cd6d00, 0xc420c42700, 0xc4214a0000)
/opt/go/src/net/http/server.go:2619 +0xb4
net/http.(*conn).serve(0xc4212375e0, 0x1cdc780, 0xc4214b8800)
/opt/go/src/net/http/server.go:1801 +0x71d
created by net/http.(*Server).Serve
/opt/go/src/net/http/server.go:2720 +0x288

mode=edge网关调用失败

mesher是master分支版本

mesher.yaml文件：

mesher:
  ingress:
    type: servicecomb
    rule:
      http: |
        - host: example.com
          apiPath: /sayhello/peter
          service:
            name: RESTServer

service:

{
  "serviceId": "df61d6a27f81b5d2d7e7a76948d6af34fc40cf2c",
  "appId": "default",
  "serviceName": "RESTServer",
  "version": "0.0.1",
  "level": "BACK",
  "schemas": [
    "RESTServer"
  ],
  "status": "UP",
  "timestamp": "1605178252",
  "alias": "default:RESTServer",
  "modTimestamp": "1605178252",
  "registerBy": "SDK",
  "framework": {
    "name": "Go-Chassis",
    "version": "v2.0.2"
  }
}

执行命令：

http_proxy=http://127.0.0.1:30101 curl 'http://example.com/sayhello/peter'
curl: (52) Empty reply from server

mesher日志：

{"level":"ERROR","timestamp":"2020-11-13 12:07:08.461 +08:00","file":"handler/transport_handler.go:53","msg":"call err [Get \"http://example.com/sayhello/peter\": http: Request.RequestURI can't be set in client requests]"}
2020/11/13 12:07:08 http: panic serving 127.0.0.1:54562: invalid WriteHeader code 0

不清楚是哪里设置不对，还是执行curl命令不对？

如何跨app_id调用service？

在microservice.yaml中有一个 allowCrossApp 配置项，允许跨app调用service，但是在调用服务时如何指定app_id呢？

容器运行时，app_id 被清空

我用本地conf文件挂载，运行gochassis/mesher:1.5rc时，
发现我在conf配置的app_id被清空，导致我访问其他微服务时失败。
运行的docker-compose文件：https://github.com/msnetc/docker-node/blob/master/docker-compose.yml

Does Mesher stop evolving?

Mesher项目不演进了吗？

mesher 如何修改健康检查时间及次数

healthCheck

mesher代理能否实现grpc请求的转发，我们的微服务都是提供的grpc接口，该怎么对接mesher

Mesher后续是否还有升级计划？

mesher当前社区版本最新1.7.0，后续社区是否有升级计划？
这个版本再往后会超出软件使用的生命周期，影响下期选型

自定义incoming handler chain覆盖原本default handler chain

我自定义了一个handler将其配置到 配置文件中。但是这导致了，chassis那边的restful服务器获取到了 provider-incomming链。这段逻辑在server_manger.go里面。
没有配置自定义handler时，restful服务器获取的是默认的 provider-default链，mesher自定义的协议如dubbo,http,grpc是获取的provider-incomming链，本来互补影响。
但是自定义后，现在restful被换成了provider-incomming链（rest-admin协议会使用chassis自带的restful）。

目前我想到的解决方案是
1.暴力的去掉 server_manager中的，for name := range providerMap逻辑，默认server都从default链中获取,这样我们只需要配置文件中同时配置default和incomming 链就可以解决问题。
2.根据不同协议获取不同 链，如incomming 这个可以当成协议名
如：

handler：
   chain:
       Provider:
           rest:
           http:
           default:

如果没有在配置文件配置则获取default链。
ps：我是不是该把issue提到chassis那边去。

File suffixes in docs inconsistent

File suffixes in docs are inconsistent. Some are markdown format while others are rst fomat. It is ok for user to read and study from mesher docs but not convenient for contributers to optimzie docs. Shall we normalized the formats?

mesher是否支持禁止Go客户端自动重定向