mosn/mosn

The Cloud-Native Network Proxy Platform

中文

MOSN (Modular Open Smart Network) is a cloud-native network proxy written in Go language. It is open sourced by Ant Group and verified by hundreds of thousands of production containers in 11.11 global shopping festival. MOSN provides the capabilities of multiple protocol, modularity, intelligent and security. It integrates a large number of cloud-native components, and also integrates a Envoy network library, which is high-performance and easy to expand. MOSN and Istio can be integrated to build Service Mesh, and can also be used as independent L4/L7 load balancers, API gateways, cloud native Ingress, and etc.

Core capabilities

• Istio integration
  • Integrates Istio 1.10 to run in full dynamic resource configuration mode
• Core forwarding
  • Supports a self-contained server
  • Supports the TCP proxy
  • Supports the UDP proxy
  • Supports transparent traffic hijack mode
• Multi-protocol
  • Supports HTTP/1.1 and HTTP/2
  • Supports protocol extension based on XProtocol framework
  • Supports protocol automatic identification
  • Supports gRPC
• Core routing
  • Supports virtual host-based routing
  • Supports headers/URL/prefix/variable/dsl routing
  • Supports redirect/direct response/traffic mirror routing
  • Supports host metadata-based subset routing
  • Supports weighted routing.
  • Supports retries and timeout configuration
  • Supports request and response headers to add/remove
• Back-end management & load balancing
  • Supports connection pools
  • Supports persistent connection's heart beat handling
  • Supports circuit breaker
  • Supports active back-end health check
  • Supports load balancing policies: random/rr/wrr/edf
  • Supports host metadata-based subset load balancing policies
  • Supports different cluster types: original dst/dns/simple
  • Supports cluster type extension
• Observability
  • Support trace module extension
  • Integrates jaeger/skywalking
  • Support metrics with prometheus style
  • Support configurable access log
  • Support admin API extension
  • Integrates Holmes to automatic trigger pprof
• TLS
  • Support multiple certificates matches, and TLS inspector mode.
  • Support SDS for certificate get and update
  • Support extensible certificate get, update and verify
  • Support CGo-based cipher suites: SM3/SM4
• Process management
  • Supports hot upgrades
  • Supports graceful shutdown
• Extension capabilities
  • Supports go-plugin based extension
  • Supports process based extension
  • Supports WASM based extension
  • Supports custom extensions configuration
  • Supports custom extensions at the TCP I/O layer and protocol layer

Download&Install

Use go get -u mosn.io/mosn, or you can git clone the repository to $GOPATH/src/mosn.io/mosn.

Documentation

• Website
• Changelog

Contributing

See our contributor guide.

Partners

Partners participate in MOSN co-development to make MOSN better.

End Users

The MOSN users. Please leave a comment here to tell us your scenario to make MOSN better!

Ecosystem

The MOSN community actively embraces the open source ecosystem and has established good relationships with the following open source communities.

Community

See our community materials on https://github.com/mosn/community.

Visit the MOSN website for more information on working groups, roadmap, community meetings, MOSN tutorials, and more.

Scan the QR code below with DingTalk(钉钉) to join the MOSN user group.

Community meeting

MOSN community holds regular meetings.

• Wednesday 8:00 PM CST(Beijing) every other week
• Meeting notes

Landscapes

  

MOSN enriches the CNCF CLOUD NATIVE Landscape.

mosn's Issues

global variable Adap in cluster/clusteradapter need fix

In pkg/upstream/cluster/clusteradapter.go defines a global variable "var Adap Adapter".
The "Adap" is an extension for cluster manager , for example, it can be used in a service discovery extension.
It is hard to understand, should be changed to factory mode, just like the other extensions.(filter, protocol, router)

"unknown cluster" or "no healthy upstream" error occur

"unknown cluster" or "no healthy upstream" error occur somtimes shortly after mosn start and the request failed.
that's because the design fo update LDS/CDS/EDS in mosn dose not fit ADS, some config become effective earlier than others.

also some tr logic remained, need delete completely

vendor directory and Gopkg.lock has been ignored

There maybe something wrong with sofa-mosn/.gitignore
record below should be remove

vendor/github.com/x/*
Gopkg.lock
vendor/**
vendor/golang.org/x/net/*
vendor/github.com/urfave/*

Reopen: Follow the golang code specification

More specification need considered

support host with weight when lb

support network filter extension

1. starter check network filter should have only one needs to be removed
2. add network filter registry

if upstream server is closed, downstream client send a connect will trigger panic

if the upstream sever is close, client make a request, then the mesh will trigger a panic, and then auto recover.

test on protocol http2 trigger the panic, no test on xprotocol, but the code same as http2, exists the same problem.
http will not panic, because http does not have the panic code;
sofarpc will not panic, because sofarpc have nil check before use.

panic code in pkg/stream/${protocol}/connpool.go:
func NewStream() {
if p.primaryClient == nil {
p.primaryClient = newActiveClient(context, p)
}
// sofarpc recheck whether the primaryClient is nil
if !CanCreate() {
} else {
p.primaryClient.totalStream++ //primaryClient is nil , http does not have it
}
}
The newActiveClient will return nil, if the server is closed (connection refused), then use the primaryClient will trigger panic

use sync.Map replace concurrentmap

Multi-core performance optimization

Optimize threading model, IO, and memory footprint, so that MOSN's IO and task scheduling mechanisms can fully utilize the capabilities of multiple cores. Performance testing report is REQUIRED.

The optimization is protocol irrelevant, and the testing is mainly based on SofaRpc flow.

Integrate with CI, code coverage and quality check tools

It is important to use the devops tools to guarantee the SOFA MOSN project.
Use travis for continued integration, codecov for coverage report, goreportcard for static code quality report.

test case randomly failed

scenetest部分测试case在执行go test的时候可能存在一定概率的失败，原因是因为这部门case逐个场景模拟，在一些性能较差的机器或者docker中运行的时候，可能会导致部分行为运行时间超过预期而失败，如上一个case的mesh占用的端口释放较慢，下一个case已经开始运行了，就会导致新的case listen 失败导致case失败；
healthchecker_test.go中的Test_timeout 也存在一定的失败概率，原因应该类似。

GC optimization & reuse in core forwarding flow

Mosn reclaims and reuses the temporary memory in each module the handling mechanism is relatively fragmented and unsystemized. It is necessary to perform an integrated memory reuse scheme based on the main stream forwarding flow, which provides simple memory usage pattern and better gc performance due to smaller memory footprint.

update docs as layout and files changed

quicstart,
examples
and so on

improvement for JSON operation

Replace native JSON operation with open source implementation:

https://github.com/json-iterator/go

Consider upgrade MOSN dependencies version

Check and decide whether to upgrade MOSN dependencies version or not.

some constraint not fit for xds mode

1. no static cluster is allow in xds mode
   func ParseClusterConfig(clusters []ClusterConfig) ([]v2.Cluster, map[string][]v2.Host) {
   if len(clusters) == 0 {
   log.StartLogger.Fatalln("No Cluster provided in cluster config")
   }
   ...
   }

2. some listener do have no VirtualHosts
   func ParseProxyFilterJSON(c *v2.Filter) *v2.Proxy {
   ...
   if len(proxyConfig.VirtualHosts) == 0 {
   log.StartLogger.Fatal("No VirtualHosts Founded")
   }
   ...
   }

3. is hard to change key when convert Proxy to Config
   type Filter struct {
   Name string
   Config map[string]interface{}
   }
   type Proxy struct {
   Name string json:"name"
   DownstreamProtocol string json:"downstream_protocol"
   UpstreamProtocol string json:"upstream_protocol"
   SupportDynamicRoute bool json:"support_dynamic_route"
   BasicRoutes []*BasicServiceRoute json:"basic_routes"
   VirtualHosts []*VirtualHost json:"virtual_hosts"
   ValidateClusters bool json:"validate_clusters"
   }

4. Cluster_STATIC must convert to SIMPLE_CLUSTER in mosn

HTTP 2 request with prefix '/' not work

Steps:

1. Config v2.RouterMatch with a prefix '/'
2. Start mosn with a running upstream
3. Call http://$ip/

the response code is 404 with no available router

参数ClusterType需要校验

在cluster类中的func NewCluster(clusterConfig v2.Cluster, sourceAddr net.Addr, addedViaAPI bool) types.Cluster函数. 返回值newCluster很有可能为nil.因为clusterType没有添加校验.调用这个函数的地方也没有看到校验.

Connections migration on process smooth upgrade

Currently, existing connections are not properly processed during the smooth upgrade process, which depends on the specific protocol's behaviour. For protocols that does not support things like 'goaway', MOSN will wait for a while and then close connections. The goal of this issue is to build a protocol-independent mechanism which transfers existing connections to new process on smooth upgrade.

hosts would be remove when cluster update

when update cluster which type is Cluster_EDS, the hosts would be remove.

function call clusterAdapter.ClusterAdap.TriggerClusterUpdate(cluster.Name, cluster.Hosts) in callback
cluster.Hosts have 0 element when cluster type is Cluster_EDS

Add README_EN.md

Add english version readme

Holding metrics in shared memory between processes

Holding metrics in shared memory between processes, and ensuring no loss of metrics during mosn process smooth upgrades.

Internal type repackage

Types in internal/api/v2 is unable to be imported by third-party project reference.

Change internal to pkg.

Single core performance optimization

Perform upper-layer protocol-independent performance optimization, optimize IO operations, and memory usage to improve overall performance in sidecar single-core mode. Performance testing report is REQUIRED.

Currently we use HTTP/1.x, HTTP/2.0 thrid-party implementation, the optimization mainly focus on SOFARPC flow

优化 mosn 编译

需要提供无 docker 化，mac 下编译linux二进制的能力

HTTP/1.x Performance Tunning

Optimize HTTP/1.x forwarding performance. Performance testing report is REQUIRED.

Support cluster update&delete in ADS integration

In ADS dynamic config mode, cluster update&delete is required to perform on ADS config change

route wildcard domain match rule: longest wildcard suffix match

two questions:
1.
golang map's iteration order is not specified and is not guaranteed to be the same from one iteration to the next.
to longest wildcard suffix match, we need to sort key in wildcardVirtualHostSuffixes.

Only unique values for domains are permitted, even if it is wildcard domain.

Build a extensible mechanism for easy extension of third-parity protocol on tcp

Based on current SOFARPC codec & stream, build a extensible mechanism for easy extension of third-party protocol on raw tcp.

there are some random failure cases when running health check test, need fix

Improve code style in 0.1.0

Improve our code for code style and related.

@junxiong Please set one target for milestone 0.1.0.

And more, in next milestone 0.2.0, we will also have one issue to trace our code style improvement.

Setup basic fuzzy tests for mosn

Cases:

• Any/all downstream reset/close randomly, and then recover
• Any/all upstream reset/close randomly, and then recover
• Inject random delay in the forward flow
• More cases...

support weighted cluster for routing

there are no file in some vender

there are no file in some vender, such as gogo and lyft, and the pkg github.com/golang/protobuf is too old

Configuration naming problem

There are two type names in the config file, like cluster_manager and MaxRequestPerConn. Use Camel case there is in order to make the config option and the field of config struct correspond I think. But why don't we use underscores to write multi-word name only, and use struct tag to make the correspond?

examples/http-sample/server.json:

  "cluster_manager":{
    "auto_discovery": false,
    "registry_use_health_check":false,
    "clusters":[
      {
        "Name": "httpCluster",
	"Type": "SIMPLE",
	"lb_type": "LB_RANDOM",
	"MaxRequestPerConn": 1024,
	"ConnBufferLimitBytes":32768,
	"hosts":[
	  {"Address":"127.0.0.1:8080","Weight": 100}
	]
      }
    ]
  }

type ClusterConfig struct {
	Name                 string
	Type                 string
	SubType              string                   `json:"sub_type"`
	LbType               string                   `json:"lb_type"`
	MaxRequestPerConn    uint32
	ConnBufferLimitBytes uint32
	CircuitBreakers      []*CircuitBreakerConfig  `json:"circuit_breakers"`
	HealthCheck          ClusterHealthCheckConfig `json:"health_check,omitempty"` //v2.HealthCheck
	ClusterSpecConfig    ClusterSpecConfig        `json:"spec,omitempty"`         //	ClusterSpecConfig
	Hosts                []v2.Host                `json:"hosts,omitempty"`        //v2.Host
	LBSubsetConfig       v2.LBSubsetConfig
	TLS                  TLSConfig                `json:"tls_context,omitempty"`
}

Follow the golang code specification

Reference golang specification: https://golang.org/doc/effective_go.html
Golint result > 60%

Enhance http2 stream impl

Enhance http2 stream impl:

• Support connection pool, which integrate with golang http2 roundtrip read loop
• Press test, fix concurrent issues
• Exception test
  + Fix stream.responseWriter panic on upstream restart randomly
  + Fix exception handle on client send error
  + Fix goroutine hang on upstream restart randomly

请统一项目名与代码中的包名

项目名是 sofa-mosn，代码里的包名是 sofamosn，应当统一为一个名字

Support UDP in network layer

Mosn network package supports TCP connections and events, need to support UDP.

delete the hack method "disable io" for http2's read loop

need delete the hack

依赖包问题

能不能依赖包都打到vendor目录去，现在编译要下载依赖包很麻烦啊？感谢

HTTP1 client receive status 200 when upstream server is closed

scene:
HTTP1 client -> mesh(HTTP1) -> mesh(HTTP1) -> HTTP1 Server ( closed)

client receive status code 200

Is it possible to use the same proxy of all connections under a listener?

When I read the code about connection accept, I found that everytime when a new connection come we create a proxy for it.
pkg/server/handler.go

func (al *activeListener) OnNewConnection(ctx context.Context, conn types.Connection) {
	//Register Proxy's Filter
	configFactory := al.networkFiltersFactory.CreateFilterFactory(ctx, al.handler.clusterManager)
	buildFilterChain(conn.FilterManager(), configFactory)
	...

this will call

func (gfcf *GenericProxyFilterConfigFactory) CreateFilterFactory(context context.Context, clusterManager types.ClusterManager) types.NetworkFilterFactoryCb {
	return func(manager types.FilterManager) {
		manager.AddReadFilter(proxy.NewProxy(context, gfcf.Proxy, clusterManager))
	}
}

Because connections are belong to the same listener(they use the same porxy config), so is it possible to use the same proxy object here?

Route's config should be validated on mosn start

Route's config should be validated on mosn start. All allowed rules should be workable.

router match failed is normal, should not print error log

Dockerfile to make rpm using Alipay's private registry

Error at make rpm:

make rpm
docker build --rm -t afenp-rpm-builder build/contrib/builder/rpm
Sending build context to Docker daemon   7.68kB
Step 1/5 : FROM acs-reg.alipay.com/acs/centos:centos7
Get https://acs-reg.alipay.com/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
Makefile:78: recipe for target 'rpm' failed

It get image centos:centos7 from acs-reg.alipay.com/acs/, this is Alipay's private docker registry.
Remove the acs-reg.alipay.com/acs/, it works.

ClusterManager recreate when MOSN server init

In starter.go func NewMosn :

for _, serverConfig := range c.Servers {
    ...
    ...
    //cluster manager filter
    cmf := &clusterManagerFilter{}
    var clusters []v2.Cluster
    clusterMap := make(map[string][]v2.Host)

    // parse cluster all in one
    clusters, clusterMap = config.ParseClusterConfig(c.ClusterManager.Clusters)

    //create cluster manager
    cm := cluster.NewClusterManager(nil, clusters, clusterMap, c.ClusterManager.AutoDiscovery, c.ClusterManager.RegistryUseHealthCheck)
    //initialize server instance
    srv = server.NewServer(sc, cmf, cm)

There create a new ClusterManager everytime in the MOSN server init loop, why should't we use a same ClusterManager?
BTW, clusterMap := make(map[string][]v2.Host) here is not necessary, we can convert

var clusters []v2.Cluster
clusterMap := make(map[string][]v2.Host)
// parse cluster all in one
clusters, clusterMap = config.ParseClusterConfig(c.ClusterManager.Clusters)

to

var clusters []v2.Cluster
var clusterMap map[string][]v2.Host
// parse cluster all in one
clusters, clusterMap := config.ParseClusterConfig(c.ClusterManager.Clusters)

tls panic when cipher_suite does not match

use config specify cipher_suite, but the cert_chain is not match the cipher_suite.
mesh will start, but receive a request will trigger panic

example:
in config:
...
"tls_context": {
....
"cipher_suites": "ECDHE-RSA-AES256-GCM-SHA384",
"cert_chain": "..." //a ecdh key
}
...

HTTP/2.0 Performance Tunning

Tunning HTTP/2.0 performance