Giter Site home page Giter Site logo

tencentfemas / femas Goto Github PK

View Code? Open in Web Editor NEW
465.0 11.0 140.0 609.88 MB

JavaAgent-based Microservice Governance Platform

Home Page: https://zilongtong.notion.site/Femas-Home-4e1ad62bc48d4f23b6063c460a337c40

License: Other

Dockerfile 0.02% Makefile 0.02% Shell 0.06% Java 73.02% Smarty 0.04% Batchfile 0.03% HTML 0.05% JavaScript 0.20% TypeScript 25.33% CSS 1.23%
microservices service-discovery authentication api-management circuit-breaker router apm metrics javaagent

femas's Introduction

Femas: Tencent Cloud Open Source Microservices Governance Platform

English | 简体中文

Introduction

Femas is an open source product form of Tencent Cloud's microservice platform TSF . It focuses on the running state of microservices and provides one-stop microservice management and control capabilities such as multi-frame unified service discovery, north-south and east-west traffic management, service observability, and configuration management. In the service architecture transformation, the core issues are difficult to reuse heterogeneous frameworks, difficult to manage and control surge traffic, and time-consuming for troubleshooting and recovery.

  • data plane: Femas uses the multi-runtime architecture design to standardize and modularize the underlying core capabilities of microservices, and assemble the basic components split in the microservice field through a reasonable architecture to meet diversified microservice scenarios. , lightweight, portable, low-cost, cloud-free vendor binding,Currently, the community provides governance data planes in two proxyLess modes: SDK and business-agnostic JVM TI Agent.
  • control plane: Femas provides a unified control plane standard protocol, a set of governance protocols, and multi-language and multi-data plane distribution.

ability

image

  • Registry manage: Femas implements the management of open source registries (currently supports Consul, nacos, eureka), including cluster management and service management. Users can configure the registry cluster on the Paas platform to view the cluster status and service list.
  • Service governance: Authentication, API management, Fuse downgrade, Access current limit, Service registration discovery, Service routing, Service event.
  • Service configuration: application configuration management, configuration hot update, Femas implements a set of standard configuration API interface, configuration is divided into governance rules,application configuration, open source side supports directly issuing governance rules through Paas platform , do not rely on other third-party components.
  • Service registration discovery: Femas implements a set of standard registration discovery API interfaces, and users can directly use the SDK provided by Femas to register and discover to mainstream open source registry centers.
  • Service monitoring:
    • Metrics: Femas implements a set of standard API interfaces for business metrics. Femas uses micrometer to implement business metrics statistics by default.

    • Tracing: Femas implements a set of standard tracing API interfaces. The SDK side is responsible for formulating OpenTracing log specifications and link collection. By default, Opentelemtry is used to collect Tracing

Feature

  • Provide SDKs for Java and Go to help users realize multi-language unified management on the same Paas platform.
  • Femas standardizes and encapsulates microservice capabilities and provides unified access to the Layer layer of irrelevant protocols, which facilitates the integration of a full set of capabilities into any protocol, and realizes the unified management of multi-protocol.
  • Femas abstracts the capabilities that a microservice application may need to use in the running process into standard API components, which are convenient for expansion and compatible with other open source component ecosystems.
  • The underlying capabilities of Femas are plug-in and standardization, and users can flexibly combine and match the microservice capability matrix according to their needs.
  • Femas is not bound to any specific components, as long as the components that conform to the Femas standardized protocol can be managed on the Femas platform, it is convenient for users at all levels to learn, use and develop secondary. image
  • Sink-type non-intrusive access, zero cost for user transformation.
  • Java Agent bytecode injection
  • ServiceMesh service mesh(TODO)
  • Femas hopes to summarize Tencent's microservice product center's understanding of microservices into a platform to help users quickly build an enterprise-level microservice ecosystem.
  • Femas supports Tencent's internal ecosystem of billions of users.

Quick start

Code Structure

.
├── femas-adaptor # adaptation layer plugin for paas plateform
│   └── femas-adaptor-opensource-admin #  The PAAS platform adaptation layer is adapted to the open source platform by default. If you want to connect to other control surfaces, you can plug-in an adapter. Secondly, you can assemble the capability matrix required by the platform here
├── femas-admin # admin 
├── femas-admin-starter # starter for admin
├── femas-agent # java agent module
│   ├── femas-agent-core # javaagent bytebuddy core module
│   ├── femas-agent-example 
│   ├── femas-agent-plugin # byte-code plugin module
│   ├── femas-agent-starter # premain entrance
│   └── femas-agent-tools 
├── femas-api # the abstract layer of microservice life cycle, is convenient for users to connect with heterogeneous RPC framework
├── femas-benchmark
├── femas-common # toolkit
├── femas-config #  abstraction layer of configuration module
├── femas-config-impl # implementation layer of configuration module
│   ├── femas-config-consul # consul for femas-config
│   ├── femas-config-nacos # nacos for femas-config
│   └── femas-config-paas # pass for femas-config
├── femas-dependencies-bom # Unified management of femas dependent versions
├── femas-example # demo
│   ├── feams-example-springcloud-hoxton
│   ├── femas-example-alibaba-dubbo-consumer
│   ├── femas-example-alibaba-dubbo-provider
│   ├── femas-example-springcloud-2020-consumer
│   ├── femas-example-springcloud-2020-provider
│   ├── femas-example-springcloud-greenwich-consumer
│   ├── femas-example-springcloud-greenwich-gateway
│   ├── femas-example-springcloud-greenwich-provider
│   └── femas-example-springcloud-greenwich-zuul
├── femas-extensions # rpc layer docking framework SDK
│   ├── femas-c-dubbo # extension for dubbo
│   └── femas-extension-springcloud #  extension for springcloud
├── femas-governance # abstraction layer of governance module
├── femas-governance-impl # implementation layer of governance module
├── femas-helm
├── femas-registry # abstraction layer of registry module
├── femas-registry-impl # implementation layer of registry module
│   ├── femas-registry-consul
│   ├── femas-registry-etcd
│   ├── femas-registry-eureka
│   ├── femas-registry-k8s
│   ├── femas-registry-nacos
│   └── femas-registry-polaris
├── femas-starters # Starter dependency of user's SDK
│   ├── femas-dubbo-starters
│   └── femas-springcloud-starters
└── jacoco-aggregate

Install the server

Runtime environment dependencies:

64 bit OS, support Linux/Unix/Mac/Windows, script startup supports Linux/Unix/Mac;

64 bit JDK 1.8+;

Maven 3.2.x+;

External database Mysql (optional)

Stand-alone deployment

mvn -Dmaven.test.skip=true clean install -U

cd femas-admin-starter/target/femas-admin-starter-$version/femas-admin/bin

sh startup.sh

The console configuration mainly includes:

  • Service port
  • Database configuration (no configuration required if using embedded database)
  • nacos address configuration (if you use configuration management, you need to configure)
  • skywalking web address configuration (requires configuration to obtain link information)
  • grafana address configuration (requires configuration to obtain metrics information)

Femas default Start with the embedded database:

The embedded database only supports single-machine deployment, and does not support cluster deployment. The embedded database data disk path is ${user.home}/rocksdb/femas/data/

To use the monitoring capability, the following configuration is required:

#Configure skywalking backend address
Femas:
  trace:
    backend:
      addr: http://skywalking WEB IP:PORT
#Configure Metrics grafana address
  metrics:
    grafana:
      addr: http://IP:PORT

Cluster deployment

Cluster deployment is the same as stand-alone deployment. The only difference is that the data source must be an external data source, so that the server side of Femas supports stateless horizontal expansion.

Configuration file configuration data source

spring:
  datasource:
    url: jdbc:mysql://IP:3306/adminDb?useUnicode=true&characterEncoding=utf8&rewriteBatchedStatements=true&useSSL=false&serverTimezone=Asia/Shanghai
    username: username
    password: password
    hikari:
      driver-class-name: com.mysql.cj.jdbc.Driver

The start command is

sh startup.sh external

Visit http://localhost:8080/index to see the console page

Login user name: admin, password: 123456, the user name and password are hard-coded, and the open source side does not impose any permission restrictions.

Preparation before the service is connected to the paas platform

step 1. Configure the registry(Before this, you must have a deployed registry cluster, Femas is not bound to any registry, you can host any registry to the Femas platform)

image

Cluster addresses support IP:prot comma-separated, or domain name mode

step 2. Create namespace

image

Registry for namespace binding configuration

The service list of the femas governance center is logically isolated in the namespace dimension. The service list is pulled from the third-party registration center. The pull condition is to access FemasSDK and the name of the service tag (namespace ID written by -D below) The space is the same as the namespace selected in the list.

After completing the above two steps, you can then access femas through the SDK to manage the service to the paas platform.

Springcloud access

Execute the script under the femas parent pom:

mvn -Dmaven.test.skip=true clean install -U

Add dependency
<!-- Native dependency of registry -->
<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-consul-discovery</artifactId>
    <version>2.1.2.RELEASE</version>
</dependency>
 
<!-- femas middleware dependency -->
<dependency>
    <groupId>com.tencent.tsf</groupId>
    <artifactId>femas-extension-springcloud-greenwich-starter</artifactId>
    <version>${femas.latest.version}</version>
</dependency>

Currently Femas supports starter list directory: cd femas-starters/

The default supported version components are:

  • springcloud greenwich
  • springcloud 2020
  • springcloud gateway
  • springcloud zuul
Configuration file
Business application native configuration file, the path is: resources/bootstrap.yaml
server:
  port: 18001
spring:
  application:
    name:
      femas-springcloud-consumer
  cloud:
    consul:
      host: 127.0.0.1
      port: 8500
      discovery:
        serviceName: femas-springcloud-consumer
        instanceId: ${spring.application.name}-${server.port}-${spring.cloud.client.hostname}
        heartbeat:
          enabled: true
# nacos:
# discovery:
# server-addr: 127.0.0.1:8848
Femas component configuration file path: resources/femas.yaml (Ymal format, used to configure femas-related local configuration, such as paas address, custom registry cluster, custom governance rules, etc.)
# Configure the paas background address, if not configured, get the rules from the local configuration file
paas_server_address: http://127.0.0.1:8080
femas_namespace_id=Namespace ID
femas_prog_version=Service version

# Use the method provided by Femas to access the registration center, how to use dubbo or self-developed protocol
femas_registry_ip: 127.0.0.1 //Registry center cluster address
femas_registry_port: 8500 //Registry center port number
femas_registry_type: consul //registry type

#The following configuration is optional, which is used to configure and load the basic component type and local governance rules. If not, load the default configuration of femas.
rateLimit:
 type: femasRateLimit
authenticate:
 type: femasAuthenticate
serviceRouter:
 chain:
   - FemasDefaultRoute
loadbalancer:
 type: random
circuitBreaker:
 enable: true
 chain:
   -femasCircuitBreaker
Start service command
-javaagent:"{skywalking agent absolute path}/agent/skywalking-agent.jar"
-Dfemas_namespace_id=Namespace ID
-Dfemas_prog_version=Service version
-Dskywalking.agent.service_name=The name of the service registered on skywalking, which needs to be consistent with the name of the registration center
-Dskywalking.collector.backend_service=skywalking backend address, which can override the agent's conf configuration
  1. Reference agent probe of skywalking
  2. The service needs to specify its own namespace
  3. Specify the group to which the service belongs, and cooperate with the realization of service governance on the SDK side
  4. The service name registered to skywalking must be the same as the name registered to the registration center, otherwise the tracing link observation will not find the corresponding service.

Dubbo access

For details, see the official document below

Documentation

Participate in Contribution

  • Actively participate in the discussion of the Issue, such as answering questions, providing ideas, or reporting unsolvable errors (Issue)
  • Write and improve project documentation (Wiki)
  • Submit patch optimization code (Coding)

You will get

  • Join the list of contributors to Tencent open source projects and display them on Tencent open source official website
  • Write CONTRIBUTING.md for specific items
  • Tencent open source contributor certificate (electronic version & paper)
  • Become a special guest of offline technology conference/salon
  • Q coins and souvenirs

contact us

群微信号

Add WeChat, note [femas].

微信号

License

LICENSE.

femas's People

Contributors

actorwarning avatar bartdong avatar bennychang0 avatar bytego avatar cheese8 avatar cstongwei avatar derekyrc avatar hongdadu avatar huyuanxin avatar jolyon avatar manderls avatar mentosl avatar mroccyen avatar o-king avatar polaris-admin avatar rottenmu avatar shedfreewu avatar touniy avatar wizhuo avatar yaoow avatar yrz15102704364 avatar yuanrengu avatar zhuolili avatar zilongtong avatar zsjzero avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

femas's Issues

关于springcloud场景下遇到的获取路由规则失败的问题。

请求进来的时候,FemasRouteLoadBalancer#getInstanceResponse方法中, com.tencent.tsf.femas.common.entity.Request femasRequest = Context.getRpcInfo().getRequest()里的targetService的namespace为null,Context.getRpcInfo()里的metadata里显示 是有 namespace。因为targetService的namespace为null,导致路由时候调用RouterRuleManager#getRouteRuleGroup的时候获取到的RouteRuleGroup为null,因为RouterRuleManager#getRouteRuleGroup传入的service是有namespace的。我测试的是springcloud 网关。不确定其它场景是不是有这样的问题。

Discussion: Breaking feedback mechanisms

环境情况:
代码分支:基于develop最新dubbo2.X系列
运行环境:macos M1

情况描述:
dubbo提供方设置超时时间大于消费方等待时间,反复调用之后,一直持续报出 Invoke remote method timeout......。只有等待提供方恢复正常状态下,消费方才能继续正常进行消费。

个人想法:
消费方调用之后,在一定时间内,根据返回异常 (such as org.apache.dubbo.rpc.RpcException#TIMEOUT_EXCEPTION)是否 是否可以在一定时间在afte***Invoke 将此接口进行降级或者限流措施,之后再根据外部触发控制 或者 内部一定时间内进行阈值放开进行调用?

@zilongTong 大佬有啥意见

RocksDB not support win10

Amd cpu, win10 home 启动 femas-admin.jar 出现 Cannot determine JNI library name for ARCH='x86' OS='windows 10' name='rocksdb', 在启动脚本上面 加上 -DdbType=external使用mysql数据库可以正常启动

限流没生效

微信图片_20220608204445
我配置了请求头为femas-ut-auth_limit=test的限流规则,但是没生效,请求全打过去了
image

eureka not support register metadata

describe issue

femas via extend RegistrationCustomizer interface to achieve registering metadata,but eureka does not support Customizer.

Expected behavior

extend springcloud common registation , register femas's matedata to eureka.

How to solve

extend EurekaRegistration or EurekaInstanceConfigBean

see #com.tencent.tsf.femas.extension.springcloud.discovery.eureka.FemasEurekaRegistrationCustomizer

entrypoint.sh script bug ?

Hi:
make build image is ok , but cannot execute docke run
`
1: docker run registry.tce.com/femas/all:5d4c00f

dirname: missing operand
Try 'dirname --help' for more information.
tail: cannot open '/../logs/*.out' for reading: No such file or directory
tail: no files remaining

2:check entrypoint.sh script , This command has no output

root@0c84359d52f8:/usr/local/src/femas# find /usr/local/src/femas -regex "./femas-admin-starter-./startup.sh$

3:root@0c84359d52f8:/usr/local/src/femas# find . -name "*.sh"
./entrypoint.sh
./femas-console/femas-admin/bin/shutdown.sh
./femas-console/femas-admin/bin/startup.sh
`

need to fix the entry script ?

When the gateway relies on springcloud Alibaba 2.2.6, the startup fails.

,An attempt was made to call a method that does not exist. The attempt was made from the following location:

com.tencent.tsf.femas.springcloud.gateway.config.FemasDiscoveryAutoConfiguration$FemasNacosConfiguration.nacosServiceDiscovery(FemasDiscoveryAutoConfiguration.java:60)

The following method did not exist:

com.alibaba.cloud.nacos.discovery.NacosServiceDiscovery.<init>(Lcom/alibaba/cloud/nacos/NacosDiscoveryProperties;)V

The method's class, com.alibaba.cloud.nacos.discovery.NacosServiceDiscovery, is available from the following locations:

jar:file:/D:/gradle_cache/caches/modules-2/files-2.1/com.alibaba.cloud/spring-cloud-starter-alibaba-nacos-discovery/2.2.6.RELEASE/b5ff1915683ade1a5ab430537d2bb6e5bd2bc586/spring-cloud-starter-alibaba-nacos-discovery-2.2.6.RELEASE.jar!/com/alibaba/cloud/nacos/discovery/NacosServiceDiscovery.class

The class hierarchy was loaded from the following locations:

com.alibaba.cloud.nacos.discovery.NacosServiceDiscovery: file:/D:/gradle_cache/caches/modules-2/files-2.1/com.alibaba.cloud/spring-cloud-starter-alibaba-nacos-discovery/2.2.6.RELEASE/b5ff1915683ade1a5ab430537d2bb6e5bd2bc586/spring-cloud-starter-alibaba-nacos-discovery-2.2.6.RELEASE.jar

Action:

Correct the classpath of your application so that it contains a single, compatible version of com.alibaba.cloud.nacos.discovery.NacosServiceDiscovery

The default ubuntu image of dockerfile cannot be accessed

The default ubuntu image of dockerfile cannot be accessed. Can you update it to public address?

FROM hub.grepcode.cn:55582/basic/ubuntu:20.04

[femas]# make
docker build -f Dockerfile -t registry.tce.com/femas/all:5d4c00f .
Sending build context to Docker daemon 704.1MB
Step 1/10 : FROM hub.grepcode.cn:55582/basic/ubuntu:20.04
Get "https://hub.grepcode.cn:55582/v2/": dial tcp 221.232.175.118:55582: connect: connection refused
make: *** [build-image] Error 1

go sdk question

Could someone tell me where I can find go SDK, I will provide a register and config watcher implement for go-kratos

路线图规划

请问路线图有具体规划吗,比如2.0版本计划发布时间

Lack of unit tests

describe issue

  • config impl
  • registry impl
  • common
  • adaptor opensource
  • gov impl

Lack of unit tests

how to solve

Refer to the example (governance-impl /test)

Help users access the registration center more conveniently.

What do we want

Users only need to depend on femas-registry,Configure the address of the registration center to be accessed,femas can help users access any registry, for example: Polaris 、nacos、consul、etcd、k8s、eureka

how to achieve

  • In the case of springcloud,we need to implement springcloud common serviceRegistry interface, for example:FemasServiceRegistry,Help springcloud access any registry
  • In the case of dubbo or self-developed framework,We need to integrate the registry modules together. users only need to depend on femas-registry to enter any registry.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.