Giter Site home page Giter Site logo

gotomicro / ego Goto Github PK

View Code? Open in Web Editor NEW
937.0 23.0 157.0 2.72 MB

A simple and component-based microservice kit for go. 一个简单的基于组件的 Go 微服务套件。

Home Page: https://ego-org.com

License: MIT License

Go 98.84% Shell 0.96% Makefile 0.20%
go grpc microservice ego gin http resty gorm redis

ego's Introduction

EGO

All Contributors Go Go Report Card codecov goproxy.cn Release License: MIT Example Doc

English | 简体中文

1 Docs

The official docs for developers.

View https://ego-org.com

2 Introduction

EGO is a microservice-oriented governance framework implemented by golang, which integrates various engineering practices. Through the component-based design pattern, it is guaranteed that the business development can use various components in a unified way.

Advantages of EGO:

  • Configure drive components
  • Shield the startup details of the underlying components
  • Observation and governance of microservice components
  • Pluggable Ego-Component
  • Fail Fast principle and friendly error prompts

2.1 Improve component proficiency

For us engineers to improve component proficiency, we must first read a lot of open source component documentation and code, and then insist on using it for a long time in order to form muscle memory and improve our speed of doing business. And the time and energy invested in all of this is enormous.

To reduce this input cost and allow more developers to better use excellent open source components, EGO's approach is to standardize all open source components, encapsulate them, and unify various behaviors.

  • Unified component file name
  • Unified component configuration parameters
  • Unified component call API
  • Unify component error behavior
  • Unified component monitoring behavior

Once you have mastered one component, you can use other components by inference.

2.2 Improve the efficiency of troubleshooting

  • Unified error code
  • Component errors, slow responses, links, regular request interceptor points (both server and client will intercept)
  • Convergence error field
  • Inject key information into components: code line number, configuration name, target address, time-consuming, request data, response data
  • Debugging stage, error highlighting, formatting friendly prompts
  • In the debugging phase, the component has a built-in debug interceptor

2.3 Automatically generate duplicate code(EGO CLI)

  • Generate code, configuration, data parsing, template separation
  • Build project code independently of language
  • Use the Go1.16 feature embed, start the webUI, and generate code
  • Project: https://github.com/gotomicro/egoctl

3 Ego Component

We have Many EGO components to support your rapid development

Component Name Code Example Doc
HTTP Server Code Example Doc
gRPC Server Code Example Doc
Governance Service Code Example Doc
Job Code Example Doc
Corn job Code Example Doc
Distributed Scheduled Job Code Example Doc
HTTP Client Code Example Doc
gRPC Client Code Example Doc
gRPC Client using ETCD Code Example Doc
gRPC Client using k8s Code Example Doc
Sentinel Code Example Doc
MySQL Code Example Doc
Redis Code Example Doc
Redis Distributed lock Code Example Doc
Mongo Code Example Doc
Kafka Code Example Doc
ETCD Code Example Doc
K8S Code Example Doc
Oauth2 Code Example

4 Definition

4.1 Framework Layer

EGO framework has three layers:

  • The core layer provides configuration, logging, monitoring and links, and is the cornerstone of other components.
  • The component layer provides various components in the client, server and task.
  • The glue layer controls the life cycle of various components, error handling.

4.2 Architecture

4.3 Life cycle

4.4 Component Layer

We consider everything to be a component and divide the component into four parts:

  • Container handles component type, configuration and component startup
  • Config configure parameters
  • Component The calling method of the component
  • Options the options of configuration and component

5 Version Requirements

  • Below v0.8.2, Go version needs to be greater than Go1.13.
  • After v0.8.3, Go version needs to be greater than Go1.16.
  • After v1.0.0, Go version needs to be greater than Go1.17.

6 Download Tool

bash <(curl -L https://raw.githubusercontent.com/gotomicro/egoctl/main/getlatest.sh)

Through the above script, you can download the protoc tools, EGO protoc plugin and egoctl.

  • /usr/local/bin/egoctl EGO Cli
  • /usr/local/bin/protoc Generate Pb tool
  • /usr/local/bin/protoc-gen-go Generate Pb tool
  • /usr/local/bin/protoc-gen-go-grpc Generate gRPC tool
  • /usr/local/bin/protoc-gen-go-errors Generate error code tool
  • /usr/local/bin/protoc-gen-openapiv2 Generate HTTP tool
  • /usr/local/bin/protoc-gen-go-http Generate HTTP tool

7 Features

  • Configuration driver The startup method of all components is component name.Load("configuration name").Build(), which can create a component instance. For example, http server below, egin is the component name, server.http is the configuration name

    egin.Load("server.http").Build()
  • friendly debug By enabling the debug configuration and export EGO_DEBUG=true on the command line,

    We can see the line number, configuration name, request address, time-consuming, request data, and response data in the request of all components in the test environment

And using Goland, you can directly click to the corresponding code path through the line number (gRPC, HTTP client support line number)

  • Tracing Use the opentelemetry protocol to automatically add Tracing to the log

    • gRPC Tracing

      image

      • Client Tracing information

      image

    • HTTP Tracing

    • Ali arms Tracing

  • Unified error message

  • Unified monitoring information

8 Quick Start

8.1 HelloWorld

Configuration

[server.http]
    port = 9001
    host = "0.0.0.0"

Code

package main
import (
   "github.com/gin-gonic/gin"
   "github.com/gotomicro/ego"
   "github.com/gotomicro/ego/core/elog"
   "github.com/gotomicro/ego/server"
   "github.com/gotomicro/ego/server/egin"
)
//  export EGO_DEBUG=true && go run main.go --config=config.toml
func main() {
   if err := ego.New().Serve(func() *egin.Component {
      server := egin.Load("server.http").Build()
      server.GET("/hello", func(ctx *gin.Context) {
         ctx.JSON(200, "Hello EGO")
         return
      })
      return server
   }()).Run(); err != nil {
      elog.Panic("startup", elog.FieldErr(err))
   }
}

8.2 Using the command line to run

export EGO_DEBUG=true # The default log is output to the logs directory, and after dev mode is enabled, the log is output to the terminal
go run main.go --config=config.toml

8.3 Result

图片

At this time, we can send a command and get the following result

➜  helloworld git:(master) ✗ curl http://127.0.0.1:9001/hello
"Hello Ego"%  

8.4 More friendly package compilation

Use build in the scripts folder, you can see the elegant version prompt.

图片

9 Changelog

Releases

10 Join us

To join our Wechat comminication group, add the ego keyword in the verification information.

wechat-qrcode

Contributors

Thanks for these wonderful people:


askuy

Wei Zheng

shaoyuan

Panda

刘文哲

zhangjunjun

devincd

Ming Deng

Angelia

Wbofeng

clannadxr

Link Duan

Costa

MEX7

LincolnZhou

optimistic9527

soeluc

Thank JetBrains for Open Source licenses support

JetBrains

ego's People

Contributors

allcontributors[bot] avatar aschenmaker avatar askuy avatar clannadxr avatar cubxxw avatar cute-angelia avatar devincd avatar flutterwang avatar flycash avatar hkesd avatar jarily avatar kl7sn avatar laojianzi avatar lincolnzhou avatar link-duan avatar luyang950 avatar neoyeelf avatar optimistic9527 avatar qingbozhang avatar sevennt avatar shaoyuan1943 avatar shuaxindiary avatar soeluc avatar testwill avatar ting817 avatar wujunze avatar zcong1993 avatar zhjunjun avatar zxmfke avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ego's Issues

希望ego框架能够新增grpc-gateway功能

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
在用grpc时我用到了grpc-gateway,我目前自己写了一个grpc-gatway的ego-componet因为对ego框架部分功能不太熟,实现得不够优雅,希望ego框架官方能够增加此模块
Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

vscode 调试

ego团队,你好!
ego框架在vscode中运行,然后怎么进行vscode调试。

ego的系统日志通过配置修改时使用赋值方式,导致ego结构体中的logger没有使用到最新的配置

if econf.Get(e.opts.configPrefix+"logger.ego") != nil {
		elog.EgoLogger = elog.Load(e.opts.configPrefix + "logger.ego").Build(elog.WithDefaultFileName(elog.EgoLoggerName))
		elog.EgoLogger.Info("reinit ego logger", elog.FieldComponent(elog.PackageName))
		e.opts.afterStopClean = append(e.opts.afterStopClean, elog.EgoLogger.Flush)
	}

ego的系统日志通过配置修改时使用赋值方式,导致ego结构体中的logger没有使用到最新的配置
从而ego结构体中的所有通过e.logger输出的日志跟本身的框架配置不一致,存在采集漏掉和清理的问题

新版ego框架econf组件存在数据竞争问题

BUG描述
在ego初始化后注册econf.OnChange回调会出现数据竞争
另外在ego初始化后操作组件也可能出现此情况(未验证)

BUG复现方法
1.代码

package main

import (
	"github.com/gotomicro/ego"
	"github.com/gotomicro/ego/core/econf"
)

type Mysql struct {
	Dsn string
}

func main() {
	ego.New()

	econf.OnChange(func(c *econf.Configuration) {})
}

2.执行命令go run -race ./cmd/ego/main.go --config=./configs/main/config.toml
3.报错信息

==================
WARNING: DATA RACE
Write at 0x00c0001228a0 by main goroutine:
  github.com/gotomicro/ego/core/econf.(*Configuration).OnChange()
      /go/pkg/mod/github.com/gotomicro/[email protected]/core/econf/conf.go:72 +0xf3
  github.com/gotomicro/ego/core/econf.OnChange()
      /go/pkg/mod/github.com/gotomicro/[email protected]/core/econf/api.go:36 +0x32
  main.main()
      /workspaces/coreadmin/cmd/ego/main.go:15 +0x31

Previous read at 0x00c0001228a0 by goroutine 29:
  github.com/gotomicro/ego/core/econf.(*Configuration).LoadFromDataSource.func1()
      /go/pkg/mod/github.com/gotomicro/[email protected]/core/econf/conf.go:92 +0x64

Goroutine 29 (running) created at:
  github.com/gotomicro/ego/core/econf.(*Configuration).LoadFromDataSource()
      /go/pkg/mod/github.com/gotomicro/[email protected]/core/econf/conf.go:90 +0x316
  github.com/gotomicro/ego/core/econf.LoadFromDataSource()
      /go/pkg/mod/github.com/gotomicro/[email protected]/core/econf/api.go:48 +0x4a6
  github.com/gotomicro/ego.loadConfig()
      /go/pkg/mod/github.com/gotomicro/[email protected]/ego_function.go:171 +0x450
  github.com/gotomicro/ego.runSerialFuncReturnError()
      /go/pkg/mod/github.com/gotomicro/[email protected]/ego_function.go:262 +0x16bd
  github.com/gotomicro/ego.New()
      /go/pkg/mod/github.com/gotomicro/[email protected]/ego.go:124 +0x1623
  main.main()
      /workspaces/coreadmin/cmd/ego/main.go:13 +0x30
==================
Found 1 data race(s)
exit status 66

3.ego中出现异常的代码行
image

希望能够提供细粒度的监控,包括服务提供的每个接口的pct99以及依赖下游的接口(包括但不限于db与下游微服务)的pct99

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

panic in handler but retrun code 200

Handler 中 panic,egin 却依然返回 200 code。

func Import(ctx *gin.Context) {
	for _, s := range something {
    	    Panic()
	}
        // 此处仍然能继续返回
	ctx.JSON(200, map[string]interface{}{"result": "ok"})
}

func  Panic(){
  panic("don't panic")
}

about config

  1. Why not use viper lib to load config data source? Viper support load config from file / env /config center, watch change too
  2. Support multi data source?
  3. Support load config center data to update local config file when config center is formal, to prevent config center crash

ego user list

Welcome to use ego To know who is using it, pls append your org info as follow,
Organization:ego (Required)
Location: Shanghai, China(Required)
Contact: Email or Official website (Optional)
Purpose:use as our basic framework (Required)

Thanks again for your participation!

欢迎使用 ego,首先感谢你的使用,其次您可以参考下面的样例来提供您的信息以收集下使用场景:

  • 组织:ego(Required)
  • 地点:**上海(Required)
  • 联系方式:邮箱或官方网站(Optional)
  • 场景:作为服务端基础框架使用(Required)

再次感谢你的参与!!!

[Feature]期望支持多配置文件

Is your feature request related to a problem? Please describe.
在接入ego过程中,遇到2类场景, 在接入过程里,比较容易出错。

  1. 多配置文件,并且都比较大,多配置文件合一的过程,容易出错,不利于执行,且可读性不好。期望对问题1,支持多配置文件。
  2. 部分项目配置文件的格式不标准,在接入ego过程中,想用到component,但是component的配置,由旧配置文件读好了,在内存中。期望问题2,对主流component(eredis,egorm,egin),支持egin.LoadFromKV(map[string]interface{}),来做到指定配置值赋予。

Describe the solution you'd like
对问题1的期望解决方式:
对econf支持载入多数据源,并且允许不同文件的key重复。

预期样式:

var redisfilepath = "/xxx/xx/redis.yaml"
var db filepath = "/xx/xx/db.yaml"
var domainfilepath = "/xxx/xx/domain.yaml"
rediscfg := econf.NewFromFile(redisfilepath)
dbcfg :=econf.NewfromFile(dbfilepath)
domaincfg :=econf.NewFromFile(domainfilepath)

部分核心component优先支持, eredis,egorm,支持指定econf.Configuration来读取键值
预期样式:
eredis.LoadFromCfg(rediscfg, "userredis.scene1")

对问题2的期望解决方式:
部分核心component支持该方法,来内存赋值: egin.LoadFromKV(map[string]interface{})

Describe alternatives you've considered
对问题1的econf,写法使用多变量来承接,而不是杂糅进econf里,写成econf.Load(file1,file2,file3),主要考虑到econf中的维护关系,都是围绕单配置源,watch相关操作,在集成多文件时,容易出现不兼容问题。其次,不同的中间件,在载入配置信息时,应该显式地知道读取哪个配置文件(egin.LoadFrom(ginconfig)) ,而不是由框架静默选择。

对问题2的入参设置为map,是基于部分component的配置key,未提供可配置字段的Set方法,因为Container结构体里面的config是小写的,很多优化配置的参数,在未正确接入econf (解决问题1前)时,无法载入。

cron task 设计问题

Is your feature request related to a problem? Please describe.
根据当前的代码,目前ecron设计上,每个cron task配置一个config后,都会启动一个cron来运行这个job,这样代价是不是有点高?

Describe the solution you'd like
大多数业务场景下,任务会根据业务场景归类,希望这里的ecron是一个配置项目支持的是一组task的管理

go 1.16 embed 特性加载配置

Is your feature request related to a problem? Please describe.

编译后可执行程序,需要配置文件,往往要把配置文件和可执行程序一起放在线上服务器,配置文件内容尤其是sql连接信息,非常敏感

Describe the solution you'd like

1.16之后,有了 embed 特性,可以在编译服务过程中过程中,就把配置打包进去, 线上真正只需要一个可执行程序

Describe alternatives you've considered

1.16之前,我们是将配置写在一个go文件里面,不是很友好

Additional context

期待更好

build constraints exclude all Go files

run demo with quickstart
https://ego.gocn.vip/frame/quickstart/quickstart.html

go mod is
github.com/gotomicro/[email protected]

when i want to start app
go run main.go --config=config.toml

get error

package command-line-arguments
        imports github.com/gotomicro/ego
        imports github.com/gotomicro/ego/core/util/xcolor: build constraints exclude all Go files in E:\go\pkg\mod\github.com\gotomicro\[email protected]\core\util\xcolor

switch to
github.com/gotomicro/[email protected]
this version is ok

egin 支持https并且支持双向认证

背景:
在安全场景下,后端服务提供给第三方,或者跟网关的情况下,虽然在网关后面但是需要与第三方或者官网进行双向认证

当前情况:
egin不支持https并且支持双向认证

Log format

Describe the bug
the value of cost key in grpc api access log should be float, but is string

To Reproduce
Steps to reproduce the behavior:

  1. launch a grpc server
  2. send grpc request
  3. check the server.grpc log

组件依赖 econf 加载配置,能不能传入自定义配置给组件呢?

例如现在的新建 egin component 代码为:

server := egin.Load("server.http").Build()

而我现在使用的一个旧项目已经在线上运行,更改 config 结构不方便,能否通过下面这种方式传入自定义配置项呢?

server := egin.Load("", egin.Config{Port: 9001, Host: "0.0.0.0"}).Build()

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.