cloudmindsrobot/dagger

Dagger 是一个基于 Loki 的日志查询和管理系统，它是由达闼科技（ CloudMinds ）云团队的`大禹基础设施平台`派生出来的一个项目。Dagger 运行在 Loki 前端，具备日志查询、搜索，保存和下载等特性，适用于云原生场景下的容器日志管理场景。

README of English

Dagger - A Log management system based on loki

Dagger 是一个基于 Loki 的日志查询和管理系统，它是由达闼科技（ CloudMinds ）云团队的大禹基础设施平台派生出来的一个项目。Dagger 运行在 Loki 前端，具备日志查询、搜索，保存和下载等特性，适用于云原生场景下的容器日志管理场景。

前端基于 vue.js 开发，使用 vuetify ui 框架进行设计，采用 axios 异步框架进行前后端交互。

后端基于 go gin 框架开发，gorm 作为数据框架，sqlite 作为数据存储端，采用 go websocket 桥接的方式进行实时数据处理。

本地快速部署请参见Quick Start

ScreenShots

Features

• 日志查询，简单的标签查询，无需复杂的 LogQL 语法
• 支持查询标签保存和快速查询，同时支持查询历史
• 支持日志查询结果快照保存，防止 loki 日志轮转后找不到记录
• 支持日志实时播放和下载功能

RoadMap

• 日志告警

  • 支持界面添加日志 rules 编辑、保存和推送
  • 兼容 AlertManager API，接受 Loki Ruler 的告警事件推送
  • 日志事件聚合，分析和告警
  • 支持邮件、阿里云 SMS、阿里云 Voice 告警

• 度量

  • 支持 LogQL 的方式在前端添加自定义日志度量

Usage

完整演示视频（youtube、bilibili）

Deployment

• Quick Start
  • 日志客户端 Fluentd 配置
  • 日志客户端 promtail 配置
  • Loki 建议

• 分布式部署

1.0 upgrade to 2.0

• 1.0 upgrade to 2.0

Release Notes

• 版本发布历史

Q&A

Support

扫描二维码关注公众号回复【入群】加入微信讨论组

dagger's Issues

登陆功能问题

正常登陆后，想在登出，没有作用，还是一样能够查询看到相应的界面

日志量分label统计及告警

loki有按标签进行日志量（存储大小）统计接口，你们这边有实现吗？
另外基于日志使用量告警有没有什么方案？

为什么在查询中额外加入k8s_pod_name标签

Is this a BUG REPORT or FEATURE REQUEST?:

Uncomment only one, leave it on its own line:

/kind bug

/kind feature

What happened:
在查询时额外添加了k8s_pod_name标签（而且还无法取消），但我的loki中并没有这个标签，导致所有查询都为空。
What you expected to happen:
删除这个自主添加的标签，或者可以取消掉。
How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:
请问增加这个的意义是什么？

dagger/backend/controllers/query.go

Line 548 in 7a63f5e

queryExprArray = append(queryExprArray, utils.GetPodExpr(pod))

占个楼！

支持马总大佬！

希望在Dagger内增加日志输出的自定义行数

feature支持

根据群友反馈，希望在Dagger内增加日志输出的自定义行数

grafana7.3版本之后的版本也逐渐意识到这个问题，希望dagger跟进

请问这个项目还在维护吗？

Is this a BUG REPORT or FEATURE REQUEST?:

Uncomment only one, leave it on its own line:

/kind bug
/kind feature

What happened:

What you expected to happen:

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:
请问这个项目还在维护吗？ 看好久没更新了，包括 #17 看是之前说要有更新修复，貌似没有下文了，感觉挺好的一个项目，希望能继续下去，加油！

Dagger有必要支持多个Loki后端吗？

grafana通过配置多个loki数据源来查询日志，但是由于不同的loki数据源中日志label并不一样，这也导致在grafana上查询日志还是具备一定的门槛。

dagger的目的是尽量简化日志查询过程中的条件选择，引入多个loki数据源可能会打破这样的初衷。

我这里另外一个解决方案，看各位有啥好的想法：

方案一、一套Loki集群收集多个集群日志，然后跟dagger对接

优势：

1. dagger无变动，日志由loki集中管理，后期对于日志告警也可以集中管理
2. 只改日志采集端label的逻辑

劣势：

1. 日志量大的话，loki架构需要进行分布式集群设计。

方案二、dagger支持多个loki数据源

优势：

1. 能集成多套环境，方便用户使用

劣势：

1. 日志告警无法统一告警，也需要针对loki实例单独关联
2. dagger些许的开发（不属于我们主要的feature）

有其他的想法，欢迎在下方留言讨论

OpenLdap支持

feature支持

根据群友反馈，希望在Dagger内增加openldap账号管理支持

登录后报 Error 获取标签数据失败 等2个错误

Is this a BUG REPORT or FEATURE REQUEST?:

Uncomment only one, leave it on its own line:

kind bug
/kind feature

What happened: 登录后报红色背景的通知： Error 获取标签数据失败 和 Error: 获取数据失败 两个错误，同时也有蓝绿背景的 Success: 登录成功

What you expected to happen: 展现出 logs

How to reproduce it (as minimally and precisely as possible):
正常安装，pods 正常，pvc 也正常 bound，登录后出现异常

Anything else we need to know?:

使用 DAGGER 各功能异常

Is this a BUG REPORT or FEATURE REQUEST?:
yes

Uncomment only one, leave it on its own line:

/kind bug

Enviroment

• loki 2.0.0
• prometheus-operator v0.6.0
• kubernetes 1.17.9

What happened:

1. 在进行查询时，默认是倒序，选择升序的时候报错（部分标签会，可能是日志量过大？）；

2020-11-04 18:33:34 | 2020-11-04T10:33:34.323Z	[INFO]	controllers/query.go:102	query expr: {app="xxxxappxxxx",namespace="dev1"} \|~ "200"
2020-11-04 18:33:34 | 2020-11-04T10:33:34.323Z	[INFO]	utils/lokiapi.go:16	loki api query url: http://loki-headless.monitoring:3100/loki/api/v1/query_range?query=%7Bapp%3D%22xxxxappxxxx%22%2Cnamespace%3D%22dev1%22%7D+%7C~+%22200%22&start=1604485500000000000&end=1604485800000000000&limit=500&direction=forward
2020-11-04 18:33:34 | 2020-11-04T10:33:34Z [INFO] "GET /api/v1/loki/query_range?namespace=dev1&app=xxxxxapppxxx&start=1604485500000000000&end=1604485800000000000&level=&limit=500&size=20&middleStart=&middleEnd=&pod=&all=true&dsc=false&filters[]=200 HTTP/1.0 500 141.573176ms "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4310.0 Safari/537.36" "
2020-11-04 18:33:34 |  
2020-11-04 18:33:34 |  
2020-11-04 18:33:34 | 2020/11/04 10:33:34 [Recovery] 2020/11/04 - 10:33:34 panic recovered:
2020-11-04 18:33:34 | runtime error: index out of range [-289]
2020-11-04 18:33:34 | /usr/local/go/src/runtime/panic.go:75 (0x438932)
2020-11-04 18:33:34 | /go/src/dagger/backend/gin/controllers/query.go:178 (0xa5b8c2)
2020-11-04 18:33:34 | /go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:161 (0xa3910a)
2020-11-04 18:33:34 | /go/src/dagger/backend/gin/middlewares/middlewares.go:72 (0xa64a71)
2020-11-04 18:33:34 | /go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:161 (0xa3910a)
2020-11-04 18:33:34 | /go/pkg/mod/github.com/gin-gonic/[email protected]/recovery.go:83 (0xa4cc03)
2020-11-04 18:33:34 | /go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:161 (0xa3910a)
2020-11-04 18:33:34 | /go/pkg/mod/github.com/gin-gonic/[email protected]/logger.go:241 (0xa4bd30)
2020-11-04 18:33:34 | /go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:161 (0xa3910a)
2020-11-04 18:33:34 | /go/pkg/mod/github.com/gin-contrib/[email protected]/sessions.go:52 (0xa4f2fd)
2020-11-04 18:33:34 | /go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:161 (0xa3910a)
2020-11-04 18:33:34 | /go/pkg/mod/github.com/gin-gonic/[email protected]/gin.go:409 (0xa4304c)
2020-11-04 18:33:34 | /go/pkg/mod/github.com/gin-gonic/[email protected]/gin.go:367 (0xa4274d)
2020-11-04 18:33:34 | /usr/local/go/src/net/http/server.go:2802 (0x81b003)
2020-11-04 18:33:34 | /usr/local/go/src/net/http/server.go:1890 (0x8168a4)
2020-11-04 18:33:34 | /usr/local/go/src/runtime/asm_amd64.s:1357 (0x467610)
2020-11-04 18:33:34

2. 选择级别和对应指定 Pod 无效，返回 200 状态码；
   backend 无日志，response 返回 200

3. 点击下载日志仍然失败

2020-11-04 18:36:59 | 2020-11-04T10:36:59.484Z	[INFO]	controllers/query.go:303	download expr: {app="xxxxapppxaxx",namespace="dev1"}
2020-11-04 18:36:59 | 2020-11-04T10:36:59.484Z	[INFO]	utils/lokiapi.go:16	loki api query url: http://loki-headless.monitoring:3100/loki/api/v1/query_range?query=%7Bapp%3D%22xxxxapppxaxx%22%2Cnamespace%3D%22dev1%22%7D&start=1604485500000000000&end=1604485800000000000&limit=5000&direction=backward
2020-11-04 18:36:59 | 2020-11-04T10:36:59.639Z	[INFO]	utils/lokiapi.go:16	loki api query url: http://loki-headless.monitoring:3100/loki/api/v1/query_range?query=%7Bapp%3D%22xxxxapppxaxx%22%2Cnamespace%3D%22dev1%22%7D&start=1604485500000000000&end=1604485092838469523&limit=5000&direction=backward
2020-11-04 18:37:00 |  
2020-11-04 18:37:00 | 2020-11-04T10:37:00Z [INFO] "GET /api/v1/loki/export?namespace=dev1&app=xxxxapppxaxx&start=1604485500000000000&end=1604485800000000000&level=&pod=&dsc=true HTTP/1.0 500 748.953057ms "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4310.0 Safari/537.36" "
2020-11-04 18:37:00 |  
2020-11-04 18:37:00 | 2020/11/04 10:37:00 [Recovery] 2020/11/04 - 10:37:00 panic recovered:
2020-11-04 18:37:00 | interface conversion: interface {} is nil, not string
2020-11-04 18:37:00 | /usr/local/go/src/runtime/iface.go:255 (0x4162f5)
2020-11-04 18:37:00 | /go/src/dagger/backend/gin/controllers/query.go:316 (0xa5db92)
2020-11-04 18:37:00 | /go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:161 (0xa3910a)
2020-11-04 18:37:00 | /go/src/dagger/backend/gin/middlewares/middlewares.go:72 (0xa64a71)
2020-11-04 18:37:00 | /go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:161 (0xa3910a)
2020-11-04 18:37:00 | /go/pkg/mod/github.com/gin-gonic/[email protected]/recovery.go:83 (0xa4cc03)
2020-11-04 18:37:00 | /go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:161 (0xa3910a)
2020-11-04 18:37:00 | /go/pkg/mod/github.com/gin-gonic/[email protected]/logger.go:241 (0xa4bd30)
2020-11-04 18:37:00 | /go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:161 (0xa3910a)
2020-11-04 18:37:00 | /go/pkg/mod/github.com/gin-contrib/[email protected]/sessions.go:52 (0xa4f2fd)
2020-11-04 18:37:00 | /go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:161 (0xa3910a)
2020-11-04 18:37:00 | /go/pkg/mod/github.com/gin-gonic/[email protected]/gin.go:409 (0xa4304c)
2020-11-04 18:37:00 | /go/pkg/mod/github.com/gin-gonic/[email protected]/gin.go:367 (0xa4274d)
2020-11-04 18:37:00 | /usr/local/go/src/net/http/server.go:2802 (0x81b003)
2020-11-04 18:37:00 | /usr/local/go/src/net/http/server.go:1890 (0x8168a4)
2020-11-04 18:37:00 | /usr/local/go/src/runtime/asm_amd64.s:1357 (0x467610)
2020-11-04 18:37:00

截图

标签过滤有问题

我选择了istio-system这个namespace，但是并没有过滤出istio相关的app，还是全量的app，在grafana上就没有这个问题

dagger docker-compose 安装遇到的一些问题

①dagger-mysql 容器无法启动
解决办法: 在docker服务器上，useradd -u 999 mysql， chonw -R mysql /usr/local/src/dagger/dagger/production/config
②dagger-backend 容器无法与loki服务交互
解决办法：修改 docker-compose.yaml dagger-backend 容器内挂载的 ../backend/dagger.ini 修改为./config/dagger.ini

用了不到10分钟，后端的服务出问题了

详细日志：
2021-01-13T04:12:09Z [INFO] "GET /api/v1/loki/snapshot?page=1&page_size=10 HTTP/1.0 200 952.044µs "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36" "
2021-01-13T04:12:12Z [INFO] "GET /api/v1/loki/history?page=1&page_size=10 HTTP/1.0 200 863.445µs "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36" "
2021-01-13T04:12:14Z [INFO] "GET /api/v1/loki/labels HTTP/1.0 200 41.342206ms "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36" "
2021-01-13T04:12:14Z [INFO] "GET /api/v1/loki/labels?start=1610510833000000000&end=1610511133000000000 HTTP/1.0 200 42.362902ms "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36" "
2021-01-13T04:12:24Z [INFO] "GET /swagger/index.html HTTP/1.0 200 163.233µs "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36" "
2021-01-13T04:12:24Z [INFO] "GET /swagger/doc.json HTTP/1.0 200 1.197638ms "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36" "
2021-01-13T04:13:05.449Z [ERROR] controllers/query.go:555 send message to viewer error, write tcp 172.27.0.2:8000->172.27.0.3:60410: write: broken pipe
dagger/backend/controllers.LokiTail
/go/src/dagger/backend/controllers/query.go:555
github.com/gin-gonic/gin.(*Context).Next
/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:161
github.com/gin-gonic/gin.RecoveryWithWriter.func1
/go/pkg/mod/github.com/gin-gonic/[email protected]/recovery.go:83
github.com/gin-gonic/gin.(*Context).Next
/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:161
github.com/gin-gonic/gin.LoggerWithConfig.func1
/go/pkg/mod/github.com/gin-gonic/[email protected]/logger.go:241
github.com/gin-gonic/gin.(*Context).Next
/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:161
github.com/gin-contrib/sessions.Sessions.func1
/go/pkg/mod/github.com/gin-contrib/[email protected]/sessions.go:52
github.com/gin-gonic/gin.(*Context).Next
/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:161
github.com/gin-gonic/gin.(*Engine).handleHTTPRequest
/go/pkg/mod/github.com/gin-gonic/[email protected]/gin.go:409
github.com/gin-gonic/gin.(*Engine).ServeHTTP
/go/pkg/mod/github.com/gin-gonic/[email protected]/gin.go:367
net/http.serverHandler.ServeHTTP
/usr/local/go/src/net/http/server.go:2802
net/http.(*conn).serve
/usr/local/go/src/net/http/server.go:1890
2021-01-13T04:13:05Z [INFO] "GET /ws/tail?namespace_service=a-prd-whzb/omni-adapter&start=1610511022000000000&level=&pod=&filters= HTTP/1.1 200 2m41.627422872s "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36" "
panic: send on closed channel

goroutine 805 [running]:
dagger/backend/utils.WebSocketServerHandler(0xc0001f3ce0, 0xc0003c0c00, 0xc0003c0c60)
/go/src/dagger/backend/utils/lokiwebsocket.go:97 +0x16c
created by dagger/backend/controllers.LokiTail
/go/src/dagger/backend/controllers/query.go:547 +0xc5b