Giter Site home page Giter Site logo

service-mw2's Introduction

FSD Service

FSD Logo

FSD (Full Stack Develop) Service - Midway.js 的最佳实践

codecov GitHub Actions status codebeat badge Codacy Badge GitHub license PRs Welcome Gitpod Ready-to-Code

Sonar Cloud

项目导览

在这个项目中,你会看到以下基于 Midway 的实践案例 (上层使用 egg.js )

我们正在做以下工程实践例子,大家你遇到什么问题,或者希望追加什么功能,或者学习内部实现。

可以关注我们的仓库(点赞,分享...三连)在 issue 留言,我们会征集你的意见,带来最干货的案例。

帮你扫清学习障碍,让你用起 Midway 来更加得心应手,提升能效,找回编码的乐趣。

所有代码均已经过工程师的测试请放心使用!!!

能力栏目 名称 进度
概述
控制器(Controller)
服务和注入
请求、响应和应用
Web 中间件
启动和部署
基础能力
依赖注入
运行环境
多环境配置
参数校验和转换
生命周期
使用组件
日志
本地调试
测试
增强
代码流程控制
方法拦截器(切面)
缓存(Cache)(目前直接使用 Redis)
Database(TypeORM)
MongoDB
Swagger
一体化研发
开发一体化项目
非 Serverless 环境使用一体化
Web技术
Cookies
Session
跨域 CORS
微服务
gRPC
RabbitMQ (生产者)
RabbitMQ (消费者)
Consul
WebSocket
SocketIO
常用能力
Admin 登录
普通用户登录-账户密码
OAuth 2.0
日志监控
本地上传文件服务
鉴权中间件
接口响应统计中间件
统一错误处理
SnowFlake 雪花算法生成分布式ID
Jaeger 链路追踪
业务能力
权限管理
角色管理
管理员管理
菜单管理
日志(操作日志,记录管理用户的实际操作)

使用项目

查看 Midway docs 获取更多信息.

运行该项目需要以下环境支持

  • Mysql
  • Redis
  • Jeager

目前该项目已经集成 Docker 环境,按以下步骤可以自动配置以上依赖。docker-compose 相关文档请查看这里

  • 1.确保机器已经安装 Docker。
  • 2.在项目目录运行 docker-compose up -d
  • 3.停止服务 docker-compose down

如果没有数据库 GUI 工具,可按照下面步骤进行数据库初始化

  • $ docker ps // 查看容器ID
  • $ docker exec -it <mysql 容器 ID> /bin/bash // 进入容器
  • $ mysql shop_development < /data/database/init.sql -uroot -ppassword // 初始化数据库

Development

先将 database 目录下到 sql 文件迁移到数据库,修改默认的config配置文件

$ npm i
$ npm run dev
$ open http://localhost:7001/

Redis

  • 使用 Redis 作为用户登录凭证存取的地方
  • RTS 收集统计数据 (开发中)

Redis划分

建议使用 Redis 的时候,对所有 key 做好命名空间划分,便于管理。可把 scope 写到对照表中。

借助 jwt 插件做签名校验,管理员的 token 中会包含 id 字段。

所有 admin 相关的缓存数据都放在 admin:xxxx 下面。

  • admin:accessToken:${id} 缓存管理员 Token 信息
  • admin:userinfo:${id} 缓存管理员基本信息

数据库

所有实体表均有 deleted_at 字段(目前基础模块不使用软删除),用于软删除。

如果要关闭软删除,将deletedAt字段注释即可

进行软删除的时候,关系表的数据不做改动。

后期根据需要,用脚本定期清理软删除的数据。

以下模块未使用软删除:

  • 权限管理
  • 角色管理
  • 菜单管理
  • 管理员管理

查询注意事项

业务软删除单独写一个 BaseModel,其他实体继承该 Model 即可

  • 实体查询,继承 BaseModel 的实体会自带软删除判断,例子查看src/app/model/base.ts
  • 在做关系查询的时候,关系表需要手动加软删除判断 IS NULL,如下:
      /**
       * 根据菜单id获取数据
       * @param id 菜单id
       */
      async getAdminMenuById(id: string) {
        const row = await this.adminMenuModel
          .createQueryBuilder()
          .select()
          .leftJoinAndSelect(
            'AdminMenuModel.roles',
            'role',
            'role.deletedAt IS NULL'
          )
          .where({ id: id })
          .getOne();
        return row;
      }

Jaeger 链路追踪

Jaeger 是 OpenTracing 的一个实现,链路追踪是一个比较复杂的话题,关于 Jaeger 的具体用法请参考它的官方文档

本实现基于 ctx 机制,结合 midway 的依赖注入可实现无侵入式 spanContext 传递

  • 默认实现接口级别的采样
  • 若需小颗粒度采样,可手动管理 span
    ctx.tracerManager.startSpan('SpanName1')
    await doSomethine()
    ctx.tracerManager.finishSpan() // 别忘记关闭

接口响应统计中间件(设计)

做接口响应数据统计的出发点,有两点(即使有类似的第三方包,但还是自己实现以下):

  • 帮助排查线上接口响应问题
  • 监控系统实时状态

虽然框架本身已经有日志功能,但是很多场景下,我们可能需要看下各个接口服务的响应状态

是在正常服务,还是已经出现问题。在有监控的帮助下,可以快速帮我们定位日志排查问题。

是对应统计实时数据而言,这里我们会使用 RTS 的技术方案,会用到 RabbitMQ 和 Redis

RabbitMQ 作用在于把统计的计算异步化,从而不影响正常的业务请求处理

(消费者的逻辑代码,需要写在单独一个工程,独立部署)

大致流程如下,手绘的,工具简陋,姑且看一下。 IMG_5365 HEIC

Test 单元测试

单元测试框架采用 Mocha , 支持对 case 进行 .skip() .only() 的随意组合以及快速实现, 适合在包含复杂、耦合的业务项目场景

注意事项:

  • 单测文件不支持 nullish 链式操作符,比如
    assert(res?.body) // 不支持
    assert(res && res.body) // 可行

拓展阅读

答疑

群里会有热心的朋友,也会有新版本发布推送。钉钉扫码加入答疑群

二群

一群(已满)

License

我们的代码使用 MIT 协议,请放心使用。

service-mw2's People

Contributors

carb-x avatar codacy-badger avatar czy88840616 avatar fossabot avatar luojiyin1987 avatar tkvern avatar verncake avatar waitingsong avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

service-mw2's Issues

jwt问题

image
大佬 接口如果不想用鉴权只能在这里配置么,不能像egg那样写在router中引入中间件么

rabbitmq文件ts检查无法通过

项目未做任何修改,

运行环境

yarn

docker-compose up -d
# 修改数据库的操作做了,这里忽略不写

yarn dev

然后就报错了。

如下第四行所示

  // src/app/service/rabbitmq.ts > RabbitmqService > connect
  async connect() {
    // 创建连接
    // @ts-ignore 这一行必须添加忽略注释才能通过
    this.connection = await amqp.connect(this.rabbitmq.url);

    // 创建 channel
    this.channel = await this.connection.createChannel();

    // 绑定队列
    await this.channel.assertQueue('my-queue', {
      exclusive: true,
      autoDelete: true,
    });
    // 绑定交换机
    await this.channel.bindQueue('my-queue', 'my-exchange', 'create');

    return this.connection;
  }

错误提示

[ Midway ] start error: ⨯ Unable to compile TypeScript:
src/app/service/rabbitmq.ts(29,5): error TS2740: Type 'IAmqpConnectionManager' is missing the following properties from type 'Connection': createConfirmChannel, off, removeAllListeners, setMaxListeners, and 6 more.

我对rabbitmq使用不多,也不知道这是个什么情况,但项目刚运行就报错。使用是docker安装的环境,忽略类型检查之后就能正常通过。

看了一些type类型定义,确实接口是对不上的,但amqp里有一个 connection 的 getter ,类型是 connection 所以能对上,但是它是 undefined 。

在自己定义的函数里使用assert并用MyError抛错,没有被全局捕获到,报unhandledRejectionError

测试了一下,dto不符合入参要求,或者controller里的MyError抛错都可以被全局的error-handler捕获到,不知道为啥自定义的函数里,没有被捕获到?自定义的函数主要也是在controller里调用的,比如在helper里定义了一个公共函数,相关校验没有通过,使用类似
const record = xxx;
assert(record, new MyError('您没有相关的操作权限', 403));
return true;
预期效果:如果record为空,MyError被捕获到,并中断处理流程
实际效果:仅在后台报了unhandledRejectionError,未中断处理流程,返回给前端的是200

数据库连接失败

运行 npm run dev; 报错数据库连不上
Host 'host.docker.internal' is not allowed to connect to this MySQL server
我找到一个问题地址,不太理解
我安装了docker desk

base model中id为string的意义?

base model中id为string的意义是什么?数据库id不是基本都是int或者bugint类型?这里数据模型id也应该是number才是啊,为啥是string。求科普一下

npm v7 执行 install 时会出现依赖报错

npm v6 执行安装(本来用v7然后版本回退改用v6)

➜  service-mw2 git:(master) ✗ npm install
npm ERR! code ERESOLVE
npm ERR! ERESOLVE unable to resolve dependency tree
npm ERR!
npm ERR! While resolving: [email protected]
npm ERR! Found: @types/[email protected]
npm ERR! node_modules/@types/jest
npm ERR!   dev @types/jest@"^26.0.24" from the root project
npm ERR!
npm ERR! Could not resolve dependency:
npm ERR! peer @types/jest@"^27.0.0" from [email protected]
npm ERR! node_modules/ts-jest
npm ERR!   dev ts-jest@"^27.0.4" from the root project
npm ERR!
npm ERR! Fix the upstream dependency conflict, or retry
npm ERR! this command with --force, or --legacy-peer-deps
npm ERR! to accept an incorrect (and potentially broken) dependency resolution.
npm ERR!
npm ERR! See /home/xmo/.npm/eresolve-report.txt for a full report.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/xmo/.npm/_logs/2021-09-06T10_18_18_436Z-debug.log
➜  service-mw2 git:(master) ✗ npm install npm@6 -g

removed 70 packages, changed 98 packages, and audited 438 packages in 6s

3 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities
➜  service-mw2 git:(master) ✗ npm install
npm WARN deprecated [email protected]: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
npm WARN deprecated [email protected]: core-js@<3.3 is no longer maintained and not recommended for usage due to the number of issues. Because of the V8 engine whims, feature detection in old core-js versions could cause a slowdown up to 100x even if nothing is polyfilled. Please, upgrade your dependencies to the actual version of core-js.
npm WARN deprecated [email protected]: some dependency vulnerabilities fixed, support for node < 10 dropped, and newer ECMAScript syntax/features added
npm WARN deprecated [email protected]: https://github.com/lydell/resolve-url#deprecated
npm WARN deprecated [email protected]: Please see https://github.com/lydell/urix#deprecated

> [email protected] postinstall /home/xmo/code/demo/service-mw2/node_modules/core-js
> node -e "try{require('./postinstall')}catch(e){}"

Thank you for using core-js ( https://github.com/zloirock/core-js ) for polyfilling JavaScript standard library!

The project needs your help! Please consider supporting of core-js on Open Collective or Patreon:
> https://opencollective.com/core-js
> https://www.patreon.com/zloirock

Also, the author of core-js ( https://github.com/zloirock ) is looking for a good job -)


> @midwayjs/[email protected] postinstall /home/xmo/code/demo/service-mw2/node_modules/@midwayjs/cli
> node ./postinstall

[midway] auto install complete
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN [email protected] requires a peer of @sap/hana-client@* but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of better-sqlite3@* but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of hdb-pool@* but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of mongodb@^3.6.0 but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of mssql@* but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of oracledb@* but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of pg@* but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of pg-native@* but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of pg-query-stream@* but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of redis@* but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of sql.js@* but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of sqlite3@* but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of typeorm-aurora-data-api-driver@* but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of @swc/core@>=1.2.50 but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of @swc/wasm@>=1.2.50 but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of @types/jest@^27.0.0 but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of babel-jest@>=27.0.0 <28 but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of bufferutil@^4.0.1 but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of utf-8-validate@^5.0.2 but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of canvas@^2.5.0 but none is installed. You must install peer dependencies yourself.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: [email protected] (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

added 2263 packages from 996 contributors in 253.695s

114 packages are looking for funding
  run `npm fund` for details

此处是 npm@7 错误重现。

➜  service-mw2 git:(master) ✗ npm -v
7.22.0
➜  service-mw2 git:(master) ✗ npm install
npm ERR! code ERESOLVE
npm ERR! ERESOLVE unable to resolve dependency tree
npm ERR!
npm ERR! While resolving: [email protected]
npm ERR! Found: @types/[email protected]
npm ERR! node_modules/@types/jest
npm ERR!   dev @types/jest@"^26.0.24" from the root project
npm ERR!
npm ERR! Could not resolve dependency:
npm ERR! peer @types/jest@"^27.0.0" from [email protected]
npm ERR! node_modules/ts-jest
npm ERR!   dev ts-jest@"^27.0.4" from the root project
npm ERR!
npm ERR! Fix the upstream dependency conflict, or retry
npm ERR! this command with --force, or --legacy-peer-deps
npm ERR! to accept an incorrect (and potentially broken) dependency resolution.
npm ERR!
npm ERR! See /home/xmo/.npm/eresolve-report.txt for a full report.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/xmo/.npm/_logs/2021-09-06T10_25_46_161Z-debug.log

虽然应该是 npm 的问题,但还是给遇到相似问题的人一些提示。如果遇到类似问题,npm install -g npm@6 版本回退即可;
另外,用yarn安装是不会产生问题的。

对 jwt auth 示例的一点小小建议

const { payload } = ctx.app.jwt.decode<{ id: number }>(token);

这里建议把 decode(...) 换成 verify(...),因为可能出现小白错误理解的情况,而且 auth 的场景也确实应该使用 verify 😅。

Reference

Warning: When the token comes from an untrusted source (e.g. user input or external requests), the returned decoded payload should be treated like any other user input; please make sure to sanitize and only work with properties that are expected
警告:当令牌来自不受信任的来源(例如用户输入或外部请求)时,应将返回的解码负载视为任何其他用户输入; 请确保进行消毒并仅使用预期的属性(纯机翻 😁)

数据库存储array list和string的转换

如果给ts定义number list,装饰器定义varchar,在service是可以解析到变量有两种类型。
但是在model当前文件是解析不到的。
image
image
即使在上述支持的情况下,转换数据类型也是会被ts捕捉的,毕竟split不存在于number list中。
这种涉及序列化的场景,有什么好的解决方案吗,我看了案例中的转换,属于强转,感觉与any无异。

redis连接超时

redis设置无密码和有密码都测试了,都提示连接超时

2020-12-24 12:23:49,281 ERROR 21990 nodejs.ETIMEDOUTError: connect ETIMEDOUT
at Socket. (/home/guo/development/javascript/midwayjs/project/service-mw2/node_modules/_ioredis@4.19.4@ioredis/built/redis/index.js:308:37)
at Object.onceWrapper (events.js:421:28)
at Socket.emit (events.js:315:20)
at Socket.EventEmitter.emit (domain.js:486:12)
at Socket._onTimeout (net.js:483:8)
at listOnTimeout (internal/timers.js:554:17)
at processTimers (internal/timers.js:497:7)
errorno: "ETIMEDOUT"
code: "ETIMEDOUT"
syscall: "connect"
name: "ETIMEDOUTError"

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.