vran-dev/databasir

数据库文档管理中心

All good things come to those who wait.

databasir's Issues

fix: diff result table show ADDED but overview show NONE

feature: description to be optional when create/update project or group

feature: suppor get table trigger from postgresql

功能建议

1. 我发现版本历史不能命名，感觉可以保存一些特定的版本会比较好，目前好像是同步一次一个版本（如果有变化的话）
2. 对比版本差异的时候能导出两个版本之间的增量sql语句

feature：数据库扩展列表采用卡片式展示

后续是否考虑集成ldap

bug: sync project failed

java.lang.IllegalStateException: Duplicate key QRTZ_CRON_TRIGGERS_ibfk_1 (attempted merging values ForeignKeyMeta(keySeq=1, pkName=PRIMARY, pkTableName=qrtz_triggers, pkColumnName=SCHED_NAME, fkName=QRTZ_CRON_TRIGGERS_ibfk_1, fkTableName=qrtz_cron_triggers, fkColumnName=SCHED_NAME, updateRule=CASCADE, deleteRule=CASCADE) and ForeignKeyMeta(keySeq=2, pkName=PRIMARY, pkTableName=qrtz_triggers, pkColumnName=TRIGGER_NAME, fkName=QRTZ_CRON_TRIGGERS_ibfk_1, fkTableName=qrtz_cron_triggers, fkColumnName=TRIGGER_NAME, updateRule=CASCADE, deleteRule=CASCADE))
        at java.base/java.util.stream.Collectors.duplicateKeyException(Collectors.java:133) ~[na:na]
        at java.base/java.util.stream.Collectors.lambda$uniqKeysMapAccumulator$1(Collectors.java:180) ~[na:na]
        at java.base/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169) ~[na:na]
        at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655) ~[na:na]
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[na:na]
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[na:na]
        at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[na:na]
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[na:na]
        at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[na:na]
        at com.databasir.core.diff.processor.DiffProcessor.toMap(DiffProcessor.java:44) ~[plugin-1.0.0-plain.jar!/:na]
        at com.databasir.core.diff.processor.DiffProcessor.diffTableField(DiffProcessor.java:22) ~[plugin-1.0.0-plain.jar!/:na]
        at com.databasir.core.diff.processor.ForeignKeyDiffProcessor.process(ForeignKeyDiffProcessor.java:12) ~[plugin-1.0.0-plain.jar!/:na]
        at com.databasir.core.diff.processor.TableDiffProcessor.diffTableField(TableDiffProcessor.java:98) ~[plugin-1.0.0-plain.jar!/:na]
        at com.databasir.core.diff.processor.TableDiffProcessor.lambda$process$0(TableDiffProcessor.java:34) ~[plugin-1.0.0-plain.jar!/:na]
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195) ~[na:na]
        at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655) ~[na:na]

bug: copy insert sql failed

我用测试帐号进不去

我刚试了一下用你们给的测试帐号进不去，另外 docker run --name databasir -e DATABASIR_DB_URL=172.17.0.4:3306 -e DATABASIR_DB_USERNAME=root -e DATABASIR_DB_PASSWORD=123456 vrantt/databasir:latest -p 8888:8080 我用这命令运行起来后，http://localhost:8080 也访问不了，启动是正常的。

oracle数据库不能生成文档

oracle数据库配置成功后，生成文档报错

Exception in thread "projectSyncTaskThreadPoolTaskExecutor-19" com.databasir.common.DatabasirException: 获取数据库信息失败 at com.databasir.core.domain.DomainErrors.exception(DomainErrors.java:55) at java.base/java.util.Optional.orElseThrow(Optional.java:403) at com.databasir.core.domain.document.service.DocumentService.retrieveDatabaseMeta(DocumentService.java:145) at com.databasir.core.domain.document.service.DocumentService.syncByProjectId(DocumentService.java:94) at com.databasir.core.domain.document.service.DocumentService$$FastClassBySpringCGLIB$$122722d1.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123) at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:388) at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at com.databasir.core.domain.document.service.DocumentService$$EnhancerBySpringCGLIB$$5a47723c.syncByProjectId(<generated>) at com.databasir.job.ProjectSyncTaskScheduler.sync(ProjectSyncTaskScheduler.java:53) at com.databasir.job.ProjectSyncTaskScheduler.lambda$startSyncTask$0(ProjectSyncTaskScheduler.java:45) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:833)

300多张表的库，同步很容易超时

orace数据库不能生成文档

数据库链接成功，生成文档时报错，

Exception in thread "projectSyncTaskThreadPoolTaskExecutor-19" com.databasir.common.DatabasirException: 获取数据库信息失败
	at com.databasir.core.domain.DomainErrors.exception(DomainErrors.java:55)
	at java.base/java.util.Optional.orElseThrow(Optional.java:403)
	at com.databasir.core.domain.document.service.DocumentService.retrieveDatabaseMeta(DocumentService.java:145)
	at com.databasir.core.domain.document.service.DocumentService.syncByProjectId(DocumentService.java:94)
	at com.databasir.core.domain.document.service.DocumentService$$FastClassBySpringCGLIB$$122722d1.invoke(<generated>)
	at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
	at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:388)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698)
	at com.databasir.core.domain.document.service.DocumentService$$EnhancerBySpringCGLIB$$5a47723c.syncByProjectId(<generated>)
	at com.databasir.job.ProjectSyncTaskScheduler.sync(ProjectSyncTaskScheduler.java:53)
	at com.databasir.job.ProjectSyncTaskScheduler.lambda$startSyncTask$0(ProjectSyncTaskScheduler.java:45)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)

Feature: support auto get driver class name

feature：支持从本地上传数据库 JDBC 驱动

• what

支持从本地上传数据库 JDBC 驱动

• why

有部分数据库的驱动版本太老，用户难以在 maven 仓库找到对应的驱动 jar 下载地址，这个时候就可以从项目的 classpath 路径中上传该驱动到 databasir，从而实现文档的加载

• how

1. 数据库扩展页面新增选项驱动加载选项：本地上传、远程下载
2. 服务端需要新增一个上传文件的接口、文件大小限制为 100mb （可能上传的时 fat jar ）
3. 服务端创建和更新接口调整
4. 兼容既有的模板导入功能

sql server 同步时列注释无法获取

databasir 版本：1.0.4
sql server 版本：2017
sql server 驱动版本：9.4.1.jre8
我在同步的时候，发现所有列的中文注释都没有同步过来，请问应该怎么配置，我在常见问题里只看到 mysql 和 Oracle 的解决方案

Application start failed: cause IllegalArgumentException

log

Caused by: java.lang.IllegalArgumentException: null
        at java.base/java.util.concurrent.ThreadPoolExecutor.<init>(ThreadPoolExecutor.java:1293) ~[na:na]
        at org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor.initializeExecutor(ThreadPoolTaskExecutor.java:262) ~[spring-context-5.3.17.jar!/:5.3.17]
        at org.springframework.scheduling.concurrent.ExecutorConfigurationSupport.initialize(ExecutorConfigurationSupport.java:186) ~[spring-context-5.3.17.jar!/:5.3.17]
        at org.springframework.scheduling.concurrent.ExecutorConfigurationSupport.afterPropertiesSet(ExecutorConfigurationSupport.java:173) ~[spring-context-5.3.17.jar!/:5.3.17]
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1863) ~[spring-beans-5.3.17.jar!/:5.3.17]
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1800) ~[spring-beans-5.3.17.jar!/:5.3.17]
        ... 24 common frames omitted

fix: mysql trigger save failed by time format error

refactor: rename `plugin` module to `meta`

feature: support search global group, project, database

一些小建议

1. 增加主流驱动，目前就两个 mysql, postgresql 如果怕增加更多导致容量，看能否以插件的形式展示
2. 我部署在 1h2g 百度云机器上 只有这个和 mysql 以及另一个docker 访问起来点击有点卡 以及点如图这个 会卡四五秒不跳转还以为不能点
   
3. UML 是否会保存呢，我测试刷新就没了。即反向操作表
   

docs: redesign doc page and content

fix: missing table trigger name

非常棒的项目，但试用的时候发现几点问题

docker部署的情况
1、总是无故退出，无论我有没有在操作（似乎是固定了时间），由于是查阅文档，最好不要随便退出
2、当选择的是概览的时候UML并没有显示，最好能显示所有表的UML及各表之间的外键关系
3、可空栏是否能够不要使用“null”表示，最好使用YES或可表示，null对表程序人员看着感觉比较不好^_^
4、导出文档后能否在显示表列的时候把表的注释也显示出来？

一点小意见，希望此项目越来越好！！！

Fix：版本变更时如果只有表的注释改变的话，无法体现该变更

feature: support extend database variable when add database type

Can add exporting excel

Could you please add the function of exporting to excel?
this tool is very practical，but it has fewer export options, especially lack of export to excel!
so ,could you please add the function of exporting to excel?

部署项目报错08S01

2022-04-10 06:52:55.105 ERROR 6 --- [ main] o.s.b.web.embedded.tomcat.TomcatStarter : Error starting Tomcat context. Exception: org.springframework.beans.factory.UnsatisfiedDependencyException. Message: Error creating bean with name 'databasirOauth2LoginFilter' defined in URL [jar:file:/app/databasir.jar!/BOOT-INF/classes!/com/databasir/api/config/oauth2/DatabasirOauth2LoginFilter.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'securityConfig' defined in URL [jar:file:/app/databasir.jar!/BOOT-INF/classes!/com/databasir/api/config/SecurityConfig.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'databasirUserDetailService' defined in URL [jar:file:/app/databasir.jar!/BOOT-INF/classes!/com/databasir/api/config/security/DatabasirUserDetailService.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'userDao': Unsatisfied dependency expressed through field 'dslContext'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flywayInitializer' defined in class path resource [org/springframework/boot/autoconfigure/flyway/FlywayAutoConfiguration$FlywayConfiguration.class]: Invocation of init method failed; nested exception is org.flywaydb.core.internal.exception.FlywaySqlException:
Unable to obtain connection from database: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.

SQL State : 08S01
Error Code : 0
Message : Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.

2022-04-10 06:52:55.144 INFO 6 --- [ main] o.apache.catalina.core.StandardService : Stopping service [Tomcat]
2022-04-10 06:52:55.149 WARN 6 --- [ main] ConfigServletWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.context.ApplicationContextException: Unable to start web server; nested exception is org.springframework.boot.web.server.WebServerException: Unable to start embedded Tomcat
2022-04-10 06:52:55.163 INFO 6 --- [ main] ConditionEvaluationReportLoggingListener :

Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2022-04-10 06:52:55.192 ERROR 6 --- [ main] o.s.boot.SpringApplication : Application run failed

org.springframework.context.ApplicationContextException: Unable to start web server; nested exception is org.springframework.boot.web.server.WebServerException: Unable to start embedded Tomcat
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.onRefresh(ServletWebServerApplicationContext.java:163) ~[spring-boot-2.5.11.jar!/:2.5.11]
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:577) ~[spring-context-5.3.17.jar!/:5.3.17]
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:145) ~[spring-boot-2.5.11.jar!/:2.5.11]
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:775) ~[spring-boot-2.5.11.jar!/:2.5.11]
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:448) ~[spring-boot-2.5.11.jar!/:2.5.11]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:339) ~[spring-boot-2.5.11.jar!/:2.5.11]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1365) ~[spring-boot-2.5.11.jar!/:2.5.11]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1354) ~[spring-boot-2.5.11.jar!/:2.5.11]
at com.databasir.DatabasirApplication.main(DatabasirApplication.java:13) ~[classes!/:na]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:na]

fix: 表搜索框 placeholder 文案改为 `输入表名、注释、描述进行搜索`

feat: add table comment below table name in exported markdown

bug：hive 同步数据表字段出现 is_autoincrement not exists

错误日志

Exception in thread "projectSyncTaskThreadPoolTaskExecutor-1" java.lang.IllegalStateException: java.sql.SQLException: Could not find IS_AUTOINCREMENT in [table_cat, table_schem, table_name, column_name, data_type, type_name, column_size, buffer_length, decimal_digits, num_prec_radix, nullable, remarks, column_def, sql_data_type, sql_datetime_sub, char_octet_length, ordinal_position, is_nullable, scope_catalog, scope_schema, scope_table, source_data_type, is_auto_increment]
 at com.databasir.core.meta.repository.impl.jdbc.JdbcColumnMetaRepository.selectColumns(JdbcColumnMetaRepository.java:25)
 at com.databasir.core.meta.repository.impl.jdbc.JdbcTableMetaRepository.doSelect(JdbcTableMetaRepository.java:54)
 at com.databasir.core.meta.repository.impl.jdbc.JdbcTableMetaRepository.selectTables(JdbcTableMetaRepository.java:32)
 at com.databasir.core.meta.repository.impl.jdbc.JdbcDatabaseMetaRepository.select(JdbcDatabaseMetaRepository.java:47)
 at com.databasir.core.Databasir.get(Databasir.java:30)
 at com.databasir.core.domain.document.service.DocumentService.retrieveDatabaseMeta(DocumentService.java:144)
 at com.databasir.core.domain.document.service.DocumentService.syncByProjectId(DocumentService.java:94)
 at com.databasir.core.domain.document.service.DocumentService$$FastClassBySpringCGLIB$$122722d1.invoke(<generated>)
 at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
 at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
 at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
 at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123)
 at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:388)
 at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
 at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
 at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698)
 at com.databasir.core.domain.document.service.DocumentService$$EnhancerBySpringCGLIB$$3da3e00e.syncByProjectId(<generated>)
 at com.databasir.job.ProjectSyncTaskScheduler.sync(ProjectSyncTaskScheduler.java:53)
 at com.databasir.job.ProjectSyncTaskScheduler.lambda$startSyncTask$0(ProjectSyncTaskScheduler.java:45)
 at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
 at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
 at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.sql.SQLException: Could not find IS_AUTOINCREMENT in [table_cat, table_schem, table_name, column_name, data_type, type_name, column_size, buffer_length, decimal_digits, num_prec_radix, nullable, remarks, column_def, sql_data_type, sql_datetime_sub, char_octet_length, ordinal_position, is_nullable, scope_catalog, scope_schema, scope_table, source_data_type, is_auto_increment]
 at org.apache.hive.jdbc.HiveBaseResultSet.findColumn(HiveBaseResultSet.java:102)
 at org.apache.hive.jdbc.HiveBaseResultSet.getString(HiveBaseResultSet.java:547)
 at com.databasir.core.meta.repository.impl.jdbc.JdbcColumnMetaRepository.doSelect(JdbcColumnMetaRepository.java:55)
 at com.databasir.core.meta.repository.impl.jdbc.JdbcColumnMetaRepository.selectColumns(JdbcColumnMetaRepository.java:23)
 ... 23 more

解决方案

升级版本到大于 1.9.2 的版本或者使用 latest 版本

feature: support get trigger from oracle

fix: miss group owner in group list page when owner's size greater than 3

feature: use virtual table instead of common table in document page

文档无法搜索列注释内容

databasir 版本：最新 docker 版本
sql server 版本：2017
sql server 驱动版本：9.4.1.jre8
现在注释显示正常了，但是左侧的搜索栏里只能搜索表名，搜索注释和描述都不可以，如下图所示

feature: increment document comment size

feature: support cancel running or new sync task

feature: support get table trigger from mariaDB

feature: support export svg uml

Mysql 表备注未能获取并显示

有中文Comment的表同步后，文档信息里表的Comment列字段为空。
测试mysql版本 5.7.34。

refactor：重构版本差异对比接口

目前的版本差异数据是通过 /api/v1.0/projects/{projectId}/diff_documents 接口获取的，所以在进行版本差异对比的时候需要调用两次接口

1. 获取文档数据
2. 获取差异数据

然后再将数据整合，再在前端进行渲染，这样就造成了以下问题

1. 前端充斥了大量业务逻辑，维护混乱
2. 差异接口每次都会对项目进行全部差异对比，粒度太大
3. 在前端做了大量的运算处理，对用户体验不友好

为了解决上面提到的问题，故需要对该逻辑进行重构，从而带来更好的用户体验

refactor: support config db url parameters

fix: sqlserver 表同步不完整

感谢开源项目 问题:创建项目中同步Schema的使用

首先感谢作者开源项目
上午刷HelloGitHub刚看到就部署下来玩玩了
我不太理解这个项目创建过程当中需要强制添加的同步Schema的作用
然后找文档也没有对应的模块
烦请作者不吝赐教 这个具体的使用以及实际的作用

fix: connection properties not work when use custom database driver

fix: group owner/member can't view project logs

jar包应用本地启动后没创建表

下载jar包，本地启动没有创建表？是需要单独新建表吗？

feature: add spring open api annotation

tips

1：表的名称，建议也附加在表名后，可参考chiner的风格；
2：卡顿，操作不顺畅，多选操作后，会有明显卡顿；
3：感觉是好东西，但是需要完善的还有很多，加油，继续搞。

Feature: support import databaseType from json or url

feature: default select nothing when change to multi selection mode