makersoft / mybatis-shards Goto Github PK
View Code? Open in Web Editor NEW基于MyBatis的数据库切分框架,可实现数据的水平切分和垂直切分。
Home Page: www.makersoft.org
License: Apache License 2.0
基于MyBatis的数据库切分框架,可实现数据的水平切分和垂直切分。
Home Page: www.makersoft.org
License: Apache License 2.0
分页查询时可能出现结果集重复或者有些数据干脆取不出来的问题。通常会在最后一页较为多见。暂时还没有想到应对策略。
如果能支持以下几种事件,操作起来将会非常方便!
EntityListeners:
@PostLoad
@PrePersist
@PostPersist
@PreRemove
@PostRemove
@PreUpdate
@postupdate
去除基本操作对于第三方jar包的过多依赖,减少体积。
欢迎大家过来一起讨论!
目前遇到一个如何合并结果集的问题:
当从多个库中查询出结果后,如何将结果集进行group by,count(xx),sum(xx)等等运算,大家有什么好的想法?
我目前想到的有:
1、对于分页查询时统计总数的问题,可按契约优先原则,规定sql map中的staement id为“findAllUser.count”,这样就可以知道此SQL的结果集要进行RowCount处理。
2、手动设置结果集合并策略。这个是最保守的办法,当然,加大了代码量。
3、最理想的办法是根据SQL语句,智能分析出需要采用哪些合并策略。分析SQL语句可以用现成的JSqlParser来完成,但是...这个还需要很多实践!当然了,嵌套SQL语句恐怕是无法支持了。
实现纵向切分
看到作者的这个项目维护了也有三年的时间了,想问一下有没有在生产上使用的案例?
我们项目中准备采用这个方案。
您好,我在公司的一份比较早期的工程中发现 有在使用这个框架的1.0.0-SNAPSHOT版本,使用的DBCP框架。这个应用在生产环境中,持续输出以下日志,该日志并不是异常,而是 DBCP针对被borrow出去的连接进行跟踪,如果超过一定时间未归还,则直接丢弃掉。这个堆栈代表的是该连接是哪次调用中申请的,不是普通异常中的堆栈。从这个可以看到应用是经过了mybatis-shards框架再到mybatis中申请的。
想和作者确认一下 是否有发现1.0.0-SNAPSHOT版本 中是否有此问题。borrow连接后未归还。
org.apache.commons.dbcp.AbandonedTrace$AbandonedObjectException: DBCP object created 2023-02-03 16:50:35 by the following code was never closed:
at org.apache.commons.dbcp.AbandonedTrace.init(AbandonedTrace.java:90)
at org.apache.commons.dbcp.AbandonedTrace.<init>(AbandonedTrace.java:73)
at org.apache.commons.dbcp.DelegatingDatabaseMetaData.<init>(DelegatingDatabaseMetaData.java:52)
at org.apache.commons.dbcp.DelegatingConnection.getMetaData(DelegatingConnection.java:345)
at org.apache.commons.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.getMetaData(PoolingDataSource.java:245)
at net.sf.log4jdbc.ConnectionSpy.getMetaData(ConnectionSpy.java:298)
at org.apache.ibatis.executor.resultset.DefaultResultSetHandler.getNextResultSet(DefaultResultSetHandler.java:245)
at org.apache.ibatis.executor.resultset.DefaultResultSetHandler.handleResultSets(DefaultResultSetHandler.java:184)
at org.apache.ibatis.executor.statement.PreparedStatementHandler.query(PreparedStatementHandler.java:64)
at org.apache.ibatis.executor.statement.RoutingStatementHandler.query(RoutingStatementHandler.java:79)
at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:63)
at com.sun.proxy.$Proxy267.query(Unknown Source)
at org.apache.ibatis.executor.SimpleExecutor.doQuery(SimpleExecutor.java:63)
at org.apache.ibatis.executor.BaseExecutor.queryFromDatabase(BaseExecutor.java:325)
at org.apache.ibatis.executor.BaseExecutor.query(BaseExecutor.java:156)
at org.apache.ibatis.executor.BaseExecutor.query(BaseExecutor.java:136)
at sun.reflect.GeneratedMethodAccessor85.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.ibatis.plugin.Invocation.proceed(Invocation.java:49)
at com.xx.bpo.common.page.PageableInterceptor.intercept(PageableInterceptor.java:84)
at org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:61)
at com.sun.proxy.$Proxy266.query(Unknown Source)
at sun.reflect.GeneratedMethodAccessor85.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.ibatis.plugin.Invocation.proceed(Invocation.java:49)
at com.xx.bpo.interceptor.TimerInterceptor.intercept(TimerInterceptor.java:50)
at org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:61)
at com.sun.proxy.$Proxy266.query(Unknown Source)
at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:148)
at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:141)
at org.apache.ibatis.session.defaults.DefaultSqlSession.selectOne(DefaultSqlSession.java:77)
at com.xx.bpo.common.page.PageableInterceptor.findCount(PageableInterceptor.java:217)
at com.xx.bpo.common.page.PageableInterceptor.intercept(PageableInterceptor.java:57)
at org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:61)
at com.sun.proxy.$Proxy266.query(Unknown Source)
at sun.reflect.GeneratedMethodAccessor85.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.ibatis.plugin.Invocation.proceed(Invocation.java:49)
at com.xx.bpo.interceptor.TimerInterceptor.intercept(TimerInterceptor.java:50)
at org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:61)
at com.sun.proxy.$Proxy266.query(Unknown Source)
at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:148)
at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:141)
at org.apache.ibatis.session.defaults.DefaultSqlSession.selectOne(DefaultSqlSession.java:77)
at sun.reflect.GeneratedMethodAccessor84.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.makersoft.shards.ShardImpl$SqlSessionInterceptor.invoke(ShardImpl.java:100)
at com.sun.proxy.$Proxy79.selectOne(Unknown Source)
at org.makersoft.shards.select.impl.ShardSelectImpl$2.execute(ShardSelectImpl.java:106)
at org.makersoft.shards.strategy.access.impl.SequentialShardAccessStrategy.apply(SequentialShardAccessStrategy.java:38)
at org.makersoft.shards.select.impl.ShardSelectImpl.getSingleResult(ShardSelectImpl.java:115)
at org.makersoft.shards.session.impl.ShardedSqlSessionImpl.selectOne(ShardedSqlSessionImpl.java:269)
at sun.reflect.GeneratedMethodAccessor84.invoke(Unknown So
...
@fengkuok , thanks a lot for sharding solution for mybatis as a big step beyond shardbatis. here comes our doubt that when the G.A releasing just like 1.0, looking forward to that with more document related.
在删除数据的时候老大建议用TRUNCATE ,之前在myBatis 中没有使用过,求救....
<bean id="shardedSqlSessionFactory" class="org.makersoft.shards.spring.ShardedSqlSessionFactoryBean">
<property name="mapperLocations" value="classpath*:/mybatis/mapper/*Mapper.xml"/>
<property name="typeAliasesPackage" value="org.makersoft.shards.domain" />
<property name="dataSources" ref="dsMap"/>
<property name="shardStrategyFactory">
<bean class="org.makersoft.shards.strategy.UserShardStrategyFactory"/>
</property>
<property name="idGenerator">
<bean class="org.makersoft.shards.id.uuid.ShardedUUIDGenerator"/>
</property>
</bean>
不仅user表要分片,还有其它的表也要分片,要怎么写配置.最好能提供一个unit case演示如何操作2表带分区的表。
比如我会写一个org.makersoft.shards.strategy.ProductShardStrategyFactory.
1、为方便Shards配置,扩展Spring schema。
2、依据扩展schema重构配置文件。
与 firstId 相关的方法都无法通过!
1、配置文件重构
2、测试基础类重构
3、测试方法重构
1)、采用mybatis拦截器机制,拦截SQL语句,采用jsqlpaser解析出order by子语句。
2)、将多数据源查询的结果集按sql解析出的排序方式进行重新排序。
结果集对象中必须包含排序字段,且该字段实现了Comparable接口。
如何对多列排序进行计算?进行嵌套多次排序?
例如如下数据:
————————————
id | age | birthday |
————————————
1 | 20 | 06-25 |
————————————
2 | 20 | 07-20 |
————————————
3 | 19 | 03-21 |
————————————
4 | 19 | 01-20 |
————————————
5 | 21 | 12-27 |
————————————
执行SQL语句:select t.age,t.birthday from user t order by t.age desc,t.birthday asc
结果集:
————————————
id | age | birthday |
————————————
5 | 21 | 12-27 |
————————————
1 | 20 | 06-25 |
————————————
2 | 20 | 07-20 |
————————————
4 | 19 | 01-20 |
————————————
3 | 19 | 03-21 |
————————————
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.