Comments (7)
很奇怪的是,我之前是2000为一次batch插入是发生这个异常,换成 1W一次batchInsert没有问题
from ngbatis.
- 这是我的mapper语句
<insert id="batchInsertVertex" >
INSERT VERTEX ${tagName}
(
@for ( param in params ) {
${param}${paramLP.last?"":"," }
@}
)
VALUES
${dataStr};
</insert>
from ngbatis.
感觉有可能是因为2000条一页的时候开的线程数太多了?
from ngbatis.
但是我用的是线程池,6 core, 12 线程 队列 是 9999
如上所述,450页的话,就会循环450次 @async里面的线程方法去batchInsert
@Bean(name = "graphAsyncImportExecutor")
public Executor graphAsyncImportExecutor() {
//在这里修改
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
//配置核心线程数
executor.setCorePoolSize(Runtime.getRuntime().availableProcessors());
//配置最大线程数
executor.setMaxPoolSize(Runtime.getRuntime().availableProcessors() * 2);
//配置队列大小
executor.setQueueCapacity(queueCapacity);
//配置线程池中的线程的名称前缀
executor.setThreadNamePrefix(namePrefix);
// rejection-policy:当pool已经达到max size的时候,如何处理新任务
// CALLER_RUNS:不在新线程中执行任务,而是有调用者所在的线程来执行
executor.setRejectedExecutionHandler( new ThreadPoolExecutor.CallerRunsPolicy());
//执行初始化
executor.initialize();
return executor;
}
from ngbatis.
既然线程有不大的上限,所开线程一致的话,按道理2000一次所占用的内存会更小才对,是有些反直觉。
不好意思,刚刚看错了... 还以为是OutOfMemoryException。
报错有 Cause by 信息吗
from ngbatis.
- 这个是报错信息,我注释掉了batchInsert的语句后,并没有出现该异常了,是不是我的xml语句不正确呢
Unexpected exception occurred invoking async method: public void com.box.modules.work.graph_import.async.GraphImportAsync.asyncBatchInsertGraph(java.lang.String,java.lang.String,java.lang.String,java.util.Map,java.util.List)
java.lang.StackOverflowError: null
at com.box.modules.work.graph_import.service.impl.GraphBaseApiImpl.batchInsertVertex(GraphBaseApiImpl.java:126)
at com.box.modules.work.graph_import.async.GraphImportAsync.asyncBatchInsertGraph(GraphImportAsync.java:45)
at com.box.modules.work.graph_import.async.GraphImportAsync$$FastClassBySpringCGLIB$$5ed53427.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
at org.springframework.aop.interceptor.AsyncExecutionInterceptor.lambda$invoke$0(AsyncExecutionInterceptor.java:115)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
from ngbatis.
xml 看上去没有问题,如果dataStr每条写入值的栏位跟params一致,应该不会有太大问题。
from ngbatis.
Related Issues (20)
- xml中指明 resultType为 java.util.Map 时程序阻塞不执行 HOT 2
- java.lang.ClassCastException[update: caused by hot-reload plugin] HOT 7
- bulk insert for edges HOT 2
- go from语句 HOT 16
- error on dependency version HOT 9
- docs: searching in en docs is broken HOT 2
- No extra connection: All servers are broken. HOT 2
- 批量插入数据报错 HOT 1
- NebulaDaoBasic是否可以实现一个listEndNodes方法 HOT 2
- 支持 springboot 3.x 有时间计划嘛? HOT 8
- The problem and solution of the first query taking a long time. 第一次查询的性能问题及解决方法
- 两个脚本都基于 USE SPACE 动态切换图空间,第一个生效,第二个不生效。
- 微信丨NgBatis交流群 · NGC22 0611-29
- StackOverflowError HOT 5
- java.net.SocketException: Socket closed HOT 1
- ResultSet error: SyntaxError: syntax error near ` go ' HOT 7
- 返回值映射问题 HOT 1
- 查询失败: Create Session failed HOT 2
- 龙行龘龘,灯火万家
- 同mybatis-plus共用提示:java.lang.NullPointerException at org.nebula.contrib.ngbatis.proxy.MapperProxy.invoke(MapperProxy.java:74) HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ngbatis.