Comments (2)
Hi,@jeffie
为了提升性能,并没有把所有响应数据都返回,过滤掉了部分数据,可以自行设置FilterPath进行控制
ClientOptions clientOptions = new ClientOptions();
clientOptions.setRefreshOption("refresh=false");//为了测试效果,能够实时查看数据,启用强制刷新机制,可是修改为"refresh=true"
//根据性能要求,控制批处理返回报文内容
clientOptions.setFilterPath("took,errors,items.*.error");
//批量添加或者修改2万个文档,将两个对象添加到索引表demo中,批量添加2万条记录耗时1.8s,
String response = clientUtil.addDocuments("demo",//索引表
demos,//批量处理数据集合
clientOptions);
异步批处理的话,亦可以自行设置FilterPath进行控制
//定义BulkProcessor批处理组件构建器
BulkProcessorBuilder bulkProcessorBuilder = new BulkProcessorBuilder();
bulkProcessorBuilder.setBlockedWaitTimeout(0)//指定bulk数据缓冲队列已满时后续添加的bulk数据排队等待时间,如果超过指定的时候数据将被拒绝处理,单位:毫秒,默认为0,不拒绝并一直等待成功为止
.setBulkFailRetry(1)//如果处理失败,重试次数,暂时不起作用
.setBulkSizes(10)//按批处理数据记录数
.setFlushInterval(5000)//强制bulk操作时间,单位毫秒,如果自上次bulk操作flushInterval毫秒后,数据量没有满足BulkSizes对应的记录数,但是有记录,那么强制进行bulk处理
.setWarnMultsRejects(1000)//bulk处理操作被每被拒绝WarnMultsRejects次(1000次),在日志文件中输出拒绝告警信息
.setWorkThreads(2)//bulk处理工作线程数
.setWorkThreadQueue(2)//bulk处理工作线程池缓冲队列大小
.setBulkProcessorName("test_bulkprocessor")//工作线程名称,实际名称为BulkProcessorName-+线程编号
.setBulkRejectMessage("Reject test bulkprocessor")//bulk处理操作被每被拒绝WarnMultsRejects次(1000次),在日志文件中输出拒绝告警信息提示前缀
.setElasticsearch("default")//指定Elasticsearch集群数据源名称,bboss可以支持多数据源
.addBulkInterceptor(new BulkInterceptor() {
public void beforeBulk(BulkCommand bulkCommand) {
logger.debug("beforeBulk");
}
public void afterBulk(BulkCommand bulkCommand, String result) {
logger.debug("afterBulk:"+result);
logger.debug("totalSize:"+bulkCommand.getTotalSize());
logger.debug("totalFailedSize:"+bulkCommand.getTotalFailedSize());
}
public void exceptionBulk(BulkCommand bulkCommand, Throwable exception) {
logger.error("exceptionBulk:",exception);
}
public void errorBulk(BulkCommand bulkCommand, String result) {
logger.warn("errorBulk:"+result);
}
})
//为了提升性能,并没有把所有响应数据都返回,过滤掉了部分数据,可以自行设置FilterPath进行控制
.setFilterPath("took,errors,items.*.error")
from bboss-elasticsearch.
设置filterPath后,可以反序列化成功了
from bboss-elasticsearch.
Related Issues (20)
- 关于All elasticServer [http://es-busi:9200] can't been connected HOT 1
- 【反馈】 HOT 1
- 实体类字段是LocalDateTime或者Date,使用dsl查询返回为null HOT 1
- 编译源码出错,无法下载依赖 HOT 1
- 6.3.0版本 能添加返回es查询语句的功能吗 HOT 11
- 分词带空格会报错 HOT 1
- 你好,请问支持分表的动态同步数据么?可以在conf中设置当前的月份对应的表 HOT 1
- 动态dsl多级循环遍历出错 HOT 1
- 请教大佬,请问支持添加自定义拦截器吗,我想根环境变量添加影子索引
- Connection pool shut down HOT 2
- serialJson=true疑问 HOT 1
- Amazon OpenSearch HOT 6
- 查询结果不支持从stored_fields取数? HOT 3
- 如何禁止一个字段不索引 HOT 1
- bboss-elasticsearch-rest-jdbc:6.3.0 错误 HOT 3
- 偶发请求失败 failed to respond HOT 2
- kerberos认证
- spring boot 执行shutdown的时候报错 HOT 9
- 什么时候能兼容springboot3.2版本? HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bboss-elasticsearch.