Comments (2)
+1
from seatunnel.
+1
I have solved this problem, you can configure it as follows:
seatunnel:
engine:
history-job-expire-minutes: 1440
backup-count: 1
queue-type: blockingqueue
print-execution-info-interval: 60000
print-job-metrics-info-interval: 60000
slot-service:
dynamic-slot: true
checkpoint:
interval: 600000
timeout: 600000
storage:
type: hdfs
max-retained: 3
plugin-config:
namespace: /data_test/seatunnel/checkpoint/
storage.type: hdfs
fs.defaultFS: hdfs://intsig-bigdata-nameservice # Ensure that the directory has written permission
seatunnel.hadoop.dfs.nameservices: intsig-bigdata-nameservice
seatunnel.hadoop.dfs.ha.namenodes.intsig-bigdata-nameservice: namenode1900,namenode183
seatunnel.hadoop.dfs.namenode.rpc-address.intsig-bigdata-nameservice.namenode1900: centos-bigdata-datanode-10-24-2-200.intsig.internal:8020
seatunnel.hadoop.dfs.namenode.rpc-address.intsig-bigdata-nameservice.namenode183: centos-bigdata-namenode-10-2-5-100.intsig.internal:8020
seatunnel.hadoop.dfs.client.failover.proxy.provider.intsig-bigdata-nameservice: org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider
seatunnel.hadoop.dfs.namenode.kerberos.principal: hdfs/[email protected]
seatunnel.hadoop.dfs.datanode.kerberos.principal: hdfs/[email protected]
seatunnel.hadoop.rpc.protection: authentication
kerberosPrincipal: [email protected]
kerberosKeytabFilePath: /home/yueyang_li/yueyang_li.keytab
from seatunnel.
Related Issues (20)
- FileNotFoundException: HADOOP_HOME and hadoop.home.dir are unset. HOT 2
- [Feature][Paimon] Support cdc write of paimon sink HOT 1
- [Feature][DebeziumJsonSerializationSchema] add table source info into debezium-json format HOT 4
- [Bug] [Connectors-V2] Sink job can't continue if one record data too long for sink table column HOT 1
- [Bug] [Connector-V2] Error reported if field name is a keyword, when sinking Oracle
- submit a job failed HOT 1
- [Bug] [mysql cdc] can't read data from mysql cdc source,Both v2.3.3 and v2.3.4 tested the same and reported errors HOT 2
- [Bug] [mysql cdc] org.apache.seatunnel.connectors.seatunnel.cdc.mysql.source.MySqlDialect.getPrimaryKey< Error NullPointerException> HOT 6
- 2.3.4版本为什么运行demo都无法成功 HOT 4
- HIVE开启并行同步报错 HOT 1
- when source is ORACLE-CDC target is ORACLE, using is_exactly_once=true then will lock target table HOT 1
- We plan to synchronize the data of MySQL, Oracle and db2 in real time. Can you support it later? Do you have the requirements of DB license
- We plan to synchronize the data of MySQL, Oracle and db2 in real time. Can you support it later? Do you have the requirements of DB license HOT 1
- [Bug] [Mysql-cdc] Mysql-cdc cannot automatically synchronize newly created fields in DDL HOT 6
- [Feature][ Zeta Rest Api] add authorization for zeta rest api HOT 1
- mysqlcdc,postgresqlcdc支持将已经被更新或删除的数据再次同步到sink吗? HOT 2
- [Bug] [SinkMetrics] the sink metrics collect will get NPE when data type is String[] HOT 1
- SeaTunnel Community Meeting HOT 6
- Missing parameter for jdbc sink savemode case HOT 1
- [Bug] [SQLserver CDC] sqlserver jdbc url encrypt=false not work HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from seatunnel.