任务提交到flink1.5.4的on yarn 模式时报“Haven't been developed yet!”

[dtstack]

任务提交到flink1.5.4的on yarn 模式时报“Haven't been developed yet!”

[yangsishu]

现在1.5的standalone 模式提交有问题，yarn没问题

[nuaabuaa07]

上面我写的有点问题。
我在使用yarn模式提交任务时，会提示Haven't been developed yet!
sh ./submit.sh -sql /home/jinzhiliang/test/flink/sqlTXT/sideSql.txt -name xctest -remoteSqlPluginPath /home/dl/projects/flink-1.5.4/plugins -localSqlPluginPath /home/dl/projects/flink-1.5.4/plugins -mode yarn -flinkconf /home/dl/projects/flink-1.5.4/conf -confProp {"time.characteristic":"EventTime","sql.checkpoint.interval":10000}

[yangsishu]

你把你启动flink的命令发下

[nuaabuaa07]

启动flink的命令是 :
./bin/yarn-session.sh -n 2 -jm 1024 -tm 1024
提交任务的命令是：
sh ./submit.sh -sql /home/jinzhiliang/test/flink/sqlTXT/sideSql.txt -name xctest -remoteSqlPluginPath /home/dl/projects/flink-1.5.4/plugins -localSqlPluginPath /home/dl/projects/flink-1.5.4/plugins -mode yarn -flinkconf /home/dl/projects/flink-1.5.4/conf -confProp {"time.characteristic":"EventTime","sql.checkpoint.interval":10000}

[zoudaokoulife]

文档描述了。你要提交到yarn上需要添加 yarnconf 配置项

[nuaabuaa07]

添加yarnconf配置参数后，不会报Haven't been developed yet!错误了。
但是的集群配置了kerberos权限验证。任务提交到yarn上时估计被限制了。flinkstreamsql项目目前还没有针对kerberos做相关开发吧~~~
错误日志为：
Exception in thread "main" java.lang.RuntimeException: org.apache.hadoop.security.AccessControlException: SIMPLE authentication is not enabled. Available:[TOKEN, KERBEROS]
at com.dtstack.flink.sql.launcher.ClusterClientFactory.createYarnClient(ClusterClientFactory.java:152)
at com.dtstack.flink.sql.launcher.ClusterClientFactory.createClusterClient(ClusterClientFactory.java:61)
at com.dtstack.flink.sql.launcher.LauncherMain.main(LauncherMain.java:60)
Caused by: org.apache.hadoop.security.AccessControlException: SIMPLE authentication is not enabled. Available:[TOKEN, KERBEROS]
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45

[yangsishu]

现在还不支持kerberos

[nuaabuaa07]

关于kerberos认证的问题，我按照Cloudera官网 （https://www.cloudera.com/documentation/enterprise/5-15-x/topics/cdh_sg_princ_auth_java.html）上说明配置了认证信息。现在提交任务时不报kerberos的问题了。
但是出现如下错误：
Principal Authentication:
Exception in thread "main" java.lang.RuntimeException: java.lang.ClassCastException: org.apache.hadoop.yarn.proto.YarnServiceProtos$GetApplicationsRequestProto cannot be cast to com.google.protobuf.Message
at com.dtstack.flink.sql.launcher.ClusterClientFactory.createYarnClient(ClusterClientFactory.java:152)
at com.dtstack.flink.sql.launcher.ClusterClientFactory.createClusterClient(ClusterClientFactory.java:61)
at com.dtstack.flink.sql.launcher.LauncherMain.main(LauncherMain.java:69)
Caused by: java.lang.ClassCastException: org.apache.hadoop.yarn.proto.YarnServiceProtos$GetApplicationsRequestProto cannot be cast to com.google.protobuf.Message
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
at com.sun.proxy.$Proxy9.getApplications(Unknown Source)
at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getApplications(ApplicationClientProtocolPBClientImpl.java:223)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
at com.sun.proxy.$Proxy10.getApplications(Unknown Source)
at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getApplications(YarnClientImpl.java:344)
at com.dtstack.flink.sql.launcher.ClusterClientFactory.createYarnClient(ClusterClientFactory.java:112)
... 2 more
貌似是类型转换错误，我的flink是1.5.2 on yarn部署的。hadoop是2.6.0版本，CDH是5.15.1，查看源代码是yarnClient.getApplications(set, enumSet) 这个方法时报错。

[nuaabuaa07]

请问，目前master分支支持1.5版本flink的standalone了吗？master里的内容打算何时同步到1.6分支里呢？

[yangsishu]

近期会flink1.6的兼容做掉

[yangsishu]

master是支持flink1.5版本的standalone