UDF功能发现bug：用户自定义函数类加载错误。

[dtstack]

UDF功能发现bug：用户自定义函数类加载错误。

[nuaabuaa07]

这样修改后可以顺利提交作业到flink了。但是在flink集群端出现如下错误：
223 2018-12-07 14:18:16,413 DEBUG org.apache.flink.runtime.blob.BlobServerConnection - Received GET request for BLOB 1db8e3f89693ac94cc7239e55df1ba8a/p-672fdcc3bcdf246851350a93b7ccb7b8960fed5a-04aa27459028ae9855c54752 c8ac7a59 from /10.90.1.113. 224 2018-12-07 14:18:16,455 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: test_flink2_STREAMSRCTABLEEE -> Map -> to: Row -> from: (NAME, HEIGHT, PROCTIME) -> select: (STRINGLENGTUDF(NAME) AS STRLE NGTH, NAME) -> to: Tuple2 -> Sink: t3 (1/1) (730ebbc0398b763f4c9eb57676f0e7b0) switched from DEPLOYING to RUNNING. 225 2018-12-07 14:18:17,176 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: test_flink2_STREAMSRCTABLEEE -> Map -> to: Row -> from: (NAME, HEIGHT, PROCTIME) -> select: (STRINGLENGTUDF(NAME) AS STRLE NGTH, NAME) -> to: Tuple2 -> Sink: t3 (1/1) (730ebbc0398b763f4c9eb57676f0e7b0) switched from RUNNING to FAILED. 226 org.apache.flink.api.common.InvalidProgramException: Table program cannot be compiled. This is a bug. Please file an issue. 227 at org.apache.flink.table.codegen.Compiler$class.compile(Compiler.scala:36) 228 at org.apache.flink.table.runtime.CRowProcessRunner.compile(CRowProcessRunner.scala:35) 229 at org.apache.flink.table.runtime.CRowProcessRunner.open(CRowProcessRunner.scala:49) 230 at org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36) 231 at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102) 232 at org.apache.flink.streaming.api.operators.ProcessOperator.open(ProcessOperator.java:56) 233 at org.apache.flink.streaming.runtime.tasks.StreamTask.openAllOperators(StreamTask.java:420) 234 at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:296) 235 at org.apache.flink.runtime.taskmanager.Task.run(Task.java:712) 236 at java.lang.Thread.run(Thread.java:748) 237 Caused by: org.codehaus.commons.compiler.CompileException: Line 5, Column 11: Cannot determine simple type name "test" 238 at org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:11672) 239 at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6416) 240 at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6177) 241 at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6190) 242 at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6190) 243 at org.codehaus.janino.UnitCompiler.getType2(UnitCompiler.java:6156) 244 at org.codehaus.janino.UnitCompiler.access$13300(UnitCompiler.java:212)
查询官网后，貌似还是用户自定义函数的类的加载问题导致的。

[nuaabuaa07]

在readme中的demo中，第一行自定义函数后缺少分号。

[ayinghaha]

在readme中的demo中，第一行自定义函数后缺少分号。

我用yarnPer模式提交的时候也遇到这个错误了，加了分号没有作用额；
另外请问如何让application记录这些异常日志，我只能在flink的web中看到

[ayinghaha]

在readme中的demo中，第一行自定义函数后缺少分号。
确实是自定义函数类加载导致的，去掉自定义函数就能正常运行了，大神知道如何解决么

[limingxi01]

有没有解决？

[yaobukeji123]

有没有交流群啊 我这个udf也没有成功。
No match found for function signature UDFTEST(
有没有成功运行的 求助

[nuaabuaa07]

类加载有问题的。我在flink的lib文件夹中放入自定义udf类jar包，重启flink，然后可以使用了。但是生产环境肯定不能这样的。

[lixiyuan123]

类加载有问题的。我在flink的lib文件夹中放入自定义udf类jar包，重启flink，然后可以使用了。但是生产环境肯定不能这样的。

我也遇到这个问题了，请问您最后是如何解决的？

[xuelvming]

这块确实应该是classloader引起的问题，但是目前在不同的提交模式下，flink的类加载逻辑不是很清楚，希望能有个交流群，一起讨论下这类问题，包括flink版本的兼容性，flink sql对于batch 批处理的支持等等

[gcstar]

yarn模式下 提交udf jar包好像也有问题吧，同样报这个错误 ,需要将udf的jar包放入flink 的lib目录下 并重启yarn-session才能运行

[hejufang]

碰到了同样的问题，这个有啥子解决方法嘛？在本地可以运行，使用yarn perjob模式跑就会出现同样的错。

[yangsishu]

把报错贴出来，这个问题很早就解决掉了

[tcx1000]

把报错贴出来，这个问题很早就解决掉了

Yarn模式，通过-C引入用户自定义jar包（是个UDF），抛下面的异常：
org.apache.flink.api.common.InvalidProgramException: Table program cannot be compiled. This is a bug. Please file an issue. at org.apache.flink.table.codegen.Compiler$class.compile(Compiler.scala:36) at org.apache.flink.table.runtime.CRowProcessRunner.compile(CRowProcessRunner.scala:35) at org.apache.flink.table.runtime.CRowProcessRunner.open(CRowProcessRunner.scala:49) at org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36) at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102) at org.apache.flink.streaming.api.operators.ProcessOperator.open(ProcessOperator.java:56) at org.apache.flink.streaming.runtime.tasks.StreamTask.openAllOperators(StreamTask.java:424) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:290) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711) at java.lang.Thread.run(Thread.java:745) Caused by: org.codehaus.commons.compiler.CompileException: Line 5, Column 10: Cannot determine simple type name "com" at org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:11672) at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6416) at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6177) at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6190) at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6190) at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6190) at org.codehaus.janino.UnitCompiler.getType2(UnitCompiler.java:6156) at org.codehaus.janino.UnitCompiler.access$13300(UnitCompiler.java:212) at org.codehaus.janino.UnitCompiler$18$1.visitReferenceType(UnitCompiler.java:6064) at org.codehaus.janino.UnitCompiler$18$1.visitReferenceType(UnitCompiler.java:6059) at org.codehaus.janino.Java$ReferenceType.accept(Java.java:3754) at org.codehaus.janino.UnitCompiler$18.visitType(UnitCompiler.java:6059) at org.codehaus.janino.UnitCompiler$18.visitType(UnitCompiler.java:6052) at org.codehaus.janino.Java$ReferenceType.accept(Java.java:3753) at org.codehaus.janino.UnitCompiler.getType(UnitCompiler.java:6052) at org.codehaus.janino.UnitCompiler.access$1200(UnitCompiler.java:212) at org.codehaus.janino.UnitCompiler$21.getType(UnitCompiler.java:7844) at org.codehaus.janino.UnitCompiler.getType2(UnitCompiler.java:6456) at org.codehaus.janino.UnitCompiler.access$13800(UnitCompiler.java:212) at org.codehaus.janino.UnitCompiler$18$2$1.visitFieldAccess(UnitCompiler.java:6082) at org.codehaus.janino.UnitCompiler$18$2$1.visitFieldAccess(UnitCompiler.java:6077) at org.codehaus.janino.Java$FieldAccess.accept(Java.java:4136) at org.codehaus.janino.UnitCompiler$18$2.visitLvalue(UnitCompiler.java:6077) at org.codehaus.janino.UnitCompiler$18$2.visitLvalue(UnitCompiler.java:6073) at org.codehaus.janino.Java$Lvalue.accept(Java.java:3974) at org.codehaus.janino.UnitCompiler$18.visitRvalue(UnitCompiler.java:6073) at org.codehaus.janino.UnitCompiler$18.visitRvalue(UnitCompiler.java:6052) at org.codehaus.janino.Java$Rvalue.accept(Java.java:3942) at org.codehaus.janino.UnitCompiler.getType(UnitCompiler.java:6052) at org.codehaus.janino.UnitCompiler.getType2(UnitCompiler.java:6438) at org.codehaus.janino.UnitCompiler.access$13600(UnitCompiler.java:212) at org.codehaus.janino.UnitCompiler$18$2$1.visitAmbiguousName(UnitCompiler.java:6080) at org.codehaus.janino.UnitCompiler$18$2$1.visitAmbiguousName(UnitCompiler.java:6077) at org.codehaus.janino.Java$AmbiguousName.accept(Java.java:4050) at org.codehaus.janino.UnitCompiler$18$2.visitLvalue(UnitCompiler.java:6077) at org.codehaus.janino.UnitCompiler$18$2.visitLvalue(UnitCompiler.java:6073) at org.codehaus.janino.Java$Lvalue.accept(Java.java:3974) at org.codehaus.janino.UnitCompiler$18.visitRvalue(UnitCompiler.java:6073) at org.codehaus.janino.UnitCompiler$18.visitRvalue(UnitCompiler.java:6052) at org.codehaus.janino.Java$Rvalue.accept(Java.java:3942) at org.codehaus.janino.UnitCompiler.getType(UnitCompiler.java:6052) at org.codehaus.janino.UnitCompiler.findIMethod(UnitCompiler.java:8591) at org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:4708) at org.codehaus.janino.UnitCompiler.access$8200(UnitCompiler.java:212) at org.codehaus.janino.UnitCompiler$12.visitMethodInvocation(UnitCompiler.java:4071) at org.codehaus.janino.UnitCompiler$12.visitMethodInvocation(UnitCompiler.java:4044) at org.codehaus.janino.Java$MethodInvocation.accept(Java.java:4874) at org.codehaus.janino.UnitCompiler.compileGet(UnitCompiler.java:4044) at org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:5224) at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:3445) at org.codehaus.janino.UnitCompiler.access$5000(UnitCompiler.java:212) at org.codehaus.janino.UnitCompiler$9.visitMethodInvocation(UnitCompiler.java:3424) at org.codehaus.janino.UnitCompiler$9.visitMethodInvocation(UnitCompiler.java:3396) at org.codehaus.janino.Java$MethodInvocation.accept(Java.java:4874) at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3396) at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2316) at org.codehaus.janino.UnitCompiler.access$1700(UnitCompiler.java:212) at org.codehaus.janino.UnitCompiler$6.visitExpressionStatement(UnitCompiler.java:1450) at org.codehaus.janino.UnitCompiler$6.visitExpressionStatement(UnitCompiler.java:1443) at org.codehaus.janino.Java$ExpressionStatement.accept(Java.java:2848) at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1443) at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1523) at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3052) at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler.java:1313) at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler.java:1286) at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:785) at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:436) at org.codehaus.janino.UnitCompiler.access$400(UnitCompiler.java:212) at org.codehaus.janino.UnitCompiler$2.visitPackageMemberClassDeclaration(UnitCompiler.java:390) at org.codehaus.janino.UnitCompiler$2.visitPackageMemberClassDeclaration(UnitCompiler.java:385) at org.codehaus.janino.Java$PackageMemberClassDeclaration.accept(Java.java:1405) at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:385) at org.codehaus.janino.UnitCompiler.compileUnit(UnitCompiler.java:357) at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:234) at org.codehaus.janino.SimpleCompiler.compileToClassLoader(SimpleCompiler.java:446) at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:213) at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:204) at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:80) at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:75) at org.apache.flink.table.codegen.Compiler$class.compile(Compiler.scala:33) ... 9 more

[dingyj]

把报错贴出来，这个问题很早就解决掉了

Yarn模式，通过-C引入用户自定义jar包（是个UDF），抛下面的异常：
org.apache.flink.api.common.InvalidProgramException: Table program cannot be compiled. This is a bug. Please file an issue. at org.apache.flink.table.codegen.Compiler$class.compile(Compiler.scala:36) at org.apache.flink.table.runtime.CRowProcessRunner.compile(CRowProcessRunner.scala:35) at org.apache.flink.table.runtime.CRowProcessRunner.open(CRowProcessRunner.scala:49) at org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36) at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102) at org.apache.flink.streaming.api.operators.ProcessOperator.open(ProcessOperator.java:56) at org.apache.flink.streaming.runtime.tasks.StreamTask.openAllOperators(StreamTask.java:424) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:290) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711) at java.lang.Thread.run(Thread.java:745) Caused by: org.codehaus.commons.compiler.CompileException: Line 5, Column 10: Cannot determine simple type name "com" at org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:11672) at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6416) at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6177) at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6190) at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6190) at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6190) at org.codehaus.janino.UnitCompiler.getType2(UnitCompiler.java:6156) at org.codehaus.janino.UnitCompiler.access$13300(UnitCompiler.java:212) at org.codehaus.janino.UnitCompiler$18$1.visitReferenceType(UnitCompiler.java:6064) at org.codehaus.janino.UnitCompiler$18$1.visitReferenceType(UnitCompiler.java:6059) at org.codehaus.janino.Java$ReferenceType.accept(Java.java:3754) at org.codehaus.janino.UnitCompiler$18.visitType(UnitCompiler.java:6059) at org.codehaus.janino.UnitCompiler$18.visitType(UnitCompiler.java:6052) at org.codehaus.janino.Java$ReferenceType.accept(Java.java:3753) at org.codehaus.janino.UnitCompiler.getType(UnitCompiler.java:6052) at org.codehaus.janino.UnitCompiler.access$1200(UnitCompiler.java:212) at org.codehaus.janino.UnitCompiler$21.getType(UnitCompiler.java:7844) at org.codehaus.janino.UnitCompiler.getType2(UnitCompiler.java:6456) at org.codehaus.janino.UnitCompiler.access$13800(UnitCompiler.java:212) at org.codehaus.janino.UnitCompiler$18$2$1.visitFieldAccess(UnitCompiler.java:6082) at org.codehaus.janino.UnitCompiler$18$2$1.visitFieldAccess(UnitCompiler.java:6077) at org.codehaus.janino.Java$FieldAccess.accept(Java.java:4136) at org.codehaus.janino.UnitCompiler$18$2.visitLvalue(UnitCompiler.java:6077) at org.codehaus.janino.UnitCompiler$18$2.visitLvalue(UnitCompiler.java:6073) at org.codehaus.janino.Java$Lvalue.accept(Java.java:3974) at org.codehaus.janino.UnitCompiler$18.visitRvalue(UnitCompiler.java:6073) at org.codehaus.janino.UnitCompiler$18.visitRvalue(UnitCompiler.java:6052) at org.codehaus.janino.Java$Rvalue.accept(Java.java:3942) at org.codehaus.janino.UnitCompiler.getType(UnitCompiler.java:6052) at org.codehaus.janino.UnitCompiler.getType2(UnitCompiler.java:6438) at org.codehaus.janino.UnitCompiler.access$13600(UnitCompiler.java:212) at org.codehaus.janino.UnitCompiler$18$2$1.visitAmbiguousName(UnitCompiler.java:6080) at org.codehaus.janino.UnitCompiler$18$2$1.visitAmbiguousName(UnitCompiler.java:6077) at org.codehaus.janino.Java$AmbiguousName.accept(Java.java:4050) at org.codehaus.janino.UnitCompiler$18$2.visitLvalue(UnitCompiler.java:6077) at org.codehaus.janino.UnitCompiler$18$2.visitLvalue(UnitCompiler.java:6073) at org.codehaus.janino.Java$Lvalue.accept(Java.java:3974) at org.codehaus.janino.UnitCompiler$18.visitRvalue(UnitCompiler.java:6073) at org.codehaus.janino.UnitCompiler$18.visitRvalue(UnitCompiler.java:6052) at org.codehaus.janino.Java$Rvalue.accept(Java.java:3942) at org.codehaus.janino.UnitCompiler.getType(UnitCompiler.java:6052) at org.codehaus.janino.UnitCompiler.findIMethod(UnitCompiler.java:8591) at org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:4708) at org.codehaus.janino.UnitCompiler.access$8200(UnitCompiler.java:212) at org.codehaus.janino.UnitCompiler$12.visitMethodInvocation(UnitCompiler.java:4071) at org.codehaus.janino.UnitCompiler$12.visitMethodInvocation(UnitCompiler.java:4044) at org.codehaus.janino.Java$MethodInvocation.accept(Java.java:4874) at org.codehaus.janino.UnitCompiler.compileGet(UnitCompiler.java:4044) at org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:5224) at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:3445) at org.codehaus.janino.UnitCompiler.access$5000(UnitCompiler.java:212) at org.codehaus.janino.UnitCompiler$9.visitMethodInvocation(UnitCompiler.java:3424) at org.codehaus.janino.UnitCompiler$9.visitMethodInvocation(UnitCompiler.java:3396) at org.codehaus.janino.Java$MethodInvocation.accept(Java.java:4874) at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3396) at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2316) at org.codehaus.janino.UnitCompiler.access$1700(UnitCompiler.java:212) at org.codehaus.janino.UnitCompiler$6.visitExpressionStatement(UnitCompiler.java:1450) at org.codehaus.janino.UnitCompiler$6.visitExpressionStatement(UnitCompiler.java:1443) at org.codehaus.janino.Java$ExpressionStatement.accept(Java.java:2848) at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1443) at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1523) at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3052) at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler.java:1313) at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler.java:1286) at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:785) at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:436) at org.codehaus.janino.UnitCompiler.access$400(UnitCompiler.java:212) at org.codehaus.janino.UnitCompiler$2.visitPackageMemberClassDeclaration(UnitCompiler.java:390) at org.codehaus.janino.UnitCompiler$2.visitPackageMemberClassDeclaration(UnitCompiler.java:385) at org.codehaus.janino.Java$PackageMemberClassDeclaration.accept(Java.java:1405) at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:385) at org.codehaus.janino.UnitCompiler.compileUnit(UnitCompiler.java:357) at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:234) at org.codehaus.janino.SimpleCompiler.compileToClassLoader(SimpleCompiler.java:446) at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:213) at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:204) at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:80) at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:75) at org.apache.flink.table.codegen.Compiler$class.compile(Compiler.scala:33) ... 9 more

请问解决了吗？多谢

[dingyj]

解决了，我也是yarn方式，我重启的yarn session，多谢