DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.
Home Page: https://github.com/WeBankFinTech/DataSphereStudio-Doc
License: Apache License 2.0
The error message is:
error code(错误码): 90002, error message(错误信息): add scheduler project failederrCode: 90019 ,desc: Connection reset ,ip: DataSphere-web1 ,port: 9004 ,serviceKind: dss-server.
Checked dss server log, found the following error messages:
2019-12-26 11:11:32.099 [ERROR] [qtp585718112-25 ] c.w.w.d.a.s.a.s.AzkabanSecurityService (80) [login] - 获取session失败: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:210) ~[?:1.8.0_232]
at java.net.SocketInputStream.read(SocketInputStream.java:141) ~[?:1.8.0_232]
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137) ~[httpcore-4.4.7.jar:4.4.7]
at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153) ~[httpcore-4.4.7.jar:4.4.7]
at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282) ~[httpcore-4.4.7.jar:4.4.7]
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) ~[httpclient-4.5.4.jar:4.5.4]
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) ~[httpclient-4.5.4.jar:4.5.4]
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259) ~[httpcore-4.4.7.jar:4.4.7]
2019-12-26 11:11:32.106 [ERROR] [qtp585718112-25 ] c.w.w.d.s.s.i.DWSProjectServiceImpl (166) [createSchedulerProject] - add scheduler project failed, com.webank.wedatasphere.dss.appjoint.exception.AppJointErrorException: errCode: 90019 ,desc: Connection reset ,ip: DataSphere-web1 ,port: 9004 ,serviceKind: dss-server
at com.webank.wedatasphere.dss.appjoint.scheduler.azkaban.service.AzkabanSecurityService.login(AzkabanSecurityService.java:81) ~[?:?]
at com.webank.wedatasphere.dss.server.function.FunctionInvoker.projectServiceAddFunction(FunctionInvoker.java:76) ~[dss-server-0.6.0.jar:?]
at com.webank.wedatasphere.dss.server.service.impl.DWSProjectServiceImpl.createSchedulerProject(DWSProjectServiceImpl.java:161) [dss-server-0.6.0.jar:?]
at com.webank.wedatasphere.dss.server.service.impl.DWSProjectServiceImpl.addProject(DWSProjectServiceImpl.java:109) [dss-server-0.6.0.jar:?]
at com.webank.wedatasphere.dss.server.service.impl.DWSProjectServiceImpl$$FastClassBySpringCGLIB$$fe55cc96.invoke(<generated>) [dss-server-0.6.0.jar:?]
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) [spring-core-5.0.7.RELEASE.jar:5.0.7.RELEASE]
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:746) [spring-aop-5.0.7.RELEASE.jar:5.0.7.RELEASE]
Azkaban is running.
tcp 0 0 0.0.0.0:12321 0.0.0.0:* LISTEN 4789/java
tcp 0 0 0.0.0.0:5005 0.0.0.0:* LISTEN 4789/java
tcp 0 0 0.0.0.0:8081 0.0.0.0:* LISTEN 4789/java
new jdbc file, open it error like image
**问题描述：登陆后无法连接上数据库
**已尝试解决方法：
已在 ${LINKIS_HOME}/linkis-metadata/conf/linkis.properties中添加了Hive meta信息，如下：
##datasource
wds.linkis.server.login.use.default=false
hive.meta.url=jdbc:mysql://127.0.0.1:3306/hive?characterEncoding=UTF-8
hive.meta.user=xxxx
hive.meta.password=xxxx
**查看日志
${LINKIS_HOME}/linkis-metadata/logs/linkis-metada.out
ERROR (main) ERROR DruidDataSource - mysql should not use 'PoolPreparedStatements'
yarn队列问题，查看spark-enginemanage的log文件，报错内容如下：
ERROR [qtp1670196451-332] com.webank.wedatasphere.linkis.rpc.RPCReceiveRestful 72 apply - error code(错误码): 111006, error message(错误信息): Get the Yarn queue information exception.(获取Yarn队列信息异常). com.webank.wedatasphere.linkis.common.exception.ErrorException: errCode: 111006 ,desc: Get the Yarn queue information exception.(获取Yarn队列信息异常) ,ip: localhost ,port: 9003 ,serviceKind: ResourceManager
at com.webank.wedatasphere.linkis.common.exception.ExceptionManager.generateException(ExceptionManager.java:52) ~[linkis-common-0.9.1.jar:?]
at com.webank.wedatasphere.linkis.rpc.transform.RPCConsumer$$anon$1.toObject(RPCConsumer.scala:67) ~[linkis-cloudRPC-0.9.1.jar:?]
at com.webank.wedatasphere.linkis.rpc.BaseRPCSender$$anonfun$ask$1.apply(BaseRPCSender.scala:87) ~[linkis-cloudRPC-0.9.1.jar:?]
at com.webank.wedatasphere.linkis.rpc.BaseRPCSender.execute(BaseRPCSender.scala:80) ~[linkis-cloudRPC-0.9.1.jar:?]
at com.webank.wedatasphere.linkis.rpc.BaseRPCSender.ask(BaseRPCSender.scala:83) ~[linkis-cloudRPC-0.9.1.jar:?]
at com.webank.wedatasphere.linkis.resourcemanager.client.ResourceManagerClient.requestResource(ResourceManagerClient.scala:56) ~[linkis-resourcemanager-client-0.9.1.jar:?]
at com.webank.wedatasphere.linkis.enginemanager.impl.ResourceRequesterImpl.request(ResourceRequesterImpl.scala:32) ~[linkis-ujes-enginemanager-0.9.1.jar:?]
at com.webank.wedatasphere.linkis.enginemanager.AbstractEngineManager.requestEngine(AbstractEngineManager.scala:64) ~[linkis-ujes-enginemanager-0.9.1.jar:?]
at com.webank.wedatasphere.linkis.enginemanager.AbstractEngineManager.requestEngine(AbstractEngineManager.scala:42) ~[linkis-ujes-enginemanager-0.9.1.jar:?]
at com.webank.wedatasphere.linkis.enginemanager.EngineManagerReceiver.receiveAndReply(EngineManagerReceiver.scala:104) ~[linkis-ujes-enginemanager-0.9.1.jar:?]
at com.webank.wedatasphere.linkis.enginemanager.EngineManagerReceiver.receiveAndReply(EngineManagerReceiver.scala:97) ~[linkis-ujes-enginemanager-0.9.1.jar:?]
at com.webank.wedatasphere.linkis.rpc.RPCReceiveRestful$$anonfun$receiveAndReply$1.apply(RPCReceiveRestful.scala:139) ~[linkis-cloudRPC-0.9.1.jar:?]
at com.webank.wedatasphere.linkis.rpc.RPCReceiveRestful$$anonfun$receiveAndReply$1.apply(RPCReceiveRestful.scala:139) ~[linkis-cloudRPC-0.9.1.jar:?]
at com.webank.wedatasphere.linkis.rpc.RPCReceiveRestful$$anonfun$com$webank$wedatasphere$linkis$rpc$RPCReceiveRestful$$receiveAndReply$1$$anonfun$apply$1.apply(RPCReceiveRestful.scala:134) ~[linkis-cloudRPC-0.9.1.jar:?]
at com.webank.wedatasphere.linkis.rpc.RPCReceiveRestful$$anonfun$com$webank$wedatasphere$linkis$rpc$RPCReceiveRestful$$receiveAndReply$1$$anonfun$apply$1.apply(RPCReceiveRestful.scala:134) ~[linkis-cloudRPC-0.9.1.jar:?]
at scala.Option.map(Option.scala:146) ~[scala-library-2.11.8.jar:?]
at com.webank.wedatasphere.linkis.rpc.RPCReceiveRestful$$anonfun$com$webank$wedatasphere$linkis$rpc$RPCReceiveRestful$$receiveAndReply$1.apply(RPCReceiveRestful.scala:134) ~[linkis-cloudRPC-0.9.1.jar:?]
at com.webank.wedatasphere.linkis.rpc.RPCReceiveRestful$$anonfun$com$webank$wedatasphere$linkis$rpc$RPCReceiveRestful$$receiveAndReply$1.apply(RPCReceiveRestful.scala:130) ~[linkis-cloudRPC-0.9.1.jar:?]
at com.webank.wedatasphere.linkis.common.utils.Utils$.tryCatch(Utils.scala:48) [linkis-common-0.9.1.jar:?]
at com.webank.wedatasphere.linkis.server.package$.catchMsg(package.scala:57) [linkis-module-0.9.1.jar:?]
at com.webank.wedatasphere.linkis.server.package$.catchIt(package.scala:89) [linkis-module-0.9.1.jar:?]
at com.webank.wedatasphere.linkis.rpc.RPCReceiveRestful.com$webank$wedatasphere$linkis$rpc$RPCReceiveRestful$$receiveAndReply(RPCReceiveRestful.scala:130) [linkis-cloudRPC-0.9.1.jar:?]
at com.webank.wedatasphere.linkis.rpc.RPCReceiveRestful.receiveAndReply(RPCReceiveRestful.scala:139) [linkis-cloudRPC-0.9.1.jar:?]
at com.webank.wedatasphere.linkis.rpc.RPCReceiveRestful$$FastClassBySpringCGLIB$$6973d04a.invoke(<generated>) [linkis-cloudRPC-0.9.1.jar:?]
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) [spring-core-5.0.7.RELEASE.jar:5.0.7.RELEASE]
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:746) [spring-aop-5.0.7.RELEASE.jar:5.0.7.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) [spring-aop-5.0.7.RELEASE.jar:5.0.7.RELEASE]
at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:88) [spring-aop-5.0.7.RELEASE.jar:5.0.7.RELEASE]
at com.webank.wedatasphere.linkis.server.restful.RestfulCatchAOP$$anonfun$dealMessageRestful$1.apply(RestfulCatchAOP.scala:39) [linkis-module-0.9.1.jar:?]
at com.webank.wedatasphere.linkis.server.restful.RestfulCatchAOP$$anonfun$dealMessageRestful$1.apply(RestfulCatchAOP.scala:39) [linkis-module-0.9.1.jar:?]
at com.webank.wedatasphere.linkis.common.utils.Utils$.tryCatch(Utils.scala:48) [linkis-common-0.9.1.jar:?]
at com.webank.wedatasphere.linkis.server.package$.catchMsg(package.scala:57) [linkis-module-0.9.1.jar:?]
at com.webank.wedatasphere.linkis.server.package$.catchIt(package.scala:89) [linkis-module-0.9.1.jar:?]
at com.webank.wedatasphere.linkis.server.restful.RestfulCatchAOP.dealMessageRestful(RestfulCatchAOP.scala:38) [linkis-module-0.9.1.jar:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_232]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_232]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_232]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_232]
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:644) [spring-aop-5.0.7.RELEASE.jar:5.0.7.RELEASE]
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:633) [spring-aop-5.0.7.RELEASE.jar:5.0.7.RELEASE]
at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70) [spring-aop-5.0.7.RELEASE.jar:5.0.7.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:174) [spring-aop-5.0.7.RELEASE.jar:5.0.7.RELEASE]
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) [spring-aop-5.0.7.RELEASE.jar:5.0.7.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) [spring-aop-5.0.7.RELEASE.jar:5.0.7.RELEASE]
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688) [spring-aop-5.0.7.RELEASE.jar:5.0.7.RELEASE]
at com.webank.wedatasphere.linkis.rpc.RPCReceiveRestful$$EnhancerBySpringCGLIB$$fda29307.receiveAndReply(<generated>) [linkis-cloudRPC-0.9.1.jar:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_232]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_232]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_232]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_232]
at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) [jaxrs-ri-2.21.jar:2.21.]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) [jaxrs-ri-2.21.jar:2.21.]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) [jaxrs-ri-2.21.jar:2.21.]
at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205) [jaxrs-ri-2.21.jar:2.21.]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) [jaxrs-ri-2.21.jar:2.21.]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) [jaxrs-ri-2.21.jar:2.21.]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) [jaxrs-ri-2.21.jar:2.21.]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) [jaxrs-ri-2.21.jar:2.21.]
at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:309) [jaxrs-ri-2.21.jar:2.21.]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [jaxrs-ri-2.21.jar:2.21.]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [jaxrs-ri-2.21.jar:2.21.]
at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [jaxrs-ri-2.21.jar:2.21.]
at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [jaxrs-ri-2.21.jar:2.21.]
at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [jaxrs-ri-2.21.jar:2.21.]
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [jaxrs-ri-2.21.jar:2.21.]
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:292) [jaxrs-ri-2.21.jar:2.21.]
at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1139) [jaxrs-ri-2.21.jar:2.21.]
at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:460) [jaxrs-ri-2.21.jar:2.21.]
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:386) [jaxrs-ri-2.21.jar:2.21.]
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:334) [jaxrs-ri-2.21.jar:2.21.]
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:221) [jaxrs-ri-2.21.jar:2.21.]
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1655) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:215) [websocket-server-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
at com.webank.wedatasphere.linkis.server.security.SecurityFilter.doFilter(SecurityFilter.scala:100) [linkis-module-0.9.1.jar:?]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200) [spring-web-5.0.7.RELEASE.jar:5.0.7.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.0.7.RELEASE.jar:5.0.7.RELEASE]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.filterAndRecordMetrics(WebMvcMetricsFilter.java:158) [spring-boot-actuator-2.0.3.RELEASE.jar:2.0.3.RELEASE]
at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.filterAndRecordMetrics(WebMvcMetricsFilter.java:126) [spring-boot-actuator-2.0.3.RELEASE.jar:2.0.3.RELEASE]
at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:111) [spring-boot-actuator-2.0.3.RELEASE.jar:2.0.3.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.0.7.RELEASE.jar:5.0.7.RELEASE]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.springframework.boot.actuate.web.trace.servlet.HttpTraceFilter.doFilterInternal(HttpTraceFilter.java:90) [spring-boot-actuator-2.0.3.RELEASE.jar:2.0.3.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.0.7.RELEASE.jar:5.0.7.RELEASE]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99) [spring-web-5.0.7.RELEASE.jar:5.0.7.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.0.7.RELEASE.jar:5.0.7.RELEASE]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:109) [spring-web-5.0.7.RELEASE.jar:5.0.7.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.0.7.RELEASE.jar:5.0.7.RELEASE]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:93) [spring-web-5.0.7.RELEASE.jar:5.0.7.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.0.7.RELEASE.jar:5.0.7.RELEASE]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200) [spring-web-5.0.7.RELEASE.jar:5.0.7.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.0.7.RELEASE.jar:5.0.7.RELEASE]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) [jetty-security-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.server.Server.handle(Server.java:531) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281) [jetty-io-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102) [jetty-io-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118) [jetty-io-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762) [jetty-util-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680) [jetty-util-9.4.11.v20180605.jar:9.4.11.v20180605]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_232]
请问如何解决？
The error message was:
"
operation failed(操作失败)s！the reason(原因)：HttpClientResultException: errCode: 10905 ,desc: URL http://127.0.0.1:9001/api/rest_j/v1/bml/upload request failed! ResponseBody is {"method":null,"status":1,"message":"error code(错误码): 50073, error message(错误信息): 提交上传资源任务失败:\n### Error updating database. Cause: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'linkis.linkis_resources_task' doesn't exist\n### The error may involve com.webank.wedatasphere.linkis.bml.dao.TaskDao.insert-Inline\n### The error occurred while setting parameters\n### SQL: INSERT INTO linkis_resources_task( resource_id,version,operation,state, submit_user,system,instance, client_ip,err_msg,start_time,end_time,last_update_time, extra_params ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)\n### Cause: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'linkis.linkis_resources_task' doesn't exist\n; bad SQL grammar []; nested exception is com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'linkis.linkis_resources_task' doesn't exist.","data":{"errorMsg":{"serviceKind":"bml-server","level":2,"port":9999,"errCode":50073,"ip":"DataSphere-web1","desc":"提交上传资源任务失败:\n### Error updating database. Cause: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'linkis.linkis_resources_task' doesn't exist\n### The error may involve com.webank.wedatasphere.linkis.bml.dao.TaskDao.insert-Inline\n### The error occurred while setting parameters\n### SQL: INSERT INTO linkis_resources_task( resource_id,version,operation,state, submit_user,system,instance, client_ip,err_msg,start_time,end_time,last_update_time, extra_params ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)\n### Cause: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'linkis.linkis_resources_task' doesn't exist\n; bad SQL grammar []; nested exception is com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'linkis.linkis_resources_task' doesn't exist"}}}. ,ip: DataSphere-web1 ,port: 9004 ,serviceKind: dss-server
"
According to the page https://github.com/WeBankFinTech/DataSphereStudio/blob/master/docs/zh_CN/ch1/DSS%E5%AE%89%E8%A3%85%E5%B8%B8%E8%A7%81%E9%97%AE%E9%A2%98%E5%88%97%E8%A1%A8.md#6-dss%E5%88%9B%E5%BB%BA%E5%B7%A5%E7%A8%8B%E6%8A%A5%E8%A1%A8linkislinkis_resources_task%E4%B8%8D%E5%AD%98%E5%9C%A8, I needed to run db/moudle/linkis-bml.sql manually.
Tried this SQL file, found only linkis_resources_task table was missing, ran the query manually on the mysql session. The above error was gone, found a new one.
"
operation failed(操作失败)s！the reason(原因)：HttpClientResultException: errCode: 10905 ,desc: URL http://127.0.0.1:9001/api/rest_j/v1/bml/upload request failed! ResponseBody is {"method":null,"status":1,"message":"error code(错误码): 50073, error message(错误信息): 提交上传资源任务失败:\n### Error updating database. Cause: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Column 'system' cannot be null\n### The error may involve com.webank.wedatasphere.linkis.bml.dao.TaskDao.insert-Inline\n### The error occurred while setting parameters\n### SQL: INSERT INTO linkis_resources_task( resource_id,version,operation,state, submit_user,system,instance, client_ip,err_msg,start_time,end_time,last_update_time, extra_params ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)\n### Cause: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Column 'system' cannot be null\n; SQL []; Column 'system' cannot be null; nested exception is com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Column 'system' cannot be null.","data":{"errorMsg":{"serviceKind":"bml-server","level":2,"port":9999,"errCode":50073,"ip":"DataSphere-web1","desc":"提交上传资源任务失败:\n### Error updating database. Cause: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Column 'system' cannot be null\n### The error may involve com.webank.wedatasphere.linkis.bml.dao.TaskDao.insert-Inline\n### The error occurred while setting parameters\n### SQL: INSERT INTO linkis_resources_task( resource_id,version,operation,state, submit_user,system,instance, client_ip,err_msg,start_time,end_time,last_update_time, extra_params ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)\n### Cause: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Column 'system' cannot be null\n; SQL []; Column 'system' cannot be null; nested exception is com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Column 'system' cannot be null"}}}. ,ip: DataSphere-web1 ,port: 9004 ,serviceKind: dss-server
"
mysql> show create table linkis_resources_task \G
*************************** 1. row ***************************
Table: linkis_resources_task
Create Table: CREATE TABLE `linkis_resources_task` (
`id` bigint(20) NOT NULL AUTO_INCREMENT,
`resource_id` varchar(50) DEFAULT NULL COMMENT '资源id，资源的uuid',
`version` varchar(20) DEFAULT NULL COMMENT '当前操作的资源版本号',
`operation` varchar(20) NOT NULL COMMENT '操作类型.upload = 0, update = 1',
`state` varchar(20) NOT NULL DEFAULT 'Schduled' COMMENT '任务当前状态:Schduled, Running, Succeed, Failed,Cancelled',
`submit_user` varchar(20) NOT NULL DEFAULT '' COMMENT '任务提交用户名',
`system` varchar(20) NOT NULL DEFAULT '' COMMENT '子系统名 wtss',
`instance` varchar(50) NOT NULL COMMENT '物料库实例',
`client_ip` varchar(50) DEFAULT NULL COMMENT '请求IP',
`extra_params` text COMMENT '额外关键信息.如批量删除的资源IDs及versions,删除资源下的所有versions',
`err_msg` varchar(2000) DEFAULT NULL COMMENT '任务失败信息.e.getMessage',
`start_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '开始时间',
`end_time` datetime DEFAULT NULL COMMENT '结束时间',
`last_update_time` datetime NOT NULL COMMENT '最后更新时间',
PRIMARY KEY (`id`),
UNIQUE KEY `resource_id_version` (`resource_id`,`version`,`operation`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4
1 row in set (0.00 sec)
Convenient for users to directly reference DSS related dependencies when compiling
问题1：在visualis界面执行sql
错误如下，前端一直转圈
后端错误
问题解决方案：
在visualis-server/lib目录删除jsr311-api-1.1.1.jar这个包【jsr包冲突导致】
验证：已经正常，如下图
问题2:工程中工作流节点保存失败
错误如下：
解决方案：在linkis-publish linkins-properties里面新增getway的key-value
wds.linkis.gateway.ip=192.168.201.85
wds.linkis.gateway.port=9011
重启linkis-publish微服务即可。
问题3:工作流执行失败
错误如下：
解决方案：
打开linkis-getway /conf/linkis.properties文件，如下：
将wds.linkis.gateway.conf.url.pass.auth 其中/api/rest_j/v1/entrance删掉，只留下/dws即可正常执行工作流。
问题4：工作流发布到azkaban里面执行错误，但是在dss单独执行工作流成功
错误如下：
问题原因：由于安装azkaban的服务器hadoop+hive jar存在低版本的netty包，存在兼容问题。
解决方案：暂未解决
问题5:工作流节点执行成功，但是保存失败
错误如下：
问题6:关于Qualitis初始化任务成功，执行任务失败的问题
错误截图如下：前提条件，scripts、visualis执行spark任务都是成功的，获取队列无异常。
问题解决方案：单独部署的Qualitis，问题已定位，是Qualitis默认用户admin的队列和Dss用户hive的队列不一致导致的。我利用admin用户创建hive用户，并为hive赋admin权限，即可解决此问题。
目前数据质量已经集成到DSS。
查看linkis-metadata组件的log日志,报错信息为
ERROR(Druid-ConnectionPool-Create) ERROR DruidDataSource - create connection error
java.sql.SQLSyntaxErrorException:Syntax error:Encountered "" at line 1 ,column 8.
After the selected node of sendemail sending item is deleted, the publishing project will report an error：NoSuchElementException:No value present
Hello,DSS community.I would like to know:
How does work flow of dss translate to azkaban's runtime?
How does azkaban communicate with dss?
I am reading the code but still confused.Very appreciated it if you could give me some hints.
问题描述：关于工程中某个工作流节点单独执行不报错，保存工作流时提示：NotClassDefFoundError：Cloud not initialize class dispatch.Http$
问题原因：linkis-publish微服务关于netty-3.6.2.Final.jar升级包缺失
问题解决：将升级包上传重启linkis-publish微服务即可。
安装linkis jobtypes
按照官方安装文档进行自动化安装，执行sh install.sh最后一步报错：{"error":"Missing required parameter 'execid'."}。并没有看到文档中所说的“如果安装成功最后会打印：{"status":"success"}”，但是能在azkaban的/plugins/jobtypes目录下看到已经安装好的linkis任务插件。通过排查在安装脚本最后一步会去调用"curl http://azkaban_ip:executor_port/executor?action=reloadJobTypePlugins"进行插件的刷新。重启azkaban executor日志中看到已经加载了插件的信息 INFO [JobTypeManager][Azkaban] Loaded jobtype linkis com.webank.wedatasphere.dss.plugins.azkaban.linkis.jobtype.AzkabanDssJobType。当时没有排查到相应的问题于是跳过。当发布linkis任务到azkaban执行成功之后反过来复盘这个问题的时候，这确定应该是个误报信息。
从dss发布project到azkaban
问题描述：日志报错azkaban不存在当前用户
问题排查：确认报用户不存在的用户是能正常访问的azkaban的，异常堆栈日志被捕获了没有太多日志。于是本地远程调试发现在AzkabanSecurityService#getSession方法执行httpClient.execute(httpPost, context)时直接报错了。我们的azkaban开启了https当前登录的接口不支持https，临时的解决方案是关闭了azkaban的https。
问题2的衍生
解决完第一个问题之后还是不能发布任务，但是response = httpClient.execute(httpPost, context);
reponse返回的信息已经是变为“incorrect login”。最后排查发现是把azkaban的登录请求中的password写成了userpwd，改了重新打包验证通过。
任务发布成功但执行失败
问题描述：
- azkaban.flow.start.month=12
05-12-2019 21:15:42 CST sql INFO - azkaban.flow.start.hour=21
05-12-2019 21:15:42 CST sql INFO - azkaban.flow.uuid=bbdc0985-4a2c-4dab-94ff-ab38e2bbde24
05-12-2019 21:15:42 CST sql INFO - azkaban.flow.flowid=publish_demo_xc_
05-12-2019 21:15:42 CST sql INFO - azkaban.flow.start.day=05
05-12-2019 21:15:42 CST sql INFO - azkaban.job.metadata.file=_job.653.sql.meta
05-12-2019 21:15:42 CST sql INFO - azkaban.flow.start.timestamp=2019-12-05T21:15:42.829+08:00
05-12-2019 21:15:42 CST sql INFO - linkistype=linkis.spark.sql
05-12-2019 21:15:42 CST sql INFO - ****** End Job properties ******
05-12-2019 21:15:42 CST sql ERROR - Job run failed!
05-12-2019 21:15:42 CST sql ERROR - nullnull
05-12-2019 21:15:42 CST sql INFO - Finishing job sql at 1575551742852 with status FAILED
排查之后发现是没拿到提交用户，获取提交用户用的是
azkaban.flow.submituser
这个参数经过3.8版本的azkaban和2.5版本的对比发现2.5没有这个参数
解决方案：临时解决方案是把azkaban.flow.submituser和user.to.proxy 这两个参数写死打包替换，用于流程测试
另外一种方案就是编译3.8版本然后重新安装部署azkaban
We’d like to thank everyone in this community for your constant support of DataSphere Studio. We’re confident that, with our effort and your support, this community could grow more prosperous and serve a greater number of users.
We would appreciate it if you could leave us a comment with the following information:
Sample：
Thanks again!!!
Your support is the biggest motivator for facilitating the progress of open-sourcing the DataSphere Studio!
Yours sincerely,
DataSphere Studio Team
感谢社区每一位关注并使用DataSphere Studio的伙伴。我们会持续投入，争取将DataSphere Studio社区和生态打造的更加繁荣，让更多伙伴从DataSphere Studio中受益。
期望您提交一条评论, 内容包括:
示例：
再次感谢!!!
您的支持是DataSphere Studio开源前进的最大动力！！
DataSphere Studio团队拜上
报错信息：
ERROR [qtp1152298548-27] controller.RedirectController: Failed to redirect to other page, caused by: java.net.ConnectException: http://null/api/rest_j/v1/user/userInfo
java.util.concurrent.ExecutionException: java.net.ConnectException: http://null/api/rest_j/v1/user/userInfo
at com.ning.http.client.providers.netty.NettyResponseFuture.abort(NettyResponseFuture.java:342) ~[async-http-client-1.8.10.jar:?]
at com.ning.http.client.providers.netty.NettyConnectListener.operationComplete(NettyConnectListener.java:107) ~[async-http-client-1.8.10.jar:?]
at org.jboss.netty.channel.DefaultChannelFuture.notifyListener(DefaultChannelFuture.java:427) ~[netty-3.9.2.Final.jar:?]
at org.jboss.netty.channel.DefaultChannelFuture.addListener(DefaultChannelFuture.java:145) ~[netty-3.9.2.Final.jar:?]
at com.ning.http.client.providers.netty.NettyAsyncHttpProvider.doConnect(NettyAsyncHttpProvider.java:1138) ~[async-http-client-1.8.10.jar:?]
at com.ning.http.client.providers.netty.NettyAsyncHttpProvider.execute(NettyAsyncHttpProvider.java:935) ~[async-http-client-1.8.10.jar:?]
at com.ning.http.client.AsyncHttpClient.executeRequest(AsyncHttpClient.java:499) ~[async-http-client-1.8.10.jar:?]
at dispatch.HttpExecutor$class.apply(execution.scala:47) ~[dispatch-core_2.11-0.11.2.jar:0.11.2]
at dispatch.Http.apply(execution.scala:12) ~[dispatch-core_2.11-0.11.2.jar:0.11.2]
at dispatch.HttpExecutor$class.apply(execution.scala:42) ~[dispatch-core_2.11-0.11.2.jar:0.11.2]
at dispatch.Http.apply(execution.scala:12) ~[dispatch-core_2.11-0.11.2.jar:0.11.2]
at com.webank.wedatasphere.linkis.httpclient.AbstractHttpClient.executeAsyncRequest(AbstractHttpClient.scala:199) ~[linkis-httpclient-0.9.0.jar:?]
at com.webank.wedatasphere.linkis.httpclient.AbstractHttpClient.executeRequest(AbstractHttpClient.scala:195) ~[linkis-httpclient-0.9.0.jar:?]
at com.webank.wedatasphere.linkis.httpclient.AbstractHttpClient.execute(AbstractHttpClient.scala:94) ~[linkis-httpclient-0.9.0.jar:?]
at com.webank.wedatasphere.linkis.httpclient.AbstractHttpClient.execute(AbstractHttpClient.scala:87) ~[linkis-httpclient-0.9.0.jar:?]
at com.webank.wedatasphere.dss.appjoint.auth.impl.AppJointAuthImpl.getRedirectMsg(AppJointAuthImpl.scala:57) ~[dss-appjoint-auth-0.5.0.jar:?]
at com.webank.wedatasphere.qualitis.controller.RedirectController.redirectToCoordinatePage(RedirectController.java:70) ~[web_app-0.7.0.jar:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_231]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_231]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_231]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_231]
at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) ~[jersey-server-2.26.jar:?]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) [jersey-server-2.26.jar:?]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) [jersey-server-2.26.jar:?]
at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:243) [jersey-server-2.26.jar:?]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) [jersey-server-2.26.jar:?]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) [jersey-server-2.26.jar:?]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) [jersey-server-2.26.jar:?]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) [jersey-server-2.26.jar:?]
at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) [jersey-server-2.26.jar:?]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) [jersey-common-2.26.jar:?]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) [jersey-common-2.26.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:316) [jersey-common-2.26.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:298) [jersey-common-2.26.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:268) [jersey-common-2.26.jar:?]
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) [jersey-common-2.26.jar:?]
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) [jersey-server-2.26.jar:?]
at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) [jersey-server-2.26.jar:?]
at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) [jersey-container-servlet-core-2.26.jar:?]
at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) [jersey-container-servlet-core-2.26.jar:?]
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) [jersey-container-servlet-core-2.26.jar:?]
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) [jersey-container-servlet-core-2.26.jar:?]
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) [jersey-container-servlet-core-2.26.jar:?]
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1655) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:215) [websocket-server-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
at com.webank.wedatasphere.qualitis.filter.Filter1AuthorizationFilter.doFilter(Filter1AuthorizationFilter.java:111) [web_user-0.7.0.jar:?]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99) [spring-web-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:109) [spring-web-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:93) [spring-web-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200) [spring-web-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) [jetty-security-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.server.Server.handle(Server.java:531) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281) [jetty-io-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102) [jetty-io-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118) [jetty-io-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) [jetty-util-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) [jetty-util-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) [jetty-util-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) [jetty-util-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) [jetty-util-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762) [jetty-util-9.4.11.v20180605.jar:9.4.11.v20180605]
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680) [jetty-util-9.4.11.v20180605.jar:9.4.11.v20180605]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_231]
Caused by: java.net.ConnectException: http://null/api/rest_j/v1/user/userInfo
at com.ning.http.client.providers.netty.NettyConnectListener.operationComplete(NettyConnectListener.java:103) ~[async-http-client-1.8.10.jar:?]
... 88 more
Caused by: java.nio.channels.UnresolvedAddressException
at sun.nio.ch.Net.checkAddress(Net.java:101) ~[?:1.8.0_231]
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:622) ~[?:1.8.0_231]
at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink.connect(NioClientSocketPipelineSink.java:108) ~[netty-3.9.2.Final.jar:?]
at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink.eventSunk(NioClientSocketPipelineSink.java:70) ~[netty-3.9.2.Final.jar:?]
at org.jboss.netty.handler.codec.oneone.OneToOneEncoder.handleDownstream(OneToOneEncoder.java:54) ~[netty-3.9.2.Final.jar:?]
at org.jboss.netty.handler.codec.http.HttpClientCodec.handleDownstream(HttpClientCodec.java:97) ~[netty-3.9.2.Final.jar:?]
at org.jboss.netty.handler.stream.ChunkedWriteHandler.handleDownstream(ChunkedWriteHandler.java:109) ~[netty-3.9.2.Final.jar:?]
at org.jboss.netty.channel.Channels.connect(Channels.java:634) ~[netty-3.9.2.Final.jar:?]
at org.jboss.netty.channel.AbstractChannel.connect(AbstractChannel.java:207) ~[netty-3.9.2.Final.jar:?]
at org.jboss.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:229) ~[netty-3.9.2.Final.jar:?]
at org.jboss.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:182) ~[netty-3.9.2.Final.jar:?]
at com.ning.http.client.providers.netty.NettyAsyncHttpProvider.doConnect(NettyAsyncHttpProvider.java:1099) ~[async-http-client-1.8.10.jar:?]
... 85 more
出现找不到脚本文件的错误！
Q:dss在启动时 visualis-server log总是提示 Web server failed to start. Port 9007 was already in use
A:经排查，dss/conf/config.sh 部署时指定了「�机器外网IP」，但是这台机器，有内网IP和外网IP之分。执行一键部署脚本时，visualis-server服务配置文件里会自动生成「内网IP」……
在scripts中新建hive表，无数据库选择项。
console log
vue.esm.js:1897 TypeError: Cannot read property 'value' of undefined
at setDbList (firstStep.vue:306)
at s.<anonymous> (firstStep.vue:292)
at Array.<anonymous> (vue.esm.js:1989)
at dt (vue.esm.js:1915)
i follow the Installation guide but
there no user in the linkis or dss database and
linkis-gateway error logs
2019-12-12 19:13:54.007 WARN [reactor-http-nio-3]
com.webank.wedatasphere.linkis.gateway.security.LDAPUserRestful 55 warn -
wrong user name or password
javax.naming.ConfigurationException:
java.naming.provider.url property does not contain a URL
help！thanks
error code(错误码): 90002, error message(错误信息): add scheduler project failederrCode: 90019 ,desc: URI does not specify a valid host name: http://:/checkin ,ip: sdw1 ,port: 9004 ,serviceKind: dss-server.
After integrating Dashboard/Display as a node of DSS, we should further enhance the integration with Visualis by adding Widget node, which accepts a Spark-SQL node as its content, and then automatically parses and creates a View based on that SQL statement with meta-data ready. This will make users to only focus on workflow editing page and do not need to jump to Visualis page to do View level construction anymore.
add jdbc Node
dss-0.6.0创建工程时，dss-sever log 提示：add scheduler project failederrCode: 90019 ,desc: Connection reset
Can I replace azkaban with another scheduler? Like airflow or easyscheduler.
简单版
安装Linkis(0.9.2)算是正常吧，除了JDBC服务没有启动，其它服务正常启动；
简单版
安装dss(0.6.0)没有明显的错误日志，不知道怎么解决
<-------------------------------->
Begin to start dss-server
INFO: + End to start dss-server
<-------------------------------->
<-------------------------------->
Begin to start dss-flow-execution-entrance
INFO: + End to start dss-flow-execution-entrance
<-------------------------------->
<-------------------------------->
Begin to start linkis-appjoint-entrance
INFO: + End to start linkis-appjoint-entrance
<-------------------------------->
<-------------------------------->
Begin to start visualis-server
INFO: + End to start visualis-server
<-------------------------------->
When I create a project in DataSphereStudio Web， it shows an Exception below:
2019-12-05 18:27:42.597 ERROR [qtp370279024-91] com.webank.wedatasphere.linkis.server.restful.RestfulCatchAOP 83 apply - operation failed(操作失败)s java.lang.IllegalAccessError: azkaban登录
失败：无此用户
at com.webank.wedatasphere.dss.appjoint.scheduler.azkaban.service.AzkabanSecurityService.lambda$getSession$1(AzkabanSecurityService.java:107) ~[dss-azkaban-scheduler-appjoint-0.5.0.jar:?]
This is my config in azkaban-user.xml in azkaban:
<azkaban-users>
<user groups="azkaban" password="azkaban" roles="admin" username="azkaban"/>
<user groups="jiaorenyu" password="jiaorenyu" roles="admin" username="jiaorenyu"/>
<user password="metrics" roles="metrics" username="metrics"/>
<role name="admin" permissions="ADMIN"/>
<role name="metrics" permissions="METRICS"/>
</azkaban-users>
This is my config in token.properties in dss:
jiaorenyu=jiaorenyu
I can login both system(azkaban and dss) use user=jiaorenyu, password=jiaorenyu
dss create workflow，报
om.webank.wedatasphere.linkis.bml.common.HdfsResourceHelper 74 upload - hadoop write to hdfs:///tmp/linkis/hadoop/20191212/09282afb-bb3c-4b9b-9cb8-c78ece3a2428.json failed, reason is, IOException: java.io.IOException: You have not permission to access path /tmp/linkis/hadoop/20191212/09282afb-bb3c-4b9b-9cb8-c78ece3a2428.json
。 my permission is
drwxrwxrwx - hadoop hadoop 0 2019-12-03 16:52 /tmp .
no problems.
I need help ,thanku!
2019-12-12 23:04:41.706 ERROR [Engine-Scheduler-ThreadPool-2] com.webank.wedatasphere.appjoint.QualitisAppJoint 106 submit - Error! Can not submit job java.lang.NullPointerException: null
at com.webank.wedatasphere.appjoint.QualitisNodeExecution.submit(QualitisNodeExecution.java:67) [dss-qualitis-appjoint-0.6.0.jar:?]
at com.webank.wedatasphere.dss.appjoint.execution.core.LongTermNodeExecution.execute(LongTermNodeExecution.java:65) [dss-appjoint-core-0.6.0.jar:?]
at com.webank.wedatasphere.dss.linkis.appjoint.entrance.execute.AppJointEntranceEngine.execute(AppJointEntranceEngine.scala:169) [linkis-appjoint-entrance-0.6.0.jar:?]
at com.webank.wedatasphere.dss.linkis.appjoint.entrance.job.AppJointEntranceJob$$anonfun$1.apply(AppJointEntranceJob.scala:75) [linkis-appjoint-entrance-0.6.0.jar:?]
at com.webank.wedatasphere.dss.linkis.appjoint.entrance.job.AppJointEntranceJob$$anonfun$1.apply(AppJointEntranceJob.scala:75) [linkis-appjoint-entrance-0.6.0.jar:?]
at com.webank.wedatasphere.linkis.common.utils.Utils$.tryCatch(Utils.scala:48) [linkis-common-0.9.1.jar:?]
at com.webank.wedatasphere.dss.linkis.appjoint.entrance.job.AppJointEntranceJob.run(AppJointEntranceJob.scala:75) [linkis-appjoint-entrance-0.6.0.jar:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_141]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_141]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_141]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_141]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_141]
添加项目报错，具体错误如下，我是一台机器部署的标准版linkis，一台部署的简单版studio， eureka上只看到dss-server一个studio的服务，重启过几次studio, 其他的几个studio服务一直看不到。
error code(错误码): 90002, error message(错误信息): add scheduler project failederrCode: 90019 ,desc: errCode: 90020 ,desc: 用户token为空 ,ip: cancer-03.hdp.com ,port: 9004 ,serviceKind: dss-server ,ip: cancer-03.hdp.com ,port: 9004 ,serviceKind: dss-server.
请求失败截图
Is your feature request related to a problem? Please describe.
DSS is not managing the user's development-> test-> production release process, it is recommended to control this process
Describe the solution you'd like
No
Describe alternatives you've considered
No
Additional context
No
保存节点后，回到主页再进入工作流，节点回退到之前的版本。
因为节点保存不会触发工作流保存。
1.The execution directory is separated from the installation directory.
2.The default path of visualis-server's yml file frontEnd is wrong.
3.Simple version does not install qualitis and azkaban by default.
Update instructions and attentions
Error java.lang.NoSuchMethodError: org.json4s.jackson.JsonMethod$.parse(Lorg.json4s/JsonInput;z).......
非hive数据源，是mysql数据源
1、View_List可以查询数据
2、Widget List无数据
Azkaban version 2.5 does not have the azkaban.flow.submituser parameter, which will cause the task to be submitted to linkis for execution failure
前后端嵌入iframe的形式, 对容器嵌入并不是特别友好.
容器中一般都指定一个service的名称这样去访问, 但是在浏览器中就不认识了.
最好可以考虑使用一个反向代理转发一下, 嵌入的还是自己服务的固定的路径, 转发到对应的服务.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.