ververica/ververica-platform-playground

Instructions for getting started with Ververica Platform on minikube.

Welcome to the Ververica Platform Playground

This playground provides a hands-on introduction to Ververica Platform with minimal prerequisites.

Please check out the corresponding getting started guide in the Ververica Platform documentation.

About

Ververica Platform is the enterprise stream processing platform by the original creators of Apache Flink.

ververica-platform-playground's Issues

Artifact section missing in the UI for the playground 2.1 branch

Hi,

I have followed https://www.ververica.com/blog/analyzing-github-activity-with-ververica-platform-community-edition but when I open the Ververica platform UI the "Artifacts" sections isn't displayed.

Could it be that the blob storage hasn't been registered properly?

Im on windows with minikube and all the pods starts correctly.

Any advice would be appreciated.

Thanks,

Regards
Dan

文档太烂

https://www.ververica.com/getting-started 文档中指出5分钟就能安装好，都是扯淡。完全没用

Getting unauthorized when I follow https://www.ververica.com/getting-started

executing the last step of getting-started.

atwong@Alberts-MBP ~ % curl localhost:8080/api/v1/namespaces/defaults/deployments -H "Accept: application/yaml"
Unauthorized%

Doesn't tell me how to fix this.

Mention the example Jar URI earlier

As I was going through the getting started guide when creating the deployment I needed to enter the Jar URI. In the screenshot I saw a long, non-copyable URL and I felt lost at whether I should copy it or what I would have to provide. Only after a little bit I noticed the info box which is way below.

I think there are two improvements to be made:

1. The list with explanations for each required input (name, deployment target, …) should come before the screenshot.
2. This description for Jar URI should already mention the example URI.

I applied for trial, why there is no response

I applied for trial on the official website, why there is no response。

ververica-platform是不需要用户名密码登录的吗？

请问ververica-platform是不需要用户名密码登录的吗？

vvp-ververica-platform deployment CrashLoopBackOff due to SQLite missing table error

Setup

I am trying to run the playground into on-prem Kubernetes cluster.
From release-2.6 with minimal helm values edit as follows:
minio

persistence:
  storageClass: directpv-min-io
  size: 100Gi

vvp

acceptCommunityEditionLicense: true
persistentVolume:
  storageClass: directpv-min-io

Then I am executing setup.sh

Error

The appmanager container on vvp-ververica-platform* pod keeps crashing due to failing to execute the following SQL query SELECT name, collection, value FROM appmanager WHERE collection = ? with the following exception org.sqlite.SQLiteException: [SQLITE_ERROR] SQL error or missing database (no such table: appmanager). This seems weird because I have logged on the kubernetes cluster host and can see that this table is there. Full container log

Here is the check that I do on the host:

[root@nuc-71032 pvc-81f069f7-af47-4819-806b-cedfbabc95ff]# ls
vvp.db	vvp.db-shm  vvp.db-wal
[root@nuc-71032 pvc-81f069f7-af47-4819-806b-cedfbabc95ff]# sqlite3 vvp.db
SQLite version 3.37.2 2022-01-06 13:25:41
Enter ".help" for usage hints.
sqlite> .tables
DATABASECHANGELOG      catalog_databases      gateway              
DATABASECHANGELOGLOCK  catalog_functions    
appmanager             catalog_tables       
sqlite> SELECT name, collection, value FROM appmanager WHERE collection = ?;
sqlite> SELECT name FROM appmanager;
575d4a23-85f6-4c93-9c45-da49c2213d4a
00000000-0000-0000-0000-000000000000
sqlite> 

---

Please tell me if I need to provide additional details for this.
Looking up to fix/workaround.

Helm 3 install does not have stable repo configured.

I think this needs to go before this line

helm repo add stable https://kubernetes-charts.storage.googleapis.com

An error when I try to upload a jar

Hello all,
I wanna upload a jar to the platform, but there was an unknown error, anybody has the same problem?

the log of container gateway:

`
2022-01-29 02:40:23.455 ERROR 1 --- [-8080-exec-6] o.a.c.c.C.[.[.[/].[dispatcherServlet] : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is com.amazonaws.SdkClientException: Unable to execute HTTP request: minio.vvp.svc] with root cause

java.net.UnknownHostException: minio.vvp.svc
at java.base/java.net.InetAddress$CachedAddresses.get(Unknown Source) ~[na:na]
at java.base/java.net.InetAddress.getAllByName0(Unknown Source) ~[na:na]
at java.base/java.net.InetAddress.getAllByName(Unknown Source) ~[na:na]
at java.base/java.net.InetAddress.getAllByName(Unknown Source) ~[na:na]
at com.amazonaws.SystemDefaultDnsResolver.resolve(SystemDefaultDnsResolver.java:27) ~[aws-java-sdk-core-1.11.951.jar:na]
at com.amazonaws.http.DelegatingDnsResolver.resolve(DelegatingDnsResolver.java:38) ~[aws-java-sdk-core-1.11.951.jar:na]
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:112) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) ~[httpclient-4.5.13.jar:4.5.13]
at jdk.internal.reflect.GeneratedMethodAccessor161.invoke(Unknown Source) ~[na:na]
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:na]
at java.base/java.lang.reflect.Method.invoke(Unknown Source) ~[na:na]
at com.amazonaws.http.conn.ClientConnectionManagerFactory$Handler.invoke(ClientConnectionManagerFactory.java:76) ~[aws-java-sdk-core-1.11.951.jar:na]
at com.amazonaws.http.conn.$Proxy161.connect(Unknown Source) ~[na:na]
`

SQLLite table not created

Hi there,

I'm facing issues launching the VVP Community Edition following this installation page https://docs.ververica.com/getting_started/installation.html#kubernetes-namespaces

in the appmanager container within the vvp pod its facing issues with creating the table/database (local)

SQL: SELECT name, collection, value FROM appmanager WHERE collection = ?
vvp-ververica-platform-7cf8f467cb-l9kh2 appmanager ### Cause: org.sqlite.SQLiteException: [SQLITE_ERROR] SQL error or missing database (no such table: appmanager)
vvp-ververica-platform-7cf8f467cb-l9kh2 appmanager ; uncategorized SQLException; SQL state [null]; error code [1]; [SQLITE_ERROR] SQL error or missing database (no such table: appmanager); nested exception is org.sqlite.SQLiteException: [SQLITE_ERROR] SQL error or missing database (no such table: appmanager)
vvp-ververica-platform-7cf8f467cb-l9kh2 appmanager 	at org.mybatis.spring.MyBatisExceptionTranslator.translateExceptionIfPossible(MyBatisExceptionTranslator.java:92) ~[mybatis-spring-2.0.6.jar:2.0.6]
vvp-ververica-platform-7cf8f467cb-l9kh2 appmanager 	at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:441) ~[mybatis-spring-2.0.6.jar:2.0.6]
vvp-ververica-platform-7cf8f467cb-l9kh2 appmanager 	at com.sun.proxy.$Proxy100.selectCursor(Unknown Source) ~[na:na]
vvp-ververica-platform-7cf8f467cb-l9kh2 appmanager 	at org.mybatis.spring.SqlSessionTemplate.selectCursor(SqlSessionTemplate.java:200) ~[mybatis-spring-2.0.6.jar:2.0.6]
vvp-ververica-platform-7cf8f467cb-l9kh2 appmanager 	at org.apache.ibatis.binding.MapperMethod.executeForCursor(MapperMethod.java:167) ~[mybatis-3.5.6.jar:3.5.6]
vvp-ververica-platform-7cf8f467cb-l9kh2 appmanager 	at org.apache.ibatis.binding.MapperMethod.execute(MapperMethod.java:84) ~[mybatis-3.5.6.jar:3.5.6]
vvp-ververica-platform-7cf8f467cb-l9kh2 appmanager 	at org.apache.ibatis.binding.MapperProxy$PlainMethodInvoker.invoke(MapperProxy.java:152) ~[mybatis-3.5.6.jar:3.5.6]
vvp-ververica-platform-7cf8f467cb-l9kh2 appmanager 	at org.apache.ibatis.binding.MapperProxy.invoke(MapperProxy.java:85) ~[mybatis-3.5.6.jar:3.5.6]
vvp-ververica-platform-7cf8f467cb-l9kh2 appmanager 	at com.sun.proxy.$Proxy106.findAllByCollection(Unknown Source) ~[na:na]

based on that comment #79 (comment) I've also given the gateway and appmanager container more resources (2gi) but It didn't helped.

Whats wrong? Why its not able to create a table/database with sqllite?

Trying it in my local k8s provided by docker-machine (apple m1)

Replace with user-provided value,plz!

spec.containers[].terminationMessagePath Ignore user-provided value
spec.containers[].terminationMessagePolicy Ignore user-provided value
these two field path can Replace with user-provided value or use the default value？

i want mount the /flink/log directory to the pod

Support metrics from the beginning

Currently once a user is essentially done with the tutorial and has a working playground, they are told that in order to support metrics they need to tear it down again and re-run the setup with a --with-metrics.

Either one of these alternatives would make for a smoother experience in my opinion:

1. Just enable it by default. IMO we need to do it like this unless there is a really good reason not to.
2. Mention the flag in the setup step already.

Allow helm name and namespace to be passed into setup.sh

It would be helpful to allow the helm --name and --namespace values from setup.sh to be configured with arguments to the script for installing it in a different location.

Ververica platform cannot start locally

I am trying to run ververica locally, using the community edition. I tried both with a license and without one. I see that the trial hasn't expired yet

but the following exception is printed in the appmanager and the vvp-ververica-platform pod is in a CrashLoopBackOff:

2023-10-03 07:09:58.751 ERROR 1 --- [           main] o.s.boot.SpringApplication               : Application run failed

org.springframework.context.ApplicationContextException: Unable to start web server; nested exception is org.springframework.boot.web.server.WebServerException: Unable to start embedded Tomcat
	at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.onRefresh(ServletWebServerApplicationContext.java:165) ~[spring-boot-2.7.16.jar:2.7.16]
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:577) ~[spring-context-5.3.30.jar:5.3.30]
	at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:147) ~[spring-boot-2.7.16.jar:2.7.16]
	at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:731) ~[spring-boot-2.7.16.jar:2.7.16]
	at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:408) ~[spring-boot-2.7.16.jar:2.7.16]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:307) ~[spring-boot-2.7.16.jar:2.7.16]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1303) ~[spring-boot-2.7.16.jar:2.7.16]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1292) ~[spring-boot-2.7.16.jar:2.7.16]
	at com.dataartisans.appmanager.apiserver.AppManagerApplication.main(AppManagerApplication.java:10) ~[app/:na]
Caused by: org.springframework.boot.web.server.WebServerException: Unable to start embedded Tomcat
	at org.springframework.boot.web.embedded.tomcat.TomcatWebServer.initialize(TomcatWebServer.java:142) ~[spring-boot-2.7.16.jar:2.7.16]
	at org.springframework.boot.web.embedded.tomcat.TomcatWebServer.<init>(TomcatWebServer.java:104) ~[spring-boot-2.7.16.jar:2.7.16]
	at org.springframework.boot.web.embedded.tomcat.TomcatServletWebServerFactory.getTomcatWebServer(TomcatServletWebServerFactory.java:481) ~[spring-boot-2.7.16.jar:2.7.16]
	at org.springframework.boot.web.embedded.tomcat.TomcatServletWebServerFactory.getWebServer(TomcatServletWebServerFactory.java:211) ~[spring-boot-2.7.16.jar:2.7.16]
	at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.createWebServer(ServletWebServerApplicationContext.java:184) ~[spring-boot-2.7.16.jar:2.7.16]
	at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.onRefresh(ServletWebServerApplicationContext.java:162) ~[spring-boot-2.7.16.jar:2.7.16]
	... 8 common frames omitted
Caused by: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'proxyConfig' defined in file [/vvp/app/com/dataartisans/appmanager/apiserver/config/ProxyConfig.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'sslKeyService' defined in class path resource [com/dataartisans/appmanager/controller/core/modules/ServicesModule.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.dataartisans.appmanager.controller.api.services.SslKeyService]: Factory method 'sslKeyService' threw exception; nested exception is java.lang.NoSuchFieldError: id_alg_zlibCompress
	at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:801) ~[spring-beans-5.3.30.jar:5.3.30]
	at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:224) ~[spring-beans-5.3.30.jar:5.3.30]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1372) ~[spring-beans-5.3.30.jar:5.3.30]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1222) ~[spring-beans-5.3.30.jar:5.3.30]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582) ~[spring-beans-5.3.30.jar:5.3.30]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542) ~[spring-beans-5.3.30.jar:5.3.30]
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335) ~[spring-beans-5.3.30.jar:5.3.30]
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[spring-beans-5.3.30.jar:5.3.30]
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333) ~[spring-beans-5.3.30.jar:5.3.30]
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208) ~[spring-beans-5.3.30.jar:5.3.30]
	at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:405) ~[spring-beans-5.3.30.jar:5.3.30]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1352) ~[spring-beans-5.3.30.jar:5.3.30]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1195) ~[spring-beans-5.3.30.jar:5.3.30]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582) ~[spring-beans-5.3.30.jar:5.3.30]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542) ~[spring-beans-5.3.30.jar:5.3.30]
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335) ~[spring-beans-5.3.30.jar:5.3.30]
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[spring-beans-5.3.30.jar:5.3.30]
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333) ~[spring-beans-5.3.30.jar:5.3.30]
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:213) ~[spring-beans-5.3.30.jar:5.3.30]
	at org.springframework.boot.web.servlet.ServletContextInitializerBeans.getOrderedBeansOfType(ServletContextInitializerBeans.java:213) ~[spring-boot-2.7.16.jar:2.7.16]
	at org.springframework.boot.web.servlet.ServletContextInitializerBeans.getOrderedBeansOfType(ServletContextInitializerBeans.java:204) ~[spring-boot-2.7.16.jar:2.7.16]
	at org.springframework.boot.web.servlet.ServletContextInitializerBeans.addServletContextInitializerBeans(ServletContextInitializerBeans.java:98) ~[spring-boot-2.7.16.jar:2.7.16]
	at org.springframework.boot.web.servlet.ServletContextInitializerBeans.<init>(ServletContextInitializerBeans.java:86) ~[spring-boot-2.7.16.jar:2.7.16]
	at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.getServletContextInitializerBeans(ServletWebServerApplicationContext.java:262) ~[spring-boot-2.7.16.jar:2.7.16]
	at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.selfInitialize(ServletWebServerApplicationContext.java:236) ~[spring-boot-2.7.16.jar:2.7.16]
	at org.springframework.boot.web.embedded.tomcat.TomcatStarter.onStartup(TomcatStarter.java:53) ~[spring-boot-2.7.16.jar:2.7.16]
	at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:4904) ~[tomcat-embed-core-9.0.80.jar:9.0.80]
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:171) ~[tomcat-embed-core-9.0.80.jar:9.0.80]
	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1332) ~[tomcat-embed-core-9.0.80.jar:9.0.80]
	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1322) ~[tomcat-embed-core-9.0.80.jar:9.0.80]
	at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ~[na:na]
	at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75) ~[tomcat-embed-core-9.0.80.jar:9.0.80]
	at java.base/java.util.concurrent.AbstractExecutorService.submit(Unknown Source) ~[na:na]
	at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:866) ~[tomcat-embed-core-9.0.80.jar:9.0.80]
	at org.apache.catalina.core.StandardHost.startInternal(StandardHost.java:795) ~[tomcat-embed-core-9.0.80.jar:9.0.80]
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:171) ~[tomcat-embed-core-9.0.80.jar:9.0.80]
	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1332) ~[tomcat-embed-core-9.0.80.jar:9.0.80]
	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1322) ~[tomcat-embed-core-9.0.80.jar:9.0.80]
	at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ~[na:na]
	at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75) ~[tomcat-embed-core-9.0.80.jar:9.0.80]
	at java.base/java.util.concurrent.AbstractExecutorService.submit(Unknown Source) ~[na:na]
	at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:866) ~[tomcat-embed-core-9.0.80.jar:9.0.80]
	at org.apache.catalina.core.StandardEngine.startInternal(StandardEngine.java:249) ~[tomcat-embed-core-9.0.80.jar:9.0.80]
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:171) ~[tomcat-embed-core-9.0.80.jar:9.0.80]
	at org.apache.catalina.core.StandardService.startInternal(StandardService.java:433) ~[tomcat-embed-core-9.0.80.jar:9.0.80]
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:171) ~[tomcat-embed-core-9.0.80.jar:9.0.80]
	at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:922) ~[tomcat-embed-core-9.0.80.jar:9.0.80]
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:171) ~[tomcat-embed-core-9.0.80.jar:9.0.80]
	at org.apache.catalina.startup.Tomcat.start(Tomcat.java:486) ~[tomcat-embed-core-9.0.80.jar:9.0.80]
	at org.springframework.boot.web.embedded.tomcat.TomcatWebServer.initialize(TomcatWebServer.java:123) ~[spring-boot-2.7.16.jar:2.7.16]
	... 13 common frames omitted
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'sslKeyService' defined in class path resource [com/dataartisans/appmanager/controller/core/modules/ServicesModule.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.dataartisans.appmanager.controller.api.services.SslKeyService]: Factory method 'sslKeyService' threw exception; nested exception is java.lang.NoSuchFieldError: id_alg_zlibCompress
	at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:653) ~[spring-beans-5.3.30.jar:5.3.30]
	at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:633) ~[spring-beans-5.3.30.jar:5.3.30]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1352) ~[spring-beans-5.3.30.jar:5.3.30]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1195) ~[spring-beans-5.3.30.jar:5.3.30]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582) ~[spring-beans-5.3.30.jar:5.3.30]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542) ~[spring-beans-5.3.30.jar:5.3.30]
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335) ~[spring-beans-5.3.30.jar:5.3.30]
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[spring-beans-5.3.30.jar:5.3.30]
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333) ~[spring-beans-5.3.30.jar:5.3.30]
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208) ~[spring-beans-5.3.30.jar:5.3.30]
	at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276) ~[spring-beans-5.3.30.jar:5.3.30]
	at org.springframework.beans.factory.support.DefaultListableBeanFactory$2.resolveCandidate(DefaultListableBeanFactory.java:1849) ~[spring-beans-5.3.30.jar:5.3.30]
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1391) ~[spring-beans-5.3.30.jar:5.3.30]
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.createOptionalDependency(DefaultListableBeanFactory.java:1852) ~[spring-beans-5.3.30.jar:5.3.30]
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1298) ~[spring-beans-5.3.30.jar:5.3.30]
	at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:911) ~[spring-beans-5.3.30.jar:5.3.30]
	at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:788) ~[spring-beans-5.3.30.jar:5.3.30]
	... 62 common frames omitted
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.dataartisans.appmanager.controller.api.services.SslKeyService]: Factory method 'sslKeyService' threw exception; nested exception is java.lang.NoSuchFieldError: id_alg_zlibCompress
	at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:185) ~[spring-beans-5.3.30.jar:5.3.30]
	at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:648) ~[spring-beans-5.3.30.jar:5.3.30]
	... 78 common frames omitted
Caused by: java.lang.NoSuchFieldError: id_alg_zlibCompress
	at org.bouncycastle.asn1.cms.CMSObjectIdentifiers.<clinit>(Unknown Source) ~[bcutil-jdk18on-1.74.jar:1.74.0.0]
	at org.bouncycastle.operator.DefaultSignatureNameFinder.<clinit>(Unknown Source) ~[bcpkix-jdk15on-1.70.jar:1.70.00.0]
	at org.bouncycastle.operator.jcajce.OperatorHelper.<clinit>(Unknown Source) ~[bcpkix-jdk15on-1.70.jar:1.70.00.0]
	at org.bouncycastle.operator.jcajce.JcaContentSignerBuilder.<init>(Unknown Source) ~[bcpkix-jdk15on-1.70.jar:1.70.00.0]
	at com.dataartisans.appmanager.controller.core.ssl.util.SslKeyFactoryBouncyCastle.buildCertificate(SslKeyFactoryBouncyCastle.java:143) ~[appmanager-controller-2.11.2-plain.jar:na]
	at com.dataartisans.appmanager.controller.core.ssl.util.SslKeyFactoryBouncyCastle.createRootCertificate(SslKeyFactoryBouncyCastle.java:105) ~[appmanager-controller-2.11.2-plain.jar:na]
	at com.dataartisans.appmanager.controller.core.ssl.KeyCertificatePair.generateRootCertificate(KeyCertificatePair.java:45) ~[appmanager-controller-2.11.2-plain.jar:na]
	at com.dataartisans.appmanager.controller.core.ssl.SslKeyHolder.generateRootCertAndSave(SslKeyHolder.java:251) ~[appmanager-controller-2.11.2-plain.jar:na]
	at com.dataartisans.appmanager.controller.core.ssl.SslKeyHolder.createKeyStore(SslKeyHolder.java:235) ~[appmanager-controller-2.11.2-plain.jar:na]
	at com.dataartisans.appmanager.controller.core.ssl.SslKeyHolder.lambda$createAndMigrate$1(SslKeyHolder.java:134) ~[appmanager-controller-2.11.2-plain.jar:na]
	at java.base/java.util.Optional.orElseGet(Unknown Source) ~[na:na]
	at com.dataartisans.appmanager.controller.core.ssl.SslKeyHolder.createAndMigrate(SslKeyHolder.java:122) ~[appmanager-controller-2.11.2-plain.jar:na]
	at com.dataartisans.appmanager.controller.core.ssl.SslKeyServiceFacade.<init>(SslKeyServiceFacade.java:43) ~[appmanager-controller-2.11.2-plain.jar:na]
	at com.dataartisans.appmanager.controller.core.modules.ServicesModule.sslKeyService(ServicesModule.java:32) ~[appmanager-controller-2.11.2-plain.jar:na]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:na]
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:na]
	at java.base/java.lang.reflect.Method.invoke(Unknown Source) ~[na:na]
	at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154) ~[spring-beans-5.3.30.jar:5.3.30]
	... 79 common frames omitted

I think vvp couldn't access from local area network by default, how can I resolve it?

I follow the document step-by-step( minikube helm vvp etc. ), https://docs.ververica.com/getting_started/installation.html#setting-up-the-playground
then I start vvp, and it's work. like...

[hadoop@hosts101-230 ververica-platform-playground]$ kubectl port-forward service/vvp-ververica-platform 8080:80

Forwarding from 127.0.0.1:8080 -> 8080

Forwarding from [::1]:8080 -> 8080

Handling connection for 8080

the port status show:

but it couldn't visit from local area network. I try to curl command locally

# it's work
curl http://localhost:8080/api/v1/namespaces/defaults/deployments -H "Accept: application/yaml"

# doesn't work
curl http://192.168.101.230:8080/api/v1/namespaces/defaults/deployments -H "Accept: application/yaml"

how can I resolve it? thx

Remote RDBMS problem

You have configured value-vvp.yaml as follows, but a strange problem still occurs

Website configuration ：

vvp:
  persistence:
    type: jdbc
    datasource:
      url: jdbc:mariadb://mysql.internal:3306/vvp
      username: vvp
      password: password

Configuration used：

vvp:
  persistence:
    type: jdbc
    datasource:
      url: jdbc:mariadb://vvp-mariadb.vvp.svc.cluster.local:3306/vvp
      username: vvp
      password: password

error log：

Error: template: ververica-platform/templates/pvc.yaml:1:11: executing "ververica-platform/templates/pvc.yaml" at <include "vvp.should.create.pvc" .>: error calling include: template: ververica-platform/templates/_helpers.tpl:75:15: executing "vvp.should.create.pvc" at <include "vvp.pvc.enabled" .>: error calling include: template: ververica-platform/templates/_helpers.tpl:67:12: executing "vvp.pvc.enabled" at <include "vvp.private.process.application.config" .>: error calling include: template: ververica-platform/templates/_helpers.tpl:139:8: executing "vvp.private.process.application.config" at <include "vvp.private.inject.datasource.config" .>: error calling include: template: ververica-platform/templates/_helpers.tpl:266:99: executing "vvp.private.inject.datasource.config" at : error calling fail: vvp.persistence.datasource.url may not be set when using 'local' persistence


How to work with kerberos with ververica hosted flink image ?

How can we do something like this : security.kerberos.login.keytab: 's3://vvp/artifacts/namespaces/default/coremojo.keytab'

Flink version number ill defined in 1.9 image

Hello,

I'm trying to use the platform with a pipeline written in beam, so I'm using the image 1.9.2-stream2-scala_2.12 because flink 1.10 is not yet compatible.

When I launch a job I can see in the logs of the task-manager pod:

INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner      [] -  Starting TaskManager (Version: <unknown>, Rev:4d56de8, Date:30.09.2019 @11:32:19 CST)

In the logs of the job-manager pod:

INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] -  Starting StandaloneJobClusterEntryPoint (Version: <unknown>, Rev:4d56de8, Date:30.09.2019 @ 11:32:19 CST)

Notice the Version: <unkown>

I then have a ververica platform event:

Invalid Flink cluster version: (expected: 1.9)
>>> Invalid Flink cluster version: <unknown> (expected: 1.9)

And the job seems to be killed at this moment, next event, at the same second is

Terminating a job due to an unrecoverable failure

When I try with the 1.10.0-stream1-scala_2.12 image, I can see the version number correctly, (but I can't use it with Beam):

INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] -  Starting StandaloneJobClusterEntryPoint (Version: 1.10.0-stream1, Rev:788c5da, Date:05.03.2020 @ 11:44:53 UTC)

Is there any way to fix this ? No problem if I have to build a custom image, but I'm not sure how I would be able to set such version number.

Also: I'm not sure if I should open this issue in this project, let me know if there is a better place for it.

Don't create cluster-wide resources if possible

The Prometheus and Grafana Helm charts both create some cluster-level resources, causing conflicts if they are installed multiple times in the same cluster.

We might be able to improve that here by tweaking their values files and possibly creating some resources manually.

Namely, these are the ones I've found:

• clusterroles/prometheus-server
• clusterrolebindings/prometheus-server
• clusterroles/grafana-clusterrrole
• clusterrolebindings/grafana-clusterrrolebinding
• podsecuritypolicies/grafana
• podsecuritypolicies/grafana-test

For Prometheus we should be able to set rbac.create=false, but I think we'll need to create a rolebinding ourselves so it can discover pods in the same namespace. (create rolebinding --clusterrole admin --serviceaccount prometheus-server should do it).

For Grafana, rbac.pspEnabled=false and rbac.namespaced=true might get us where we want.

No UI after installation on 8080

After installing vvp as per the quickstart, no UI is showing on 8080

I did a port-forwarding as per the instructions

kubectl --namespace vvp port-forward services/vvp-ververica-platform 8080:80

The browser is redirected to /ui but only finds a 404 there.

Since this is such an essential part I wonder: Is VVP maintained at all?

Thanks for a quick clarification!

How to drain a job

Hello, great project thanks for sharing ! My company currently uses GCP dataflow, we're considering migrating to another platform. Demo in local is working fine. I'm very new to flink so I may have missed some basic stuff.
I see it is possible in flink to drain with flink stop --drain.

Is it be possible to perform a pipeline "drain" operation in the ververica system?

I see that there is the state CANCELED for a Deployment but it's not clear to me if it would stop with or without draining.

For flink job, I see the upgrade strategy can be stateless or stateful.
In my case, I would like to have a flink job being stateful, for instance to easily update the parallelism.
Then when doing a change that will make the new job not compatible with the current checkpoints, I would like to drain the current running flink job, and perform a new deployment.

Can't reach helm repo

helm repo add ververica https://charts.ververica.com
Error: looks like "https://charts.ververica.com" is not a valid chart repository or cannot be reached: Get "https://charts.ververica.com/index.yaml": tls: failed to verify certificate: x509: “Amazon” certificate is revoked

Set URL for the guide to the repository

On GitHub repositories you can set a URL which will be shown next to the description here:

I think we should provide the URL to the getting started guide. This is a more accessible place than the README (where it should also be linked, of course) as no scrolling down or parsing text is required and users just sort of expect the most relevant link to be there.

Is there a way to install a specific vesrion of flink? eg. 1.1.4?

Reading https://docs.ververica.com/installation/helm/index.html, how do you install a specific version of flink?

Pass service account to taskmanager and jobmanager pods

Hello, I (and @sysC0D) have a hard time understanding how we are supposed to
pass service accounts to taskmanager and jobmanager pods.

We defined a service account flink-project1, which is supposed to be used for pods in our project1 node-pool, where vvp-jobs pods are run.

$ kubectl -n vvp-jobs describe serviceaccounts flink-project1
Name:                flink-project1
Namespace:           vvp-jobs
Labels:              <none>
Annotations:         iam.gke.io/gcp-service-account: [email protected]
Image pull secrets:  <none>
Mountable secrets:   flink-project1-token-jjkrt
Tokens:              flink-project1-token-jjkrt
Events:              <none>

But it looks like when pods are launched, service accounts are generated for them:
for instance for task manager we can see such service account being generated and used :

$ kubectl -n vvp-jobs describe serviceaccount job-0fbe6901-2295-4bd0-8067-1b1c90906041-flink-ha-taskmanager
Name:                job-0fbe6901-2295-4bd0-8067-1b1c90906041-flink-ha-taskmanager
Namespace:           vvp-jobs
Labels:              app=flink-job
                     deploymentId=468922bc-bc27-4c15-b1dd-5ad143547fcb
                     jobId=0fbe6901-2295-4bd0-8067-1b1c90906041
                     system=ververica-platform
Annotations:         <none>
Image pull secrets:  <none>
Mountable secrets:   job-0fbe6901-2295-4bd0-8067-1b1c90906041-flink-ha-taskmana5ktb4
Tokens:              job-0fbe6901-2295-4bd0-8067-1b1c90906041-flink-ha-taskmana5ktb4
Events:              <none>

We can see it the the taskmanager deployment yaml config:

apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "1"
  creationTimestamp: "2020-05-04T15:56:25Z"
  generation: 1
  labels:
    app: flink-job
    component: taskmanager
    deploymentId: 4364f18c-a074-4c4c-9cce-31a219169a48
    jobId: c3961d20-133d-4716-aad5-632a652ee41a
    system: ververica-platform
  name: job-c3961d20-133d-4716-aad5-632a652ee41a-taskmanager
  namespace: vvp-jobs
spec:
  template:
    metadata:
      annotations:
        kubernetes.io/service-account.name: flink-diddykong
    spec:
      dnsPolicy: ClusterFirst
      initContainers:
      - args:
        - some_args
      nodeSelector:
        nodepool-role: project1
      serviceAccount: job-c3961d20-133d-4716-aad5-632a652ee41a-flink-ha-taskmanager
      serviceAccountName: job-c3961d20-133d-4716-aad5-632a652ee41a-flink-ha-taskmanager
status:
  availableReplicas: 1

(yaml is not complete)
Notice serviceAccount and serviceAccountName.
(service accounts don't have the same names in my kubectl command and in the yaml because they do not come from the same deployment but you get the idea)

We can see the same in the jobmanager Job yaml config:

apiVersion: batch/v1
kind: Job
metadata:
  creationTimestamp: "2020-05-04T15:56:25Z"
  labels:
    app: flink-job
    component: jobmanager
    deploymentId: 4364f18c-a074-4c4c-9cce-31a219169a48
    jobId: c3961d20-133d-4716-aad5-632a652ee41a
    system: ververica-platform
  name: job-c3961d20-133d-4716-aad5-632a652ee41a-jobmanager
  namespace: vvp-jobs
spec:
  template:
    metadata:
      annotations:
        kubernetes.io/service-account.name: flink-project1
      creationTimestamp: null
      labels:
        app: flink-job
    spec:
      containers:
      - args:
        - some_args
      dnsPolicy: ClusterFirst
      nodeSelector:
        nodepool-role: project1
      serviceAccount: job-c3961d20-133d-4716-aad5-632a652ee41a-flink-ha-jobmanager
      serviceAccountName: job-c3961d20-133d-4716-aad5-632a652ee41a-flink-ha-jobmanager
      terminationGracePeriodSeconds: 30
status:
  active: 1
  startTime: "2020-05-04T15:56:25Z"

Would there be a way to pass our flink-project1 service account instead of the generated service accounts ?