open-datastudio/zeppelin

Zeppelin notebook online

zeppelin's Issues

how can I import module/function from another notebook/.py file?

I mean if Zeppelin is set up on cloud by another team and I cannot touch the Zeppelin source code.
For OOD, we need to chop code to multiple module/library. how can I import function/module from another Zeppelin notebook/.py file? or or we need to lay out the code so it's top-down and self-inclusive in only one Zeppelin notebook and then copy/paste to multiple .py files?

get runtime exception when running neural network model

I tried to run word2vec (a popular neural network model in NLP), and got the runtime exception error. see below for more detail:

Can't cd to /zeppelin

skaffold dev -f .staroid/skaffold.yaml -p prebuilt,minikube --port-forward
Listing files to watch...

• zeppelin-server
• zeppelin-interpreter
  Generating tags...
• zeppelin-server -> zeppelin-server:b9cc017ae
• zeppelin-interpreter -> zeppelin-interpreter:b9cc017ae
  Checking cache...
• zeppelin-server: Not found. Building
• zeppelin-interpreter: Not found. Building
  Starting build...
  Found [minikube] context, using local docker daemon.
  Building 2 artifacts in parallel
  Building [zeppelin-interpreter]...
  Sending build context to Docker daemon 5.12kB
  Step 1/18 : FROM apache/zeppelin:0.9.0 AS builder
  0.9.0: Pulling from apache/zeppelin
  83ee3a23efb7: Pulling fs layer
  db98fc6f11f0: Pulling fs layer
  f611acd52c6c: Pulling fs layer
  c68f056c1360: Pulling fs layer
  d2893470c61e: Pulling fs layer
  f8cf2bd9216a: Pulling fs layer
  78e3dee51c69: Pulling fs layer
  8bfdc346b9be: Pulling fs layer
  d760722ddd93: Pulling fs layer
  c68f056c1360: Waiting
  d2893470c61e: Waiting
  f8cf2bd9216a: Waiting
  78e3dee51c69: Waiting
  8bfdc346b9be: Waiting
  d760722ddd93: Waiting
  f611acd52c6c: Download complete
  db98fc6f11f0: Verifying Checksum
  db98fc6f11f0: Download complete
  83ee3a23efb7: Verifying Checksum
  83ee3a23efb7: Download complete
  d2893470c61e: Verifying Checksum
  d2893470c61e: Download complete
  c68f056c1360: Verifying Checksum
  c68f056c1360: Download complete
  83ee3a23efb7: Pull complete
  8bfdc346b9be: Verifying Checksum
  8bfdc346b9be: Download complete
  d760722ddd93: Verifying Checksum
  d760722ddd93: Download complete
  db98fc6f11f0: Pull complete
  f611acd52c6c: Pull complete
  f8cf2bd9216a: Verifying Checksum
  f8cf2bd9216a: Download complete
  78e3dee51c69: Verifying Checksum
  78e3dee51c69: Download complete
  c68f056c1360: Pull complete
  d2893470c61e: Pull complete
  f8cf2bd9216a: Pull complete
  78e3dee51c69: Pull complete
  8bfdc346b9be: Pull complete
  d760722ddd93: Pull complete
  Digest: sha256:20ad134275d061a27bc8888ab6d7d44534f4c751bdc0fb8a0df2111437e57d1d
  Status: Downloaded newer image for apache/zeppelin:0.9.0
  ---> 1672c2a624fb
  Step 2/18 : FROM ubuntu:18.04
  18.04: Pulling from library/ubuntu
  4bbfd2c87b75: Pulling fs layer
  d2e110be24e1: Pulling fs layer
  889a7173dcfe: Pulling fs layer
  889a7173dcfe: Verifying Checksum
  889a7173dcfe: Download complete
  d2e110be24e1: Verifying Checksum
  d2e110be24e1: Download complete
  4bbfd2c87b75: Verifying Checksum
  4bbfd2c87b75: Download complete
  4bbfd2c87b75: Pull complete
  d2e110be24e1: Pull complete
  889a7173dcfe: Pull complete
  Digest: sha256:67b730ece0d34429b455c08124ffd444f021b81e06fa2d9cd0adaf0d0b875182
  Status: Downloaded newer image for ubuntu:18.04
  ---> 81bcf752ac3d
  Step 3/18 : RUN apt-get -y update && apt-get install -y openjdk-8-jre-headless python3-minimal curl python3-distutils && ln -s /usr/bin/python3 /usr/bin/python && curl https://bootstrap.pypa.io/get-pip.py | python && apt-get clean
  ---> Running in cd5a528aefc2
  Building [zeppelin-server]...
  Sending build context to Docker daemon 37.38kB
  Step 1/21 : FROM apache/zeppelin:0.9.0 AS builder
  0.9.0: Pulling from apache/zeppelin
  83ee3a23efb7: Pulling fs layer
  db98fc6f11f0: Pulling fs layer
  f611acd52c6c: Pulling fs layer
  c68f056c1360: Pulling fs layer
  d2893470c61e: Pulling fs layer
  f8cf2bd9216a: Pulling fs layer
  78e3dee51c69: Pulling fs layer
  8bfdc346b9be: Pulling fs layer
  d760722ddd93: Pulling fs layer
  d760722ddd93: Waiting
  f8cf2bd9216a: Waiting
  78e3dee51c69: Waiting
  c68f056c1360: Waiting
  d2893470c61e: Waiting
  8bfdc346b9be: Waiting
  f611acd52c6c: Verifying Checksum
  f611acd52c6c: Download complete
  db98fc6f11f0: Verifying Checksum
  db98fc6f11f0: Download complete
  83ee3a23efb7: Verifying Checksum
  83ee3a23efb7: Download complete
  d2893470c61e: Verifying Checksum
  d2893470c61e: Download complete
  c68f056c1360: Verifying Checksum
  c68f056c1360: Download complete
  83ee3a23efb7: Pull complete
  8bfdc346b9be: Verifying Checksum
  8bfdc346b9be: Download complete
  d760722ddd93: Verifying Checksum
  d760722ddd93: Download complete
  db98fc6f11f0: Pull complete
  f611acd52c6c: Pull complete
  f8cf2bd9216a: Verifying Checksum
  f8cf2bd9216a: Download complete
  78e3dee51c69: Verifying Checksum
  78e3dee51c69: Download complete
  c68f056c1360: Pull complete
  d2893470c61e: Pull complete
  f8cf2bd9216a: Pull complete
  78e3dee51c69: Pull complete
  8bfdc346b9be: Pull complete
  d760722ddd93: Pull complete
  Digest: sha256:20ad134275d061a27bc8888ab6d7d44534f4c751bdc0fb8a0df2111437e57d1d
  Status: Downloaded newer image for apache/zeppelin:0.9.0
  ---> 1672c2a624fb
  Step 2/21 : ENV Z_VERSION="0.9.0-preview1" Z_HOME="/zeppelin"
  ---> Running in 37f955b15796
  ---> dfd0431168da
  Step 3/21 : RUN cd $Z_HOME && mkdir -p zeppelin-web/dist && mkdir -p zeppelin-web-angular/dist && cd zeppelin-web/dist && jar xf ../../zeppelin-web-$Z_VERSION.war && rm -f ../../zeppelin-web-$Z_VERSION.war && cd ../../zeppelin-web-angular/dist && jar xf ../../zeppelin-web-angular-$Z_VERSION.war && rm -f ../../zeppelin-web-angular-$Z_VERSION.war && cd ../..
  ---> Running in 293282d36c6d
  /bin/sh: 1: cd: can't cd to /zeppelin

unable to stream build output: The command '/bin/sh -c cd $Z_HOME && mkdir -p zeppelin-web/dist && mkdir -p zeppelin-web-angular/dist && cd zeppelin-web/dist && jar xf ../../zeppelin-web-$Z_VERSION.war && rm -f ../../zeppelin-web-$Z_VERSION.war && cd ../../zeppelin-web-angular/dist && jar xf ../../zeppelin-web-angular-$Z_VERSION.war && rm -f ../../zeppelin-web-angular-$Z_VERSION.war && cd ../..' returned a non-zero code: 2. Please fix the Dockerfile and try again..

In Spark tutorial 3. Spark SQL (PySpark), the "people.json" not exist

I'm trying the Spark tutorial 3. Spark SQL (PySpark), in the "Create DataFrame via DataFrameReader" paragraph, I got the "people.json" not exist issue

No spark interpreter, k8s deployment failing

Hi,

After cloning this repo - I rebuild locally the images via

skaffold dev -f .staroid/skaffold.yaml
then tag, and deployed in my own k8s cluster using the .staroid k8s template - after few trick in both the docker image and k8s template.

But no spark interpreter is deployed, and all the other interpreter like md - i end up with the following error:

org.apache.zeppelin.interpreter.InterpreterException: java.io.IOException: Fail to instantiate Launcher K8sStandardInterpreterLauncher from plugin pluginDir: /zeppelin/plugins at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.open(RemoteInterpreter.java:129) at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:271) at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:444) at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:72) at org.apache.zeppelin.scheduler.Job.run(Job.java:172) at org.apache.zeppelin.scheduler.AbstractScheduler.runJob(AbstractScheduler.java:132)

am i missing something to get zeppelin working ?

Can I upload and store data within the cluster, and read it from python?

I just started trying Zeppelin here. I don't want to store data in S3 or my remote repository. Is it possible to upload the data files and store them within the cluster?

pyspark is not working

Pyspark is not working with the following error

org.apache.zeppelin.interpreter.InterpreterException: org.apache.zeppelin.interpreter.InterpreterException: Fail to open PythonInterpreter
	at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:76)
	at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:668)
	at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:577)
	at org.apache.zeppelin.scheduler.Job.run(Job.java:172)
	at org.apache.zeppelin.scheduler.AbstractScheduler.runJob(AbstractScheduler.java:130)
	at org.apache.zeppelin.scheduler.FIFOScheduler.lambda$runJobInScheduler$0(FIFOScheduler.java:39)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.zeppelin.interpreter.InterpreterException: Fail to open PythonInterpreter
	at org.apache.zeppelin.python.PythonInterpreter.open(PythonInterpreter.java:115)
	at org.apache.zeppelin.spark.PySparkInterpreter.open(PySparkInterpreter.java:93)
	at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:70)
	... 8 more
Caused by: java.io.IOException: Fail to launch python process.
Traceback (most recent call last):
  File "/tmp/1597290814731-0/zeppelin_python.py", line 20, in <module>
    from py4j.java_gateway import java_import, JavaGateway, GatewayClient
ModuleNotFoundError: No module named 'py4j'

	at org.apache.zeppelin.python.PythonInterpreter.createGatewayServerAndStartScript(PythonInterpreter.java:160)
	at org.apache.zeppelin.python.PythonInterpreter.open(PythonInterpreter.java:112)
	... 10 more

Is it possible to run open-datastudio Zeppelin on your own K8S cluster?

From the documentation, I see that Zeppelin can be run on minikube. Is it also possible to run it on your own cluster?