Giter Site home page Giter Site logo

Comments (3)

Chabane avatar Chabane commented on May 24, 2024

hello,
thanks for your contribution
I will check

from bigdata-playground.

staminna avatar staminna commented on May 24, 2024

Hi, @Chabane
Thanks for quick reply.
I hope you can reproduce

from bigdata-playground.

Chabane avatar Chabane commented on May 24, 2024

I have to investigate for 0.14.x

It works with OSX 10.13.6

$ git clone https://github.com/Chabane/bigdata-playground
$ cd bigdata-playground
$ cd streaming/spark && sbt clean assembly && cd ../..
$ cd docker
$ docker-compose -f dev/streaming-spark.yml up -d
Building streaming-spark-dev
Step 1/11 : FROM openjdk:8-jre
8-jre: Pulling from library/openjdk
741437d97401: Pull complete
34d8874714d7: Pull complete
0a108aa26679: Pull complete
eea0f08c7492: Pull complete
ca74b3d90c15: Pull complete
f11a643c4a85: Pull complete
05eafed73b65: Pull complete
Digest: sha256:2216ccda45993afaa42b3b269aa1ae8832cb2fabbf8207f63d528fdc7c8a087b
Status: Downloaded newer image for openjdk:8-jre
 ---> ed287c436e66
Step 2/11 : VOLUME /tmp
 ---> Running in 88436c00ffe6
Removing intermediate container 88436c00ffe6
 ---> 8b4ae4f0795f
Step 3/11 : ENV SPARK_VERSION spark-2.4.0-bin-hadoop2.7
 ---> Running in 18f69569d784
Removing intermediate container 18f69569d784
 ---> 43ec961a3688
Step 4/11 : ENV SPARK_HOME /usr/local/spark
 ---> Running in 3185f165f176
Removing intermediate container 3185f165f176
 ---> d45e0984fe65
Step 5/11 : RUN curl https://archive.apache.org/dist/spark/spark-2.4.0/$SPARK_VERSION.tgz -o $SPARK_VERSION.tgz;     tar xzf $SPARK_VERSION.tgz -C /usr/local/;
 ---> Running in f8d5c2aeea09
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  217M  100  217M    0     0   9.8M      0  0:00:22  0:00:22 --:--:-- 9401k
Removing intermediate container f8d5c2aeea09
 ---> 904bc6c8ec6b
Step 6/11 : RUN cd /usr/local && ln -s $SPARK_VERSION spark
 ---> Running in 1afb5821bc35
Removing intermediate container 1afb5821bc35
 ---> d4d112977ae5
Step 7/11 : ADD hbase-site.xml $SPARK_HOME/conf
 ---> 855d1f6f0aaf
Step 8/11 : ADD target/scala-2.11/search-flight-spark-streaming-assembly-0.1.0.jar $SPARK_HOME/libs/search-flight-streaming.jar
 ---> 7912943b00bd
Step 9/11 : RUN sh -c 'touch $SPARK_HOME/libs/search-flight-streaming.jar'
 ---> Running in 596e323e7cd5
Removing intermediate container 596e323e7cd5
 ---> 42bcd548a74e
Step 10/11 : WORKDIR $SPARK_HOME
 ---> Running in 5e91d58ce3fa
Removing intermediate container 5e91d58ce3fa
 ---> 09217c527d4e
Step 11/11 : CMD ["bin/spark-submit", "--class", "com.mitosis.Main", "--master", "local[*]", "--files", "/usr/local/spark/conf/hbase-site.xml", "./libs/search-flight-streaming.jar"]
 ---> Running in 05cea3bcbca9
Removing intermediate container 05cea3bcbca9
 ---> d16a3d45765a

Successfully built d16a3d45765a
Successfully tagged dev_streaming-spark-dev:latest
WARNING: Image for service streaming-spark-dev was built because it did not already exist. To rebuild this image you must use `docker-compose build` or `docker-compose up --build`.
Creating streaming_spark_dev ... done

from bigdata-playground.

Related Issues (11)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.