Comments (3)
hello,
thanks for your contribution
I will check
from bigdata-playground.
Hi, @Chabane
Thanks for quick reply.
I hope you can reproduce
from bigdata-playground.
I have to investigate for 0.14.x
It works with OSX 10.13.6
$ git clone https://github.com/Chabane/bigdata-playground
$ cd bigdata-playground
$ cd streaming/spark && sbt clean assembly && cd ../..
$ cd docker
$ docker-compose -f dev/streaming-spark.yml up -d
Building streaming-spark-dev
Step 1/11 : FROM openjdk:8-jre
8-jre: Pulling from library/openjdk
741437d97401: Pull complete
34d8874714d7: Pull complete
0a108aa26679: Pull complete
eea0f08c7492: Pull complete
ca74b3d90c15: Pull complete
f11a643c4a85: Pull complete
05eafed73b65: Pull complete
Digest: sha256:2216ccda45993afaa42b3b269aa1ae8832cb2fabbf8207f63d528fdc7c8a087b
Status: Downloaded newer image for openjdk:8-jre
---> ed287c436e66
Step 2/11 : VOLUME /tmp
---> Running in 88436c00ffe6
Removing intermediate container 88436c00ffe6
---> 8b4ae4f0795f
Step 3/11 : ENV SPARK_VERSION spark-2.4.0-bin-hadoop2.7
---> Running in 18f69569d784
Removing intermediate container 18f69569d784
---> 43ec961a3688
Step 4/11 : ENV SPARK_HOME /usr/local/spark
---> Running in 3185f165f176
Removing intermediate container 3185f165f176
---> d45e0984fe65
Step 5/11 : RUN curl https://archive.apache.org/dist/spark/spark-2.4.0/$SPARK_VERSION.tgz -o $SPARK_VERSION.tgz; tar xzf $SPARK_VERSION.tgz -C /usr/local/;
---> Running in f8d5c2aeea09
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 217M 100 217M 0 0 9.8M 0 0:00:22 0:00:22 --:--:-- 9401k
Removing intermediate container f8d5c2aeea09
---> 904bc6c8ec6b
Step 6/11 : RUN cd /usr/local && ln -s $SPARK_VERSION spark
---> Running in 1afb5821bc35
Removing intermediate container 1afb5821bc35
---> d4d112977ae5
Step 7/11 : ADD hbase-site.xml $SPARK_HOME/conf
---> 855d1f6f0aaf
Step 8/11 : ADD target/scala-2.11/search-flight-spark-streaming-assembly-0.1.0.jar $SPARK_HOME/libs/search-flight-streaming.jar
---> 7912943b00bd
Step 9/11 : RUN sh -c 'touch $SPARK_HOME/libs/search-flight-streaming.jar'
---> Running in 596e323e7cd5
Removing intermediate container 596e323e7cd5
---> 42bcd548a74e
Step 10/11 : WORKDIR $SPARK_HOME
---> Running in 5e91d58ce3fa
Removing intermediate container 5e91d58ce3fa
---> 09217c527d4e
Step 11/11 : CMD ["bin/spark-submit", "--class", "com.mitosis.Main", "--master", "local[*]", "--files", "/usr/local/spark/conf/hbase-site.xml", "./libs/search-flight-streaming.jar"]
---> Running in 05cea3bcbca9
Removing intermediate container 05cea3bcbca9
---> d16a3d45765a
Successfully built d16a3d45765a
Successfully tagged dev_streaming-spark-dev:latest
WARNING: Image for service streaming-spark-dev was built because it did not already exist. To rebuild this image you must use `docker-compose build` or `docker-compose up --build`.
Creating streaming_spark_dev ... done
from bigdata-playground.
Related Issues (11)
- com.fasterxml.jackson.databind.exc.InvalidFormatException: Can not construct instance of long from String value '2018-06-13': not a valid Long value HOT 1
- Question of framework selection HOT 8
- error TS2307: Cannot find module 'rxjs/rx'. HOT 12
- Dev mode
- How to import data into the the app? HOT 2
- Deployment Challenge
- WebApp seach flights issue
- multi-stage build / simplify setup
- admin.tableExists(tableName)
- Le
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bigdata-playground.