neil-rubens/docker-spark

fork of sequenceiq/docker-spark modified to work with scala 2.11

docker-spark's Issues

Build Distributable Spark version

make-distribution.sh allows you to build a distributable/runnable image (similar to the binary images).

Doing this would allow spark to have the same layout; e.g. currently jars are in target folder instead of lib

UnSolved Issues

Current attempt to achieve this; has been unsuccessful. The script gets stuck in the beginning with no messages.

Solved Issues

providing input to the script

A part of the script requires input; couldn't quite figure out how to do it through dockerfile; so instead could simply delete the lines from the script (via dockerfile):

# UPGRADE: when upgrading spark need to double-check the lines that are being deleted bellow
# delete lines that require user input (asking if it is ok to proceed): e.g.:
#  read -p "Would you like to continue anyways? [y,n]: " -r
#  if [[ ! "$REPLY" =~ ^[Yy]$ ]]; then
RUN sed -i.bak -e '153,157d' /usr/local/spark-1.4.0/make-distribution.sh
RUN cat /usr/local/spark-1.4.0/make-distribution.sh