First, setup the Kafka broker:
cd kafka-docker
docker-compose -f docker-compose-expose.yml up
Then, run the Kafka producer:
cd stream/producer
mvn spring-boot:run
Lastly, run the Spark compute pipeline (a Kafka consumer)
cd process/app
sbt run
The Spark UI is at http://localhost:4040
If you run into problems with OutOfMemoryError, try running
export SBT_OPTS="-Xms1024M -Xmx4G -Xss2M -XX:MaxMetaspaceSize=2G"