mminella/cloud-native-batch

• Configure the Spring Cloud Config Server with an application.properties or application.yml in batch-job/src/main/resources with the following properties configured:
  • spring.datasource.driverClassName
  • spring.datasource.url
  • spring.datasource.username
  • spring.datasource.password
  • spring.datasource.schema - this should point to the schema-mysql.sql in the same directory
  • job.resource-path - this should be the S3 bucket
  • cloud.aws.credentials.accessKey
  • cloud.aws.credentials.secretKey
  • cloud.aws.region.static - the region the S3 bucket exists in
  • cloud.aws.region.auto - this should be false unless you are running this on AWS
• Launch the Config server & eureka via spring cloud configserver eureka
• Build the project from the root via ./mvnw clean install
• Launch Skipper via:

java -jar bin/spring-cloud-skipper-server-2.0.0.RELEASE.jar \
    --spring.datasource.url=jdbc:mysql://localhost:3306/cloud_native_batch \
    --spring.datasource.username=root \
    --spring.datasource.password=p@ssw0rd \
    --spring.datasource.driver-class-name=org.mariadb.jdbc.Driver

• Launch Spring Cloud Data Flow (SCDF) via:

java -jar bin/spring-cloud-dataflow-server-2.0.1.RELEASE.jar \
    --spring.datasource.url=jdbc:mysql://localhost:3306/cloud_native_batch \
    --spring.datasource.username=root \
    --spring.datasource.password=p@ssw0rd \
    --spring.datasource.driver-class-name=org.mariadb.jdbc.Driver

• Launch Spring Cloud Data Flow Shell via java -jar bin/spring-cloud-dataflow-shell-2.0.1.RELEASE.jar
• Register the app with SCDF in the shell via app register --name S3JDBC --type task --uri "maven://io.spring.cloud-native-batch:batch-job:0.0.1-SNAPSHOT"
• Launch the UI from http://localhost:9393/dashboard
• Launch the task via the UI
• Verify results via the query select * from cloud_native_batch.foo; assuming your schema is called cloud_native_batch

cloud-native-batch's Issues

k8s3 branch

Not seeing the branch k8s3 that was shown in the youtube video. Can you push it up here?

Spring Batch & SCDF - Limitations ?

Michael - I'd like to take opportunity to thank for doing great tutorials and videos. I'd request you to do the same for Spring Batch & SCDF more.

I've small concerns on SCDF & Batch

I learn that SCDF has no support for MongoDB, also the kind of batch jobs that I've is loading data from relational db and loading data to MongoDB, I am not able to monitor batch jobs through SCDF UI and neither see any workaround to save Task enabled Jobs metadata into different schema or something similar.

Could you please guide ion this please ?