Goal of this repository is to provide some few easy examples of demo to run on CML.
It relies on Data generated previously be Datagen.
To generate data, there are two ways:
-
Use the script gen_data.sh to generate data. You will need to change variables at the top of the script to match your environment. WARNING: install jq before running the script, example: yum install jq
-
Do it manually by following below instructions to generate data using swagger (or directly API) of Datagen (a service deployed in your base cluster).
Use Datagen to generate data in hdfs-parquet with following parameters:
-
Model file: weather-model
-
Batches: 10
-
Rows: 100000
-
Threads: 10
Then play weather-model.py file in a CML Session.
Use Datagen to generate data in hdfs-parquet with following parameters:
-
Model file: bank-account-model
-
Batches: 10
-
Rows: 1000000
-
Threads: 10
Then play bank-accounts.py file in a CML Session.