monochromegane/banditsflow

🎰 A building workflow and tracking its information framework for bandits.

BanditsFlow: 🎰 A building workflow and tracking its information framework for bandits.

BanditsFlow is a framework that supports the construction of a typical evaluation workflow for comparing bandit algorithms. Your experimental modules on this framework are automatically executed in the Metaflow workflow. In addition, the workflow incorporates experiment management with MLflow Tracking and hyperparameter optimization with Optuna. Combined with code management using Git, you will be able to manage your experiments with high reproducibility.

Usage

1. Generate scaffold and first run your experiment

$ YOUR_BANDIT_FLOW_NAME='sample'
$ python -m banditsflow scaffold $YOUR_BANDIT_FLOW_NAME
$ git add .
$ git commit -m 'Initial commit'
$ git tag first-experiment
$ make run
$ mlflow ui

And access to http://127.0.0.1:5000

2. Customize experiment

1. Implement our scenario.
2. Implement an actor who acts the scenario.
3. Implement a reporter who reports result of actions of the actor.
4. Prepare a parameter suggestion for each actor. (optional)

The scenario, actor and reporter must follow each protocol. See each protocol (scenario.Scenario, actor.Actor and reporter.Reporter).

Note that each module has a loader.Loader class which returns its instance by name.

├── actor
│   └── loader.py
├── reporter
│   └── loader.py
├── scenario
│   └── loader.py
└── suggestion
    ├── ACTOR_NAME.yml
    └── loader.py

3. Update experiment and run

$ git add .
$ git commit -m 'Customize modules'
$ git tag second-experiment
$ make run
$ mlflow ui

Repeat 2 and 3.

Workflow

BanditsFlow provides the following workflow. The workflow has optimize and evaluate and report steps. The each step result are saved by Metaflow and MLflow Tracking.

                                   ┌─────────┐
                                   │  start  │
                                   └────┬────┘
                         ┌──────────────┼──────────────┐
                     (actor-1)      (actor-2)      (actor-3)
 {suggestion}       ┌────┴────┐    ┌────┴────┐    ┌────┴────┐
 {scenario  } ─────►│optimize │    │optimize │    │optimize ├───► <best_params>
 {actor     }       └────┬────┘    └────┬────┘    └────┬────┘
                    best_params    best_params    best_params
                    ┌────┴────┐    ┌────┴────┐    ┌────┴────┐
 {scenario  } ─────►│evaluate │    │evaluate │    │evaluate ├─┬─► <result>
 {actor     }       └────┬────┘    └────┬────┘    └────┬────┘ │
                         │              │              │      ├─► [Parameter]
                      result         result         result    └─► [Metric]
                         └──────────────┼──────────────┘
                                   ┌────┴────┐
                                   │  join   │
                                   └────┬────┘
                                     results
                                   ┌────┴────┐
 {reporter  } ────────────────────►│ report  ├──────────────────► [Artifact]
                                   └────┬────┘
                                   ┌────┴────┐
                                   │   end   │
                                   └─────────┘

{}: Module
[]: MLflow Tracking
<>: Metaflow

Data location and relation

                              Metaflow                     MLflow Tracking

                     ┌─Flow(BanditsFlow)───────┐     ┌─Experiments─────────────┐
                     │                         │     │                         │
    RAW DATA         │                         │     │ ┌─exp-1───────────────┐ │       REPORT DATA
                     │   ┌─Run─────────────┐   │     │ │ ┌─Run (actor-1)───┐ │ │
<best_params> ◄─┬────────┤ ID:   mt-run-1  ├──────┬─────►│ ID:   ml-run-A  ├───────┬─► [Parameter]
Each <result> ◄─┤    │   │ Tag:  exp-1     │   │  │  │ │ │ Name: mt-run-1  │ │ │   └─► [Metric]
    <results> ◄─┘    │   └─────────────────┘   │  │  │ │ └─────────────────┘ │ │
                     │                         │  │  │ │ ┌─Run (actor-2)───┐ │ │
                     │                         │  ├─────►│ ID:   ml-run-B  ├───────┬─► [Parameter]
                     │                         │  │  │ │ │ Name: mt-run-1  │ │ │   └─► [Metric]
                     │                         │  │  │ │ └─────────────────┘ │ │
                     │                         │  │  │ │ ┌─Run (reporter)──┐ │ │
                     │                         │  └─────►│ ID:   ml-run-C  ├─────────► [Artifact]
                     │                         │     │ │ │ Name: mt-run-1  │ │ │
                     │                         │     │ │ └─────────────────┘ │ │
                     │                         │     │ │                     │ │
                     │                         │     │ │  -----------------  │ │
                     │                         │     │ │                     │ │
                     │   ┌─Run─────────────┐   │     │ │ ┌─Run (actor-1)───┐ │ │
                     │   │ ID:   mt-run-2  ├──────┬─────►│ ID:   ml-run-D  │ │ │
                     │   │ Tag:  exp-1     │   │  │  │ │ │ Name: mt-run-2  │ │ │
                     │   └─────────────────┘   │  │  │ │ └─────────────────┘ │ │
                     │                         │  │  │ │ ┌─Run (actor-2)───┐ │ │
                     │                         │  ├─────►│ ID:   ml-run-E  │ │ │
                     │                         │  │  │ │ │ Name: mt-run-2  │ │ │
                     │                         │  │  │ │ └─────────────────┘ │ │
                     │                         │  │  │ │ ┌─Run (reporter)──┐ │ │
                     │                         │  └─────►│ ID:   ml-run-F  │ │ │
                     │                         │     │ │ │ Name: mt-run-2  │ │ │
                     │                         │     │ │ └─────────────────┘ │ │
                     │                         │     │ └─────────────────────┘ │
                     │                         │     │                         │
                     │                         │     │ ┌─exp-2───────────────┐ │
                     │   ┌─Run─────────────┐   │     │ │ ...                 │ │
                     │   │ ID:   mt-run-3  ├────────────►...                 │ │
                     │   │ Tag:  exp-2     │   │     │ │ ...                 │ │
                     │   └─────────────────┘   │     │ │ ...                 │ │
                     │                         │     │ └─────────────────────┘ │
                     └─────────────────────────┘     └─────────────────────────┘

Cache

BanditsFlow stores metrics, results and reports for every run. BanditsFlow assumes that the results are the same for the same experiment, and reduces the time needed to re-run the experiment by using previous results. These caches are searched using the experiment name, scenario name, and actor name as keys. You can re-run the experiment by specifying the --revival_from_optimization_by or --revival_from_evaluation_by option or changing the name of the experiment by setting another git tag.

Optimization

BanditsFlow uses Optuna for optimization. Your suggestion loader class returns parameter suggestions for its actor. If you use the loader made by scaffold, each actor receives its suggestion which is prepared in suggestion module as YAML of its name. The YAML has suggestions dictionary which has list of parameter suggestion dictionary. Each parameter suggestion has name, type and parameter for each type.

An example of categorical parameter is the following:

suggestions:
  - name: epsilon
    type: discrete_uniform
    low: 0.1
    high: 1.0
    q: 0.1

See Optuna document for other type and parameter for each type.

Installation

$ pip install git+https://github.com/monochromegane/banditsflow

License

MIT

Author

monochromegane