The lime-experiments from jbdatascience

lime-experiments's Introduction

This repository contains the code to run the experiments present in this paper. The code here is frozen to what it was when we originally wrote the paper. If you're interested in using LIME, check out this repository, where we have packaged it up, improved the code quality, added visualizations and other improvements.

Running the commands below should be enough to get all of the results. You need specific versions python, sklearn, numpy, scipy. Install requirements in a virtualenv using:

pip install -r requirements.txt

If we forgot something, please email the first author.

Experiment in section 5.2:

DATASET -> 'multi_polarity_books', 'multi_polarity_kitchen', 'multi_polarity_dvd', 'multi_polarity_kitchen'
ALGORITHM -> 'l1logreg', 'tree'

EXPLAINER -> 'lime', 'parzen', 'greedy' or 'random'

  python evaluate_explanations.py --dataset DATASET --algorithm ALGORITHM --explainer EXPLAINER

Experiment in section 5.3:

DATASET -> 'multi_polarity_books', 'multi_polarity_kitchen', 'multi_polarity_dvd', 'multi_polarity_kitchen'
ALGORITHM -> 'logreg', 'random_forest', 'svm', 'tree' or 'embforest', although you would need to set up word2vec for embforest
```
  python data_trusting.py -d DATASET -a ALGORITHM -k 10 -u .25 -r NUM_ROUNDS
```

Experiment in section 5.4:

NUM_ROUNDS -> Desired number of rounds
DATASET -> 'multi_polarity_books', 'multi_polarity_kitchen', 'multi_polarity_dvd', 'multi_polarity_kitchen'

PICK -> 'submodular' or 'random' Run the following with the desired number of rounds:

  mkdir out_comparing

  python generate_data_for_compare_classifiers.py -d DATASET -o out_comparing/ -k 10 -r NUM_ROUNDS

  python compare_classifiers.py -d DATASET -o out_comparing/ -k 10 -n 10 -p PICK