greysun/dialogue-state-tracking-using-lstm

Source files to replicate experiments in my IWSDS 2016 paper.

Todo

Translate manual (and comments on source codes) to English

Name

DSTracker4DSTC4

Overview

Dialogue state tracker for DSTC4

Description

DSTracker4DSTC4 is a dialogue state tracker for DSTC4, written in Python. This tracker is based on Long-shot term memory, and estimates dialogue states from an input utterance and past dialogue history. This program 1) constructs trackers from training data, and 2) evaluates these trackers.

Demo

Execute "main.py" in DSTC4 directory.

Requirement

Mandatory

• Dialogue data following the DSTC4 specification
• Python (version 2.7.6+)
• Pybrain and its dependencies (0.3.3+)
• Scikit-learn (version 1.5+)
• fuzzywuzzy (0.5.0+)
• NLTK (3.0.2+)
• gensim (0.12.1+)

Optional

• python_Levenshtein

Usage

• [Construction of trackers:] Set the variable "isLearnLSTM" in "DSTC4/main.py" as "True".
• [Construction and evaluation of trackers with Sentence2Vec] 1) Set variables "isLearnDoc2vec4LSTM", "isLearnLSTM" and "ifFindTheBestOneOverLearnedNetworks" in "DSTC4/main.py" as "True". 2) Set the variable "isUseSentenceRepresentationInsteadofBOW" in "DSTC4/dstc4_traindev/scripts/LSTMWithBow.py" as "True". 3) Execute "main.py"
• [Evaluation of trackers:] Set the variable "ifFindTheBestOneOverLearnedNetworks" in "DSTC4/main.py" as "True", then exucte "main.py".
• [Construction and evaluation of trackers with Committee:] Set the variable "isLearnAndEvaluateNaiveEnsembler" in "DSTC4/main.py" as "True", and execute "main.py".

Install

1. install all requirment in "Mandatory".
2. download this project and set python binary path to DSTC4 directory.
3. put dialogue data into DSTC4\dstc4_traindev\data.

Introducing new feature

In order to append new feature, we need implement following part in "DSTC4\dstc4_traindev\scripts\LSTMWithBOW.py":

• [Registration of new feature:] __rejisterM1sInputFeatureLabel
• [Calculation of registered feature:] __calculateM1sInputFeature

Tips

TBA

Contribution

TBA

Licence

TBA

Author

TakuyaHiroka