zequnl/cmaml

The implementation of the ACL 2020 paper "Learning to Customize Model Structures for Few-shot Dialogue Generation Tasks"

Learning to Customize Model Structures for Few-shot Dialogue Generation Tasks

This is the implementation of our ACL 2020 paper:

Learning to Customize Model Structures for Few-shot Dialogue Generation Tasks.

Yiping Song, Zequn Liu, Wei Bi, Rui Yan, Ming Zhang

https://arxiv.org/abs/1910.14326

Please cite our paper when you use this code in your work.

Dependency

❱❱❱ pip install -r requirements.txt

Put the Pre-trained glove embedding: glove.6B.300d.txt in /vectors/.

Trained NLI model pytorch_model.bin in /data/nli_model/.

Experiment

The code is for the experiment of our model CMAML-Seq2SPG on Persona-chat. The scripts for training and evaluation are "train.sh" and "test.sh".

After training, please set the "--save_model" as the model with the lowest PPL in validation set to evaluate the model.

Acknowledgement

We use the framework of PAML and the Seq2seq implementation in https://github.com/MaximumEntropy/Seq2Seq-PyTorch

cmaml's Issues

About the formulation of "meta-train" and "meta-test" in the paper.

Hi @zequnl,

In the paper, you refer the episode training/testing as "meta-train" and "meta-test", while I think it is more common to refer the process of obtaining the meta-initialized model as "meta-train" and the adaptation process as "meta-test".

Question about some variables in the code

In seq2spg you update both loss and 'ppl' in the MAML process the function 'train_one_batch'. Usually ppl refers to the perplexity of the model but in your code I found that ppl = math.exp(min(loss.item(), 100)). Could you tell me what's that for? thx:)

About the hyperparameter setting.

Hi @zequnl,
I followed the default hyper-parameter setting during training. But I tried many times and the best validation loss I could get is around 96 at the 36-th meta iteration during the training process. But I found that the note in train.sh says that the loss should be around 45 at 55-th meta-iteration. So I wonder whther there is something wrong about the hyper-parameter setting.

您好，代码什么时候会公布呢？