ratishsp/data2text-plan-py

Code for AAAI 2019 paper on Data-to-Text Generation with Content Selection and Planning

data2text-plan-py

This repo contains code for Data-to-Text Generation with Content Selection and Planning (Puduppully, R., Dong, L., & Lapata, M.; AAAI 2019); this code is based on an earlier fork of OpenNMT-py. The Pytorch version is 0.3.1.

Update: For a model with better relation generation precision (RG P%) and other metrics, please see the macro planning repository and the corresponding TACL 2021 paper.

Citations

@inproceedings{DBLP:conf/aaai/Puduppully0L19,
  author    = {Ratish Puduppully and
               Li Dong and
               Mirella Lapata},
  title     = {Data-to-Text Generation with Content Selection and Planning},
  booktitle = {The Thirty-Third {AAAI} Conference on Artificial Intelligence, {AAAI}
               2019, The Thirty-First Innovative Applications of Artificial Intelligence
               Conference, {IAAI} 2019, The Ninth {AAAI} Symposium on Educational
               Advances in Artificial Intelligence, {EAAI} 2019, Honolulu, Hawaii,
               USA, January 27 - February 1, 2019},
  pages     = {6908--6915},
  publisher = {{AAAI} Press},
  year      = {2019},
  url       = {https://doi.org/10.1609/aaai.v33i01.33016908},
  doi       = {10.1609/aaai.v33i01.33016908},
  timestamp = {Tue, 02 Feb 2021 08:00:48 +0100},
  biburl    = {https://dblp.org/rec/conf/aaai/Puduppully0L19.bib},
  bibsource = {dblp computer science bibliography, https://dblp.org}
}

Test set output

The test set output for the model can be found here

Requirements

All dependencies can be installed via:

pip install -r requirements.txt

Note that the Pytorch version is 0.3.1 and Python version is 2.7. The path to Pytorch wheel in requirements.txt is configured with CUDA 8.0. You may change it to the desired CUDA version.

Dataset

The boxscore-data json files can be downloaded from the boxscore-data repo.

The input dataset for data2text-plan-py can be created by running the script create_dataset.py in scripts folder. The dataset so obtained is available at link https://drive.google.com/open?id=1R_82ifGiybHKuXnVnC8JhBTW8BAkdwek

Preprocessing

Assuming the OpenNMT-py input files reside at ~/boxscore-data, the following command will preprocess the data

BASE=~/boxscore-data
IDENTIFIER=cc

mkdir $BASE/preprocess
python preprocess.py -train_src1 $BASE/rotowire/src_train.txt -train_tgt1 $BASE/rotowire/train_content_plan.txt -train_src2 $BASE/rotowire/inter/train_content_plan.txt -train_tgt2 $BASE/rotowire/tgt_train.txt -valid_src1 $BASE/rotowire/src_valid.txt -valid_tgt1 $BASE/rotowire/valid_content_plan.txt -valid_src2 $BASE/rotowire/inter/valid_content_plan.txt -valid_tgt2 $BASE/rotowire/tgt_valid.txt -save_data $BASE/preprocess/roto -src_seq_length 1000 -tgt_seq_length 1000 -dynamic_dict -train_ptr $BASE/rotowire/train-roto-ptrs.txt

The train-roto-ptrs.txt file is available along with the dataset and can also be created by the following command

python data_utils.py -mode ptrs -input_path $BASE/rotowire/train.json -train_content_plan $BASE/rotowire/inter/train_content_plan.txt -output_fi $BASE/rotowire/train-roto-ptrs.txt

Training (and Downloading Trained Models)

The command for training the Neural Content Planning model with conditional copy NCP+CC is as follows:

BASE=~/boxscore-data
IDENTIFIER=cc

python train.py -data $BASE/preprocess/roto -save_model $BASE/gen_model/$IDENTIFIER/roto -encoder_type1 mean -decoder_type1 pointer -enc_layers1 1 -dec_layers1 1 -encoder_type2 brnn -decoder_type2 rnn -enc_layers2 2 -dec_layers2 2 -batch_size 5 -feat_merge mlp -feat_vec_size 600 -word_vec_size 600 -rnn_size 600 -seed 1234 -start_checkpoint_at 4 -epochs 25 -optim adagrad -learning_rate 0.15 -adagrad_accumulator_init 0.1 -report_every 100 -copy_attn -truncated_decoder 100 -gpuid $GPUID -attn_hidden 64 -reuse_copy_attn -start_decay_at 4 -learning_rate_decay 0.97 -valid_batch_size 5

The NCP+CC model can be downloaded from https://www.dropbox.com/sh/vo5wb2fuq7m0bk0/AABikW0KomOKIor24wD8VSFWa?dl=0

Generation

During inference, we first generate the content plan

MODEL_PATH=<path to model1>

python translate.py -model $MODEL_PATH -src1 $BASE/rotowire/inf_src_valid.txt -output $BASE/gen/roto_stage1_$IDENTIFIER-beam5_gens.txt -batch_size 10 -max_length 80 -gpu $GPUID -min_length 35 -stage1 

This script generates the content plan with records from input of content plan with indices

python scripts/create_content_plan_from_index.py $BASE/rotowire/inf_src_valid.txt $BASE/gen/roto_stage1_$IDENTIFIER-beam5_gens.txt $BASE/transform_gen/roto_stage1_$IDENTIFIER-beam5_gens.h5-tuples.txt  $BASE/gen/roto_stage1_inter_$IDENTIFIER-beam5_gens.txt

The accuracy of content plan in first stage can be evaluated using the following command

python non_rg_metrics.py $BASE/transform_gen/roto-gold-val-beam5_gens.h5-tuples.txt $BASE/transform_gen/roto_stage1_$IDENTIFIER-beam5_gens.h5-tuples.txt 

The output summary is generated using the command

MODEL_PATH2=<path to model2>

python translate.py -model $MODEL_PATH -model2 $MODEL_PATH2 -src1 $BASE/rotowire/inf_src_valid.txt -tgt1 $BASE/gen/roto_stage1_$IDENTIFIER-beam5_gens.txt -src2 $BASE/gen/roto_stage1_inter_$IDENTIFIER-beam5_gens.txt -output $BASE/gen/roto_stage2_$IDENTIFIER-beam5_gens.txt -batch_size 10 -max_length 850 -min_length 150 -gpu $GPUID

Automatic evaluation using IE metrics

Metrics of RG, CS, CO are computed using the below commands.

python data_utils.py -mode prep_gen_data -gen_fi $BASE/gen/roto_stage2_$IDENTIFIER-beam5_gens.txt -dict_pfx "roto-ie" -output_fi $BASE/transform_gen/roto_stage2_$IDENTIFIER-beam5_gens.h5 -input_path "/boxcore-json/rotowire"

th extractor.lua -gpuid  $GPUID -datafile roto-ie.h5 -preddata $BASE/transform_gen/roto_stage2_$IDENTIFIER-beam5_gens.h5 -dict_pfx "roto-ie" -just_eval

python non_rg_metrics.py $BASE/transform_gen/roto-gold-val-beam5_gens.h5-tuples.txt $BASE/transform_gen/roto_stage2_$IDENTIFIER-beam5_gens.h5-tuples.txt 

Evaluation using BLEU script

The BLEU perl script can be obtained from https://github.com/moses-smt/mosesdecoder/blob/master/scripts/generic/multi-bleu.perl Command to compute BLEU score:

~/multi-bleu.perl $BASE/rotowire/inf_tgt_valid.txt < $BASE/gen/roto_stage2_$IDENTIFIER-beam5_gens.txt

IE models

For training the IE models, follow the updated code in https://github.com/ratishsp/data2text-1 which contains bug fixes for number handling. The repo contains the downloadable links for IE models too.

data2text-plan-py's Issues

RuntimeError: 'lengths' array has to be sorted in decreasing order

Thanks for your sharing. when I used your dataset to drive your model, I get the problem. But I have followed your requirements to install the environment. Do you know how to solve it?

Question for setup

Hi @ratishsp ,

Thanks a lot for your continuous contribution for this project. I am currently working on a similar project and want to try your approach on our settings.

From the instructions, I can see the code is in Python 2.7 but it seems that 2.7 is not available anymore and I am using CUDA 11

Just wonder if it works with other Python version and which torch version should I use for this project?

Thanks a lot for your help.

Best wishes,
Tong

Error: No LuaRocks module found for onmt.modules.MarginalNLLCriterion

Hello, anyone knows why "No LuaRocks module found for onmt.modules.MarginalNLLCriterion " error happened when running extractor.lua file?

/content/torch/install/bin/luajit: /content/torch/install/share/lua/5.1/trepl/init.lua:389: module 'onmt.modules.MarginalNLLCriterion' not found:No LuaRocks module found for onmt.modules.MarginalNLLCriterion
	no field package.preload['onmt.modules.MarginalNLLCriterion']
	no file '/root/.luarocks/share/lua/5.1/onmt/modules/MarginalNLLCriterion.lua'
	no file '/root/.luarocks/share/lua/5.1/onmt/modules/MarginalNLLCriterion/init.lua'
	no file '/content/torch/install/share/lua/5.1/onmt/modules/MarginalNLLCriterion.lua'
	no file '/content/torch/install/share/lua/5.1/onmt/modules/MarginalNLLCriterion/init.lua'
	no file './onmt/modules/MarginalNLLCriterion.lua'
	no file '/content/torch/install/share/luajit-2.1.0-beta1/onmt/modules/MarginalNLLCriterion.lua'
	no file '/usr/local/share/lua/5.1/onmt/modules/MarginalNLLCriterion.lua'
	no file '/usr/local/share/lua/5.1/onmt/modules/MarginalNLLCriterion/init.lua'
	no file '/root/.luarocks/lib/lua/5.1/onmt/modules/MarginalNLLCriterion.so'
	no file '/content/torch/install/lib/lua/5.1/onmt/modules/MarginalNLLCriterion.so'
	no file './onmt/modules/MarginalNLLCriterion.so'
	no file '/usr/local/lib/lua/5.1/onmt/modules/MarginalNLLCriterion.so'
	no file '/usr/local/lib/lua/5.1/loadall.so'
	no file '/root/.luarocks/lib/lua/5.1/onmt.so'
	no file '/content/torch/install/lib/lua/5.1/onmt.so'
	no file './onmt.so'
	no file '/usr/local/lib/lua/5.1/onmt.so'
	no file '/usr/local/lib/lua/5.1/loadall.so'
stack traceback:
	[C]: in function 'error'
	/content/torch/install/share/lua/5.1/trepl/init.lua:389: in function 'require'
	../../drive/My Drive/data2text-1/extractor.lua:7: in main chunk
	[C]: in function 'dofile'
	...tent/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
	[C]: at 0x55c4cd1b5570

I am running the file in Google Colab to use GPU there.
Thanks.

Question about content plan

Hello, thank you so much for sharing such a nice project.

I have read the paper and I want to ask one question about content plan.

Could you tell me whether the data used for training the model included content plan? Or it only used original ROTOWIRE (Wiseman et al. 2017) game summary and box-score dataset?

Thank you so much.

I can't download the trained model from your mentioned link.

The web has some problems? Can you give me another link? thank you very much. I really need the trained model emergently.

Training Failed

Hi Ratish,

Thanks to your comments I was able to create all the required files (from another dataset) for preprocess.py script. After running this script, in the folder preprocess I have 3 files: roto.train.1.pt, roto.valid.1.pt, roto.vocab.pt.

These files are used as input to the model training. Unfortunately, running train.py fails, with the following error:

RuntimeError: weight tensor should be defined either for all 14 classes or no classes but got weight tensor of shape: [606] at /pytorch/torch/lib/THCUNN/generic/ClassNLLCriterion.cu:44

Have you encountered such an error while you were running your experiments?

I understand, that the error is related to training_dataset --> OpenNMT --> pytorch. But since I can successfully execute code on boxscore-dataset, I think that the problem is still with my data_preparation steps (Basically I want to create input files *.pt for the model but from another dataset.).
Are there special requirements for the format/structure of the dataset?

I just made sure that files like src_trian.txt, tgt_train.txt, train_content_plan.txt look identical as files obtained after running create_datasets.py script. Is there smth I need to take care of additionally ?

I would highly appreciate your help on this issue, since after extensive debugging I still could not figure out the problem!

Many thanks in advance!
Ruslan

(data2text_plan_py) ruslan_yermakov@ip-10-123-133-120:~/reproducibility/data2text-plan-py$ python train.py -data $BASE/preprocess/roto -save_model $BASE/gen_model/$IDENTIFIER/roto -encoder_type1 mean -decoder_type1 pointer -enc_layers1 1 -dec_layers1 1 -encoder_type2 brnn -decoder_type2 rnn -enc_layers2 2 -dec_layers2 2 -batch_size 5 -feat_merge mlp -feat_vec_size 600 -word_vec_size 600 -rnn_size 600 -seed 1234 -start_checkpoint_at 4 -epochs 25 -optim adagrad -learning_rate 0.15 -adagrad_accumulator_init 0.1 -report_every 100 -copy_attn -truncated_decoder 100 -gpuid $GPUID -attn_hidden 64 -reuse_copy_attn -start_decay_at 4 -learning_rate_decay 0.97 -valid_batch_size 5
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longermaintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip, can be foundat https://pip.pypa.io/en/latest/development/release-process/#python-2-support
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longermaintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip, can be foundat https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Experiment 22-4.4 using attn_dim of 64
Loading train dataset from /home/ruslan_yermakov/reproducibility/data2text-plan-py/new_dataset/preprocess/roto.train.1.pt, number of examples: 1000
 * vocabulary size. source1 = 4660; target1 = 105, source2 = 4658; target2 = 7382
 * src feature 0 size = 3
 * src feature 1 size = 22
 * src feature 2 size = 4
Building model...
Intializing model parameters.
Intializing model parameters.
NMTModel(
  (encoder): MeanEncoder(
    (embeddings): Embeddings(
      (make_embedding): Sequential(
        (emb_luts): Elementwise(
          (0): Embedding(4660, 600, padding_idx=1)
          (1): Embedding(3, 600, padding_idx=1)
          (2): Embedding(22, 600, padding_idx=1)
          (3): Embedding(4, 600, padding_idx=1)
        )
        (mlp): Sequential(
          (0): Linear(in_features=2400, out_features=600, bias=True)
          (1): ReLU()
        )
      )
    )
    (dropout): Dropout(p=0.3)
    (attn): GlobalSelfAttention(
      (transform_in): Sequential(
        (0): Linear(in_features=600, out_features=64, bias=True)
        (1): ELU(alpha=0.1)
      )
      (linear_in): Linear(in_features=64, out_features=64, bias=False)
      (linear_out): Linear(in_features=1200, out_features=600, bias=False)
      (sm): Softmax()
      (tanh): Tanh()
      (dropout): Dropout(p=0.3)
    )
  )
  (decoder): PointerRNNDecoder(
    (embeddings): Embeddings(
      (make_embedding): Sequential(
        (emb_luts): Elementwise(
          (0): Embedding(105, 600, padding_idx=1)
        )
      )
    )
    (dropout): Dropout(p=0.3)
    (rnn): LSTM(600, 600, dropout=0.3)
    (attn): PointerAttention(
      (linear_in): Linear(in_features=600, out_features=600, bias=False)
      (sm): LogSoftmax()
    )
  )
  (generator): Sequential(
    (0): Linear(in_features=600, out_features=105, bias=True)
    (1): LogSoftmax()
  )
)
NMTModel(
  (encoder): RNNEncoder(
    (embeddings): Embeddings(
      (make_embedding): Sequential(
        (emb_luts): Elementwise(
          (0): Embedding(4658, 600, padding_idx=1)
          (1): Embedding(3, 600, padding_idx=1)
          (2): Embedding(22, 600, padding_idx=1)
          (3): Embedding(4, 600, padding_idx=1)
        )
        (mlp): Sequential(
          (0): Linear(in_features=2400, out_features=600, bias=True)
          (1): ReLU()
        )
      )
    )
    (rnn): LSTM(600, 300, num_layers=2, dropout=0.3, bidirectional=True)
  )
  (decoder): InputFeedRNNDecoder(
    (embeddings): Embeddings(
      (make_embedding): Sequential(
        (emb_luts): Elementwise(
          (0): Embedding(7382, 600, padding_idx=1)
        )
      )
    )
    (dropout): Dropout(p=0.3)
    (rnn): StackedLSTM(
      (dropout): Dropout(p=0.3)
      (layers): ModuleList(
        (0): LSTMCell(1200, 600)
        (1): LSTMCell(600, 600)
      )
    )
    (attn): GlobalAttention(
      (linear_in): Linear(in_features=600, out_features=600, bias=False)
      (linear_out): Linear(in_features=1200, out_features=600, bias=False)
      (sm): Softmax()
      (tanh): Tanh()
    )
  )
  (generator): CopyGenerator(
    (linear): Linear(in_features=600, out_features=7382, bias=True)
    (linear_copy): Linear(in_features=600, out_features=1, bias=True)
  )
)
* number of parameters: 8387465
('encoder: ', 5016560)
('decoder: ', 3370905)
* number of parameters: 25738383
('encoder: ', 8582400)
('decoder: ', 17155983)
Making optimizer for training.
Making optimizer for training.
Start training...
 * number of epochs: 25, starting from Epoch 1
 * batch size: 5
Loading train dataset from /home/ruslan_yermakov/reproducibility/data2text-plan-py/new_dataset/preprocess/roto.train.1.pt, number of examples: 1000
Traceback (most recent call last):
  File "train.py", line 454, in <module>
    main()
  File "train.py", line 446, in main
    train_model(model1, model2, fields, optim1, optim2, data_type, model_opt)
  File "train.py", line 256, in train_model
    train_stats, train_stats2 = trainer.train(train_iter, epoch, report_func)
  File "/home/ruslan_yermakov/reproducibility/data2text-plan-py/onmt/Trainer.py", line 181, in train
    report_stats, total_stats2, report_stats2, normalization)
  File "/home/ruslan_yermakov/reproducibility/data2text-plan-py/onmt/Trainer.py", line 350, in _gradient_accumulation
    trunc_size, self.shard_size, normalization, retain_graph=True)
  File "/home/ruslan_yermakov/reproducibility/data2text-plan-py/onmt/Loss.py", line 127, in sharded_compute_loss
    loss, stats = self._compute_loss(batch, **shard)
  File "/home/ruslan_yermakov/reproducibility/data2text-plan-py/onmt/Loss.py", line 219, in _compute_loss
    loss = self.criterion(scores, gtruth)
  File "/home/ruslan_yermakov/anaconda3/envs/data2text_plan_py/lib/python2.7/site-packages/torch/nn/modules/module.py", line 357, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/ruslan_yermakov/anaconda3/envs/data2text_plan_py/lib/python2.7/site-packages/torch/nn/modules/loss.py", line 170,in forward
    self.ignore_index, self.reduce)
  File "/home/ruslan_yermakov/anaconda3/envs/data2text_plan_py/lib/python2.7/site-packages/torch/nn/functional.py", line 1052, in nll_loss
    return torch._C._nn.nll_loss(input, target, weight, size_average, ignore_index, reduce)
RuntimeError: weight tensor should be defined either for all 14 classes or no classes but got weight tensor of shape: [606] at /pytorch/torch/lib/THCUNN/generic/ClassNLLCriterion.cu:44
(data2text_plan_py) ruslan_yermakov:~/reproducibility/data2text-plan-py$

Log of running preprocess.py

(data2text_plan_py) ruslan_yermakov:~/reproducibility/data2text-plan-py$ python preprocess.py -train_src1 $BASE/src_train.txt -train_tgt1 $BASE/train_content_plan.txt -train_src2 $BASE/inter/train_content_plan.txt -train_tgt2 $BASE/tgt_train.txt -valid_src1 $BASE/src_valid.txt -valid_tgt1 $BASE/valid_content_plan.txt -valid_src2 $BASE/inter/valid_content_plan.txt -valid_tgt2 $BASE/tgt_valid.txt -save_data $BASE/preprocess/roto -src_seq_length 1000 -tgt_seq_length 1000 -dynamic_dict-train_ptr $BASE/train-roto-ptrs.txt
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longermaintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip, can be foundat https://pip.pypa.io/en/latest/development/release-process/#python-2-support
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longermaintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip, can be foundat https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Extracting features...

• number of source features- stage 1: 3.
• number of target features- stage 1: 0.
• number of source features- stage 2: 3.
• number of target features- stage 2: 0.
  Building Fields object...
  Building & saving training data...
  ('average src size', 26, 1000)
• saving train data shard to /home/ruslan_yermakov/reproducibility/data2text-plan-py/new_dataset/preprocess/roto.train.1.pt.
  Building & saving vocabulary...
• reloading /home/ruslan_yermakov/reproducibility/data2text-plan-py/new_dataset/preprocess/roto.train.1.pt.
• tgt1 vocab size: 105.
• tgt2 vocab size: 7382.
• src1 vocab size: 4660.
• src1_feat_0 vocab size: 3.
• src1_feat_1 vocab size: 22.
• src1_feat_2 vocab size: 4.
• src2 vocab size: 4658.
• src2_feat_0 vocab size: 3.
• src2_feat_1 vocab size: 22.
• src2_feat_2 vocab size: 4.
  Building & saving validation data...
  ('average src size', 25, 120)
• saving valid data shard to /home/ruslan_yermakov/reproducibility/data2text-plan-py/new_dataset/preprocess/roto.valid.1.pt.
  (data2text_plan_py) ruslan_yermakov:~/reproducibility/data2text-plan-py$

IOError: [Errno 2] No such file or directory

Hi,

I am following steps on the README to reproduce your results. In the Generation step, there is a command:
python scripts/create_content_plan_from_index.py $BASE/rotowire/inf_src_valid.txt $BASE/gen/roto_stage1_$IDENTIFIER-beam5_gens.txt $BASE/transform_gen/roto_stage1_$IDENTIFIER-beam5_gens.h5-tuples.txt $BASE/gen/roto_stage1_inter_$IDENTIFIER-beam5_gens.txt
and it says to generate the content plan with records from input of content plan with indices.

While running this script, I got the following error:

IOError: [Errno 2] No such file or directory: data2text-plan-py/boxscore-data/transform_gen/roto_stage1_cc-beam5_gens.h5-tuples.txt'

Is the file roto_stage1_cc-beam5_gens.h5-tuples.txt' same as file roto_train-beam5_gens.h5-tuples.txt from here?

If no, could you please briefly explain how to create the required file?

I run successfully all the steps, but at this point I do not even have a transform_gen directory.

Many thanks,
Ruslan

version of pyxdameraulevenshtein

I installed it using pip install pyxdameraulevenshtein

However, I am facing this ValueError:

  File "non_rg_metrics.py", line 2, in <module>
    from pyxdameraulevenshtein import normalized_damerau_levenshtein_distance
  File "__init__.pxd", line 918, in init pyxdameraulevenshtein
ValueError: numpy.ufunc size changed, may indicate binary incompatibility. Expected 216 from C header, got 192 from PyObject

After loss.backward(), why needs torch.autograd.backward(inputs, grads, retain_graph=retain_graph)

Hi Ratish, I have a problem here. After _compute_loss() and loss.div(normalization).backward(), why in function shards , there is torch.autograd.backward(inputs, grads, retain_graph=retain_graph) ? is it usual we use loss to do backward job? why use inputs backward again?

About Accury and Perplexity

Hello, I would like to ask why the accuracy of the results I trained is 0, and I did not change the code. Furthermore, why is the perplexity between the training set and the verification set always the same?

remember to use torchtext==0.2.3

I tried running the preprocessing script after installing the latest torchtext, which seems to be incompatible by throwing this error:

***/torchtext/data/field.py", line 118, in Field torch.float32: float, AttributeError: 'module' object has no attribute 'float32'

Solved by referring to this post:
https://github.com/OpenNMT/OpenNMT-py/issues/900#issuecomment-437919442

Hope it helps

Train failed: inconsistent sequence length

Hi,
I am trying to run this work on my own dataset. After preprocessing the dataset, I got train.1.pt/valid.1.pt/vocab.pt. I notice that I do not have the train-roto-ptr.txt so I followed the isssue#26 and commented out all the related codes that used the supervision through pointers.
And then, I met the same problem in this issue#28. So I reset the TGT_VOCAB_SIZE and finally, I got this error message:

Traceback (most recent call last):
File "train.py", line 454, in
main()
File "train.py", line 446, in main
train_model(model1, model2, fields, optim1, optim2, data_type, model_opt)
File "train.py", line 256, in train_model
train_stats, train_stats2 = trainer.train(train_iter, epoch, report_func)
File "/data2text-plan-py-master/onmt/Trainer.py", line 164, in train
for i, batch in enumerate(train_iter):
File "train.py", line 137, in iter
for batch in self.cur_iter:
File "/anaconda2/lib/python2.7/site-packages/torchtext/data/iterator.py", line 151, in iter
self.train)
File "/anaconda2/lib/python2.7/site-packages/torchtext/data/batch.py", line 27, in init
setattr(self, name, field.process(batch, device=device, train=train))
File "/data2text-plan-py-master/onmt/io/BoxField.py", line 134, in process
tensor = self.numericalize(padded, device=device, train=train)
File "/data2text-plan-py-master/onmt/io/BoxField.py", line 253, in numericalize
arr = self.tensor_type(arr)
RuntimeError: inconsistent sequence length at index (3, 20) - expected 21 but got 20

I am trying to figure it out, probably it has something to do with the TextDataset method get_fields as it is related to BoxField:

@staticmethod
    def get_fields(n_src_features, n_tgt_features):
        """
        Args:
            n_src_features (int): the number of source features to
                create `torchtext.data.Field` for.
            n_tgt_features (int): the number of target features to
                create `torchtext.data.Field` for.

        Returns:
            A dictionary whose keys are strings and whose values
            are the corresponding Field objects.
        """
        fields = {}

        fields["src1"] = BoxField(
            sequential=False,
            init_token=BOS_WORD,
            eos_token=EOS_WORD,
            pad_token=PAD_WORD)

But I have no idea how to fix it.

requirements.txt

Hi Ratish,
Thank you for providing the code. Could you also share the requirements file?

where are the files.txt?

Hello, thanks for your opening source, but I still have some problems with your code.
(1): In the create_dataset.py, I can't find the train_content_plan.txt, the src_train.txt, the tgt_train.txt, and train_content_plan.txt. Because lack these files, I can't run the create_dataset.py document.
(2): I don't know the downloaded dataset should be placed in where, and the dataset has any relation with the openNMT?
(3): In the create_content_plan_from_index.py, from the line 4 to 7, we should input 4 parameters, but I don't know what parameters I should input.
I'll be very happy to get your response.

run train.py TypeError: process() missing 1 required positional argument: 'train'

run train.py TypeError: process() missing 1 required positional argument: 'train'

what the inf_src_valid.txt

When I drove the translate.py, I need the inf_src_valid.txt, but you don't mention the file. Could you tell me how can I get the file?

Focus only on text generation

Hi @ratishsp,

Thanks a lot for your continuous support of the project! I still plan to apply your model in my research paper (soon I will write an update on #28) on the dataset different from RotoWire.

From your paper:

(a) content selection and planning - operates on the input records of a database and produces a content plan specifying which records are to be verbalized in the document and in which order
(b) text generation produces the output text given the content plan as input; at each decoding step, the generation model attends over vector representations of the records in the content plan

In my case, there is no need to have an explicit plan specifying both what to say and in which order. Basically, I want to use all input records and these records are already ordered according to their location in the target summary. In my case, the (training) content plan is exactly the same as the input. So ideally the model at stage1 should be identity function (since content plan same as training data).

For text generation (model at stage2), you use the encoder-decoder architecture with an attention mechanism (encode - BiLSTM, decoder - LSTM with copy mechanism).

Is it correct to say that in my data2text scenario, I only use the encoder-decoder architecture part by leveraging your implemented data2text model? In this case, would encoder-decoder architecture with an attention mechanism be a sufficient approach for data2text generation? Is there something I am missing?

Looking forward to your reply :)

Many thanks in advance,
Ruslan

wherer i can get this file.txt？

hello,where i can get this file :
ORACLE_IE_OUTPUT = 'roto_train-beam5_gens.h5-tuples.txt' # oracle content plan obtained from IE tool
i want to rep your experiment on my own dataset. i can't produce the dataset due to the lack of the above file.

While evaluating accuracy, IndexError: tuple index out of range

When I am evaluating accuracy using below command available in the readme.md

python non_rg_metrics.py $BASE/transform_gen/roto-gold-val-beam5_gens.h5-tuples.txt $BASE/transform_gen/roto_stage1_$IDENTIFIER-beam5_gens.h5-tuples.txt

I was not able to find roto-gold-val-beam5_gens.h5-tuples.txt
So, based on #2 issue I have replace it with https://github.com/ratishsp/data2text-1/blob/master/roto-gold-val.h5-tuples.txt

But after getting the file I am again facing following issue.

Traceback (most recent call last):
File "non_rg_metrics.py", line 133, in
calc_precrec(sys.argv[1], sys.argv[2])
File "non_rg_metrics.py", line 81, in calc_precrec
gold_triples = get_triples(goldfi)
File "non_rg_metrics.py", line 71, in get_triples
all_triples.append(dedup_triples(curr))
File "non_rg_metrics.py", line 60, in dedup_triples
if trip_match(triplist[i], triplist[j]):
File "non_rg_metrics.py", line 51, in trip_match
return int_value(t1[1]) == int_value(t2[1]) and t1[2] == t2[2] and same_ent(t1[0], t2[0])
IndexError: tuple index out of range

I guess it is because of data length not matching.
Do I need to generate the file myself if yes then how? Please help me to resolve the issue.

Understanding train-roto-ptrs.txt file

Hi Ratish,

Thanks a lot for the insightful research paper as well as making the codebase publicly available! I was able to train the model on boxscore-data and then used it for inference. Now I am interested in training your model on my dataset and then use it for text generation.

Unfortunately, I encountered a problem with the following step (from the README page):

The train-roto-ptrs.txt file is available along with the dataset and can also be created by the following command:

python data_utils.py -mode ptrs -input_path $BASE/rotowire/train.json -train_content_plan $BASE/rotowire/inter/train_content_plan.txt -output_fi $BASE/rotowire/train-roto-ptrs.txt

Since my dataset is different from boxscore-data, I basically have to perform transformations manually to my dataset in order to have it in a suitable format for model training.

1. "The input dataset for data2text-plan-py can be created by running the script create_dataset.py in scripts folder."

I have successfully prepared my dataset in the same format as files in boxscore-data/rotowire/, namely, the following files: src_train.txt, train_content_plan.txt, tgt_train.txt, inter/train_content_plan.txt and src_valid.txt, tgt_valid.txt, valid_content_plan.txt, inter/valid_content_plan.txt and test/src_test.txt , test/tgt_test.txt. The structure of these files is the same as in files obtained from boxscore-data

2. [Preprocessing] I could not run preprocess.py script, since to run this script I need train-roto-ptrs.txt file.

The function for creating this file is:

data2text-plan-py/data_utils.py

Line 574 in 4b74535

def make_pointerfi(outfi, inp_file="rotowire/train.json", content_plan_inp="inter/train_content_plan", resolve_prons=False):

Because I have another dataset, I can not use the above-mentioned function, thus I need to create train-roto-ptrs.txt file by myself. Unfortunately, going multiple times through the function implementation and analyzing the content of a file (please see comments), I could not figure out how to create such file from my dataset.

Can you please elaborate on the purposes of the train-roto-ptrs.txt file and briefly describe the steps on how it was created?

This is my current bottleneck and I would highly appreciate your help with this issue!

Many thanks in advance,
Ruslan

file missing while evaluation

I try to use the automatic evaluation using IE metrics. There was an IOError: [Errno 2] No such file or directory: 'roto-ie.dict'.
I wonder where I can obtain this file.

And I wonder if there is any code provided to calculate the BLEU score?

Thanks.

NOT able to find the file 'roto_train-beam5_gens.h5-tuples.txt'

I have downloaded the data from rotowire but not able to find the file name roto_train-beam5_gens.h5-tuples.txt,please guide me the way for it.

questions about cpu and content plan tracing back

Hello, thank you so much for sharing such a nice project.

I have read the paper and I want to ask two questions.

1. Could you tell me whether extractor.lua file could be run under CPU environment using the model you provided? I have tried to set the label -gpu -1 , but unfortunately it was nor working.

2. Besides, I am wondering whether it is possible to extract the data items that are related to each sentence generated by the model.
   For example, there is one sentence in the generated summary Tristan Thompson chipped in seven points and 13 rebounds as the starting power forward. Whether it is possible to trace back to which content plan items (TT's PTS related content plan item and TT's Rebound related content plan item) generated this sentence? Are there any variables in the code that I could use to trace back?

Thank you so much for your time and consideration.

data_utils.py List index out of range

While creating train-roto-ptrs.txt using ptrs mode, I am getting this index error:

Traceback (most recent call last):
  File "data_utils.py", line 859, in <module>
    make_pointerfi(args.output_fi, inp_file=args.input_path, content_plan_inp=args.train_content_plan)
  File "data_utils.py", line 593, in make_pointerfi
    content_plan_entry = [content_plan_record for content_plan_record in content_plan[i]]
IndexError: list index out of range

Any quick suggestion?

how do you generate the file "roto_train-beam5_gens.h5-tuples.txt"

Hi,
Thanks for the excellent paper and code.
(1) What is the purpose of the file "roto_train-beam5_gens.h5-tuples.txt"?
(2) How do you generate it?
(3) If I use other datasets, how to generate files like it?

Thank you very much.

Which metric is used in development?

Thanks for your sharing.
In model training, there are two metric 'ppl' and 'accuracy'. What's more , there are other metrics in evaluation, such as RG, CO and CS.
So which one is used to mentor parameters choosing in development.
Thanks so much

Raise a ValueError（invalid literal for int() with base 10: 'tensor(589,'）on the step 2 of generate

Hi @ratishsp

Traceback (most recent call last):
File "scripts/create_content_plan_from_index.py", line 29, in
output.append(input[int(record)].encode("utf-8"))
ValueError: invalid literal for int() with base 10: 'tensor(589,'

The first line of file which is generated from step 1:
'tensor(589, device='cuda:0') tensor(590, device='cuda:0') tensor(582, device='cuda:0') tensor(420, device='cuda:0') tensor(421, device='cuda:0') tensor(402, device='cuda:0') tensor(414, device='cuda:0') tensor(415, device='cuda:0') tensor(376, device='cuda:0') tensor(377, device='cuda:0') tensor(358, device='cuda:0') tensor(370, device='cuda:0') tensor(371, device='cuda:0') tensor(373, device='cuda:0') tensor(332, device='cuda:0') tensor(333, device='cuda:0') tensor(314, device='cuda:0') tensor(326, device='cuda:0') tensor(327, device='cuda:0') tensor(156, device='cuda:0') tensor(157, device='cuda:0') tensor(138, device='cuda:0') tensor(150, device='cuda:0') tensor(90, device='cuda:0') tensor(91, device='cuda:0') tensor(72, device='cuda:0') tensor(84, device='cuda:0') tensor(85, device='cuda:0') tensor(68, device='cuda:0') tensor(69, device='cuda:0') tensor(50, device='cuda:0') tensor(62, device='cuda:0') tensor(66, device='cuda:0') tensor(24, device='cuda:0') tensor(25, device='cuda:0') tensor(6, device='cuda:0') tensor(18, device='cuda:0') tensor(19, device='cuda:0')
'
Am I running something wrong? Could you please take a look at it for me?

Missing of roto_val-beam5_gens.h5-tuples.txt

When I tried to evaluate the accuracy of the first stage, I found that roto_val-beam5_gens.h5-tuples.txt is missing. I assumed that this is the gold tuple file. I wonder if I need to generate by myself. Or I can find it somewhere.

Thanks

How to generate the content selection in a more difficult scenario?

I have a dataset that also needs content selection. But the table values don't match with summary tokens. For example, in the table, the value is "4993 million", but in the summary, the value is expressed as 2.5 billion. In the scenario, I can't use string matching to generate content selection and planning. Could you give me some advice about the problem?

the length of train.json is different from train_content_plan.txt

when I drive the data_utils.py, the Terminal output that the list out of range. I find that the length of train.json is different from train_content_plan.txt. I guess the reason is that the two files have different lengths. Am I right? Can you tell me how to solve the problem?

environment config for running extractor.lua?

Hi, thanks for the awesome code and paper !

I got this problem when I run the extractor.lua: /usr/bin/luajit: /usr/share/lua/5.1/torch/File.lua:351: cuda runtime error (2) : out of memory at /tmp/luarocks_cutorch-scm-1-4656/cutorch/lib/THC/generic/THCStorage.cu:66

The issue seems to be related to the versions of torch or lua.

So could you please share the running environment config for extractor.lua?

Thanks a lot !

Issues replicating automatic eval results

Hi ratishsp,
Thanks for the great paper and especially for the accompanying code!

For some reason I'm having some issues replicating the results from the paper when using the automatic eval scripts. BLEU score is consistent, however I am getting much lower numbers for RG, CS, and CO (I've tested on the template, ws2017 model, and ncp+cc model linked on the github).

For example, when running non_rg_metrics.py -test on the aaai_19_rotowire_test.txt linked on the github, I get RG% = 0.778, CS P% = 0.286, CS R% = 0.014, and CO DLD% = 0.022. I'm getting a similar range of numbers for the rest of the models too.

Just wondering if there is anything else that was done during the score calculation. I followed the "automatic evaluation using IE metrics" section in the README (using the updated IE model with fix for number words and order of relations).

I also have a question about how the RG # is calculated. The extractor.lua gives a "nodup correct" number; do I divide that by the total number of generated summaries?

Thanks for the help,
Ethan

Train JC model

Hi Ratish,

AFAIK, this code runs the model with NCP+CC using the flag "-copy_attn". What is the significance of this reuse flag "-reuse_copy_attn"?

How do I train the model with NCP+JC?

A problem about copygenerator

Hi, I've been sticking to the problem for five hours. I don't know how to solve the problem. Could you give me some advice? Thank you

Use which type of data to calculate BLEU? Content plan or summary?

Hi Ratish,
Seems the code doesn't contain BLEU part. I have problems about how to calculate BLEU. I used the BLEU tools and inputed gold summaries and generate summaries, and got very high BLEU scores(about 40).
I also tried content plan tuples files, and got high scores too(about 32).

How to generate the file named "transform_gen/roto-gold-val-beam5_gens.h5-tuples.txt"

Hello, thank you so much for sharing your work.

python non_rg_metrics.py transform_gen/roto-gold-val-beam5_gens.h5-tuples.txt transform_gen/roto_stage1_$IDENTIFIER-beam5_gens.h5-tuples.txt

I want to ask one question about the file named "transform_gen/roto-gold-val-beam5_gens.h5-tuples.txt". Is it generated by the IE approach? I don't find it in the rotowire directory.

Version of Python and libraies

Hi,
Thanks for the code and the paper.
For the preprocess.py, it has a FileNotFoundError for 'roto.train.1.pt' occurs in line 113(torch.save(dataset,pt_file).
Could you please help me take a look at this?
Thanks.

Run into TypeError while trying to train the model

Hello! Currently I was following all the steps in the readme file. While running the model training part I ran into some TypeError. Here are the details:

python version: 2.7.16
torch version: 10.1
torchtext version: 0.2.3
CUDA version: 10.1

When running this:
python train.py -data $BASE/preprocess/roto -save_model $BASE/gen_model/$IDENTIFIER/roto -encoder_type1 mean -decoder_type1 pointer -enc_layers1 1 -dec_layers1 1 -encoder_type2 brnn -decoder_type2 rnn -enc_layers2 2 -dec_layers2 2 -batch_size 5 -feat_merge mlp -feat_vec_size 600 -word_vec_size 600 -rnn_size 600 -seed 1234 -start_checkpoint_at 4 -epochs 25 -optim adagrad -learning_rate 0.15 -adagrad_accumulator_init 0.1 -report_every 100 -copy_attn -truncated_decoder 100 -gpuid $GPUID -attn_hidden 64 -reuse_copy_attn -start_decay_at 4 -learning_rate_decay 0.97 -valid_batch_size 5

I got the following message:
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
/home/aiden/anaconda/anaconda2/lib/python2.7/site-packages/torch/nn/modules/rnn.py:54: UserWarning: dropout option adds dropout after all but last recurrent layer, so non-zero dropout expects num_layers greater than 1, but got dropout=0.3 and num_layers=1
"num_layers={}".format(dropout, num_layers))
/home/aiden/anaconda/anaconda2/lib/python2.7/site-packages/torch/nn/_reduction.py:46: UserWarning: size_average and reduce args will be deprecated, please use reduction='sum' instead.
warnings.warn(warning.format(ret))
Experiment 22-4.4 using attn_dim of 64
Loading train dataset from ../boxscore-data//preprocess/roto.train.1.pt, number of examples: 3371

• vocabulary size. source1 = 1164; target1 = 391, source2 = 956; target2 = 9902
• src feature 0 size = 702
• src feature 1 size = 39
• src feature 2 size = 4
  Building model...
  Intializing model parameters.
  Intializing model parameters.
  NMTModel(
  (encoder): MeanEncoder(
  (embeddings): Embeddings(
  (make_embedding): Sequential(
  (emb_luts): Elementwise(
  (0): Embedding(1164, 600, padding_idx=1)
  (1): Embedding(702, 600, padding_idx=1)
  (2): Embedding(39, 600, padding_idx=1)
  (3): Embedding(4, 600, padding_idx=1)
  )
  (mlp): Sequential(
  (0): Linear(in_features=2400, out_features=600, bias=True)
  (1): ReLU()
  )
  )
  )
  (dropout): Dropout(p=0.3)
  (attn): GlobalSelfAttention(
  (transform_in): Sequential(
  (0): Linear(in_features=600, out_features=64, bias=True)
  (1): ELU(alpha=0.1)
  )
  (linear_in): Linear(in_features=64, out_features=64, bias=False)
  (linear_out): Linear(in_features=1200, out_features=600, bias=False)
  (sm): Softmax()
  (tanh): Tanh()
  (dropout): Dropout(p=0.3)
  )
  )
  (decoder): PointerRNNDecoder(
  (embeddings): Embeddings(
  (make_embedding): Sequential(
  (emb_luts): Elementwise(
  (0): Embedding(391, 600, padding_idx=1)
  )
  )
  )
  (dropout): Dropout(p=0.3)
  (rnn): LSTM(600, 600, dropout=0.3)
  (attn): PointerAttention(
  (linear_in): Linear(in_features=600, out_features=600, bias=False)
  (sm): LogSoftmax()
  )
  )
  (generator): Sequential(
  (0): Linear(in_features=600, out_features=391, bias=True)
  (1): LogSoftmax()
  )
  )
  NMTModel(
  (encoder): RNNEncoder(
  (embeddings): Embeddings(
  (make_embedding): Sequential(
  (emb_luts): Elementwise(
  (0): Embedding(956, 600, padding_idx=1)
  (1): Embedding(547, 600, padding_idx=1)
  (2): Embedding(33, 600, padding_idx=1)
  (3): Embedding(4, 600, padding_idx=1)
  )
  (mlp): Sequential(
  (0): Linear(in_features=2400, out_features=600, bias=True)
  (1): ReLU()
  )
  )
  )
  (rnn): LSTM(600, 300, num_layers=2, dropout=0.3, bidirectional=True)
  )
  (decoder): InputFeedRNNDecoder(
  (embeddings): Embeddings(
  (make_embedding): Sequential(
  (emb_luts): Elementwise(
  (0): Embedding(9902, 600, padding_idx=1)
  )
  )
  )
  (dropout): Dropout(p=0.3)
  (rnn): StackedLSTM(
  (dropout): Dropout(p=0.3)
  (layers): ModuleList(
  (0): LSTMCell(1200, 600)
  (1): LSTMCell(600, 600)
  )
  )
  (attn): GlobalAttention(
  (linear_in): Linear(in_features=600, out_features=600, bias=False)
  (linear_out): Linear(in_features=1200, out_features=600, bias=False)
  (sm): Softmax()
  (tanh): Tanh()
  )
  )
  (generator): CopyGenerator(
  (linear): Linear(in_features=600, out_features=9902, bias=True)
  (linear_copy): Linear(in_features=600, out_features=1, bias=True)
  )
  )
• number of parameters: 7062951
  ('encoder: ', 3348560)
  ('decoder: ', 3714391)
• number of parameters: 26876703
  ('encoder: ', 6694200)
  ('decoder: ', 20182503)
  Making optimizer for training.
  Making optimizer for training.

Start training...

• number of epochs: 25, starting from Epoch 1
• batch size: 5

Loading train dataset from ../boxscore-data//preprocess/roto.train.1.pt, number of examples: 3371
/home/aiden/anaconda/anaconda2/lib/python2.7/site-packages/torch/nn/functional.py:1386: UserWarning: nn.functional.sigmoid is deprecated. Use torch.sigmoid instead.
warnings.warn("nn.functional.sigmoid is deprecated. Use torch.sigmoid instead.")
Traceback (most recent call last):
File "train.py", line 454, in
main()
File "train.py", line 446, in main
train_model(model1, model2, fields, optim1, optim2, data_type, model_opt)
File "train.py", line 256, in train_model
train_stats, train_stats2 = trainer.train(train_iter, epoch, report_func)
File "/home/aiden/files/harvardnlp/data2text-plan-py/onmt/Trainer.py", line 181, in train
report_stats, total_stats2, report_stats2, normalization)
File "/home/aiden/files/harvardnlp/data2text-plan-py/onmt/Trainer.py", line 345, in _gradient_accumulation
self.model(src, tgt, src_lengths, dec_state)
File "/home/aiden/anaconda/anaconda2/lib/python2.7/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/home/aiden/files/harvardnlp/data2text-plan-py/onmt/Models.py", line 684, in forward
memory_lengths=lengths)
File "/home/aiden/anaconda/anaconda2/lib/python2.7/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/home/aiden/files/harvardnlp/data2text-plan-py/onmt/Models.py", line 339, in forward
attns[k] = torch.stack(attns[k])
TypeError: stack(): argument 'tensors' (position 1) must be tuple of Tensors, not Tensor

When I check Models.py, I got this from line 339 onwards:
for k in attns:
attns[k] = torch.stack(attns[k])

And this from _run_forward_pass():
# Calculate the attention.
p_attn = self.attn(rnn_output.transpose(0, 1).contiguous(), memory_bank.transpose(0, 1), memory_lengths=memory_lengths)
attns["std"] = p_attn

# decoder_outputs = self.dropout(decoder_outputs)
return decoder_final, None, attns

Is there anything wrong with _run_forward_pass()? Thanks.

AttributeError: 'Field' object has no attribute 'vocab'

when i run the stage2,

BASE=.
MODEL_PATH=$BASE/gen_model/$IDENTIFIER/roto_stage1_acc_73.2760_ppl_2.9340_e9.pt
MODEL_PATH2=$BASE/gen_model/$IDENTIFIER/roto_stage2_acc_57.7477_ppl_7.8817_e24.pt

python translate.py
-model $MODEL_PATH
-model2 $MODEL_PATH2
-src1 $BASE/rotowire/inf_src_valid.txt
-tgt1 $BASE/gen/roto_stage1_$IDENTIFIER-beam5_gens.txt
-src2 $BASE/gen/roto_stage1_inter_$IDENTIFIER-beam5_gens.txt
-output $BASE/gen/roto_stage2_$IDENTIFIER-beam5_gens.txt
-batch_size 10
-max_length 850
-min_length 150
-gpu 0

it reports this error, i don't know why.

CUDNN_STATUS_NOT_SUPPORTED problem when running extractor.lua for retraining convolutional extraction model

Hi~Need some help here. When retraining convolution IE model using extractor.lua, I met Error in CuDNN: CUDNN_STATUS_NOT_SUPPORTED (cudnnSetConvolutionNdDescriptor)
I wonder whether is cuda version problem. I currently uses CUDA 9.0 and CUDNN 5. I try CUDA8.0 and CUDA7.5, still can't work. Error info:

Found Environment variable CUDNN_PATH = /home/work/cudnn/cudnn_v5/cuda/lib64/libcudnn.so.5
epoch 1 lr: 0.7
~/torch/install/bin/lua: .../torch/install/share/lua/5.2/nn/Container.lua:67:
In 3 module of nn.Sequential:
In 2 module of nn.ConcatTable:
In 1 module of nn.Sequential:
.../torch/install/share/lua/5.2/cudnn/init.lua:162: Error in CuDNN: CUDNN_STATUS_NOT_SUPPORTED (cudnnSetConvolutionNdDescriptor)

Thanks

Load model problems

Hello, ratishsp!
I find the code can't load two checkpoint files at the same time. The parameter 'train_from' can pass in only one value. How do you load the checkpoints from the previous training?