caoyu-noob/embedding_augmentation

embedding_augmentation's Issues

Error when reproduce dailydialog dataset

Traceback (most recent call last):
  File "train.py", line 301, in <module>
    main()
  File "train.py", line 298, in main
    last_checkpoint_path, interrupt_checkpoint_path, log_dir, test_data_type=args.test_data_type)
  File "train.py", line 144, in training_procedure
    pointer_gen=args.pointer_gen)
  File "/home/hx/github项目/embedding_augmentation/model/trainer.py", line 97, in __init__
    train_sampler = RandomSampler(train_dataset)
  File "/home/hx/anaconda3/envs/torch1.4/lib/python3.7/site-packages/torch/utils/data/sampler.py", line 103, in __init__
    "value, but got num_samples={}".format(self.num_samples))
ValueError: num_samples should be a positive integer value, but got num_samples=0

I print the len(train_dataset) ,and its value is 0
Is the data process code wrong?

Some files are missing

Hi, I am very interested in your work and admire your open-sourced code.
I find some files, e.g., "vocab.bin", are missing and there are no introductions about how to generate these files.
Could you give some suggestions, or share your processed files?
Your contribution will be highly appreciated.
Thank you.

Error when reproduce personachat

  File "/home/hx/github项目/embedding_augmentation/new_metrics.py", line 68, in calc_nist_bleu
    nist_score = lines[-22].strip('\r').split()[3]
IndexError: list index out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "train.py", line 301, in <module>
    main()
  File "train.py", line 298, in main
    last_checkpoint_path, interrupt_checkpoint_path, log_dir, test_data_type=args.test_data_type)
  File "train.py", line 260, in training_procedure
    raise e
  File "train.py", line 258, in training_procedure
    risk_func=get_risk_metric_func(trainer_config.risk_metric))
  File "/home/hx/github项目/embedding_augmentation/model/trainer.py", line 506, in train
    func(epoch)
  File "train.py", line 224, in test_func
    model_trainer.test(metric_funcs, external_metrics_func, epoch)
  File "/home/hx/github项目/embedding_augmentation/model/trainer.py", line 488, in test
    self._eval_test(metric_funcs, external_metrics_func, epoch, inference, raw_entail_data=raw_entail_data)
  File "/home/hx/github项目/embedding_augmentation/model/trainer.py", line 443, in _eval_test
    external_metrics = external_metrics_func(full_references, full_predictions, epoch, is_best)
  File "train.py", line 176, in external_metrics_func
    rouge_l, f1_score, avg_length = nlp_metrics(references_file_path, predictions_file_path, root_path=log_dir)
  File "/home/hx/github项目/embedding_augmentation/new_metrics.py", line 622, in nlp_metrics
    nist, nist_bleu, nist_list, nist_bleu_list = calc_nist_bleu([ref_file], pred_file, fld_out=root_path)
  File "/home/hx/github项目/embedding_augmentation/new_metrics.py", line 78, in calc_nist_bleu
    print(error.decode())
AttributeError: 'NoneType' object has no attribute 'decode'