caoyu-noob/multi-gpt2

The implementation of EMNLP2020-Findings paper "Pretrained Language Models for Dialogue Generation with Multiple Input Sources"

multi-gpt2's Issues

RuntimeError: _th_gather not supported on CUDAType for Bool

When I run the experiment as readme, the following error occursed.
Is the version of pytorch really 1.1.0 as stated in requirements.txt？
Traceback (most recent call last):
File "train.py", line 390, in
main()
File "train.py", line 387, in main
last_checkpoint_path, interrupt_checkpoint_path, log_dir, test_data_type=args.test_data_type)
File "train.py", line 330, in training_procedure
raise e
File "train.py", line 326, in training_procedure
risk_func=get_risk_metric_func(trainer_config.risk_metric))
File "/home/lys/workspace/Multi-GPT2-main/model/trainer.py", line 808, in train
func(epoch)
File "train.py", line 292, in test_func
model_trainer.test(metric_funcs, external_metrics_func, epoch)
File "/home/lys/workspace/Multi-GPT2-main/model/trainer.py", line 790, in test
self._eval_test(metric_funcs, external_metrics_func, epoch, inference)
File "/home/lys/workspace/Multi-GPT2-main/model/trainer.py", line 683, in _eval_test
metrics, metric_funcs, external_metrics_func, i)
File "/home/lys/workspace/Multi-GPT2-main/model/trainer.py", line 637, in _get_eval_predictions
predictions = self.model.inference(enc_contexts=enc_contexts)
File "/home/lys/workspace/Multi-GPT2-main/model/gpt2_model.py", line 1410, in inference
return self.beam_search(enc_contexts, return_beams, beam_starts)
File "/home/lys/workspace/Multi-GPT2-main/model/gpt2_model.py", line 1540, in beam_search
is_end = torch.gather(is_end, 1, beam_idxs)
RuntimeError: _th_gather not supported on CUDAType for Bool

issue in new_metrics.py

Hello, i try to use your code to train with these args:

!python train.py
--train_datasets datasets/ConvAI2/train_self_original.txt
--valid_datasets datasets/ConvAI2/valid_self_original.txt
--test_datasets datasets/ConvAI2/test_self_original.txt
--train_datasets_cache datasets/train_cache_gpt2
--valid_datasets_cache datasets/valid_cache_gpt2
--test_datasets_cache datasets/test_cache_gpt2
--model_type gpt2
--attention_fusion_type sw
--train_batch_size 4
--batch_split 2
--test_batch_size 2
--lr 5e-4
--extra_module_lr_rate 5.0
--shared_module 0
--shared_attention 0
--max_history_size 2
--n_epochs 1 \

But after the epoch ends i got the following issue in the "new_metrics.py" :

Test: 0% 0/3756 [00:00<?, ?it/s]10/06/2021 12:36:50 - INFO - root - Starting testing on Test dataset
Test: 100% 3756/3756 [11:00<00:00, 5.69it/s, s2s_loss=118, lm_loss=95.6, hits_acc=0, lm_ppl=6.37e+44, s2s_ppl=1.18e+55, f1_score=0]
Can't locate XML/Twig.pm in @inc (you may need to install the XML::Twig module) (@inc contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.26.1 /usr/local/share/perl/5.26.1 /usr/lib/x86_64-linux-gnu/perl5/5.26 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.26 /usr/share/perl/5.26 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at metrics/mteval-v14c.pl line 7.
BEGIN failed--compilation aborted at metrics/mteval-v14c.pl line 7.
mteval-v14c.pl returns unexpected message
cmd = ['perl', 'metrics/mteval-v14c.pl', '-s', 'runs/Oct06_12-36-30_gpt2_persona_sw_nm_na/src.xml', '-t', 'runs/Oct06_12-36-30_gpt2_persona_sw_nm_na/hyp.xml', '-r', 'runs/Oct06_12-36-30_gpt2_persona_sw_nm_na/ref.xml']

Traceback (most recent call last):
File "/content/drive/MyDrive/Multi-GPT2/new_metrics.py", line 63, in calc_nist_bleu
nist_score = lines[-22].strip('\r').split()[3]
IndexError: list index out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "train.py", line 390, in
main()
File "train.py", line 387, in main
last_checkpoint_path, interrupt_checkpoint_path, log_dir, test_data_type=args.test_data_type)
File "train.py", line 330, in training_procedure
raise e
File "train.py", line 326, in training_procedure
risk_func=get_risk_metric_func(trainer_config.risk_metric))
File "/content/drive/MyDrive/Multi-GPT2/model/trainer.py", line 780, in train
func(-1)
File "train.py", line 292, in test_func
model_trainer.test(metric_funcs, external_metrics_func, epoch)
File "/content/drive/MyDrive/Multi-GPT2/model/trainer.py", line 750, in test
self._eval_test(metric_funcs, external_metrics_func, epoch, inference)
File "/content/drive/MyDrive/Multi-GPT2/model/trainer.py", line 690, in _eval_test
external_metrics = external_metrics_func(full_references, full_predictions, epoch, is_best)
File "train.py", line 244, in external_metrics_func
rouge_l, f1_score, avg_length = nlp_metrics(references_file_path, predictions_file_path, root_path=log_dir)
File "/content/drive/MyDrive/Multi-GPT2/new_metrics.py", line 565, in nlp_metrics
nist, nist_bleu, nist_list, nist_bleu_list = calc_nist_bleu([ref_file], pred_file, fld_out=root_path)
File "/content/drive/MyDrive/Multi-GPT2/new_metrics.py", line 73, in calc_nist_bleu
print(error.decode())
AttributeError: 'NoneType' object has no attribute 'decode'

Could you please explain me why it happens that if you have any idea? thanks.

git.exc.InvalidGitRepositoryError

Hello! When i ran "python train.py --train_datasets datasets/C......",i had encountered the following problem:

Traceback (most recent call last):
File "train.py", line 13, in
from config import get_trainer_config
File "D:\papers\Multi-GPT2-main\Multi-GPT2-main\config.py", line 9, in
repo = git.Repo(search_parent_directories=True)
File "D:\environment\python\testvitualenv\multigpt2\lib\site-packages\git\repo\base.py", line 220, in init
self.working_dir = self._working_tree_dir or self.common_dir # type: Optional[PathLike]
File "D:\environment\python\testvitualenv\multigpt2\lib\site-packages\git\repo\base.py", line 303, in common_dir
raise InvalidGitRepositoryError()
git.exc.InvalidGitRepositoryError

What can i do to solve this problem? thanks!

requirements

Greetings and great job .... could you provide a txt with all the requirements ? it would be useful thanks .

ValueError: Reference at 'HEAD' does not exist