dair-iitd/imojie

Neural generation model for Open Information Extraction

imojie's Issues

Train model with Multiple GPUs

Hi, thank you for your excellent work.

I was trying to reproduce the performance of baseline model imojie with multiple GPUs, so that I commented out the lines including "ipdb" code in "copy_seq2seq_bahdanu.py" file.
However, there was a trouble when I used multiple GPUs to train the model (using only one GPU was fine). Some key error messages were shown in bellow:

'''
Traceback (most recent call last):
File "allennlp_script.py", line 151, in
main()
File "allennlp_script.py", line 138, in main
overrides=args.overrides
File "/home/qj/imojie/allennlp/allennlp/commands/train.py", line 168, in train_model_from_file
cache_directory, cache_prefix)
File "/home/qj/imojie/allennlp/allennlp/commands/train.py", line 254, in train_model
metrics = trainer.train()
File "/home/qj/imojie/allennlp/allennlp/training/trainer.py", line 478, in train
train_metrics = self._train_epoch(epoch)
File "/home/qj/imojie/allennlp/allennlp/training/trainer.py", line 320, in _train_epoch
loss = self.batch_loss(batch_group, for_training=True, optimizer=self.optimizer)
File "/home/qj/imojie/allennlp/allennlp/training/trainer.py", line 256, in batch_loss
output_dict = training_util.data_parallel(batch_group, self.model, self._cuda_devices)
File "/home/qj/imojie/allennlp/allennlp/training/util.py", line 335, in data_parallel
outputs = parallel_apply(replicas, inputs, moved, used_device_ids)
File "/home/qj/miniconda3/lib/python3.7/site-packages/torch/nn/parallel/parallel_apply.py", line 86, in parallel_apply
output.reraise()
File "/home/qj/miniconda3/lib/python3.7/site-packages/torch/_utils.py", line 425, in reraise
raise self.exc_type(msg)
StopIteration: Caught StopIteration in replica 0 on device 0.
Original Traceback (most recent call last):
File "/home/qj/miniconda3/lib/python3.7/site-packages/torch/nn/parallel/parallel_apply.py", line 61, in _worker
output = module(*input, **kwargs)
File "/home/qj/miniconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/home/qj/imojie/imojie/models/copy_seq2seq_bahdanu.py", line 389, in forward
output_dict = self.forward_append(source_tokens, source_token_ids, source_to_target, metadata, target_tokens, target_token_ids, optimizer)
File "/home/qj/imojie/imojie/models/copy_seq2seq_bahdanu.py", line 288, in forward_append
output_dict = self.train_append(source_tokens, source_token_ids, source_to_target, metadata, target_tokens, target_token_ids, optimizer=optimizer)
File "/home/qj/imojie/imojie/models/copy_seq2seq_bahdanu.py", line 509, in train_append
state = self._encode({'tokens': append_tokens})
File "/home/qj/imojie/imojie/models/copy_seq2seq_bahdanu.py", line 633, in _encode
embedded_input = self._source_embedder(source_tokens)
File "/home/qj/miniconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/home/qj/imojie/allennlp/allennlp/modules/text_field_embedders/basic_text_field_embedder.py", line 131, in forward
token_vectors = embedder(*tensors, **forward_params_values)
File "/home/qj/miniconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/home/qj/imojie/allennlp/allennlp/modules/token_embedders/pretrained_transformer_embedder.py", line 28, in forward
return self.transformer_model(token_ids)[0]
File "/home/qj/miniconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/home/qj/miniconda3/lib/python3.7/site-packages/pytorch_transformers/modeling_bert.py", line 696, in forward
extended_attention_mask = extended_attention_mask.to(dtype=next(self.parameters()).dtype) # fp16 compatibility
StopIteration
'''

Can we fix this trouble and train model with multiple GPUs soon?

Standalone.py outputs only the last sentence

See fix (Pull request: #8)

Cannot reproduce model

Hi,

My goal is to reproduce the model with the defined settings in config file: imojie.json.
I ran as suggested: python3 allennlp_script.py --param_path imojie/configs/imojie.json --s models/imojie --mode train_test
As for the data, because it was a test run, I used the first 100 instances from the data file: data/train/4cr_qpbo_extractions.tsv

The issue can be seen below: it hangs after validating, and validation seems to be off. Could you give some direction here on what the problem could be?

INFO:allennlp.common.params:CURRENTLY DEFINED PARAMETERS: 
INFO:allennlp.common.params:trainer.optimizer.lr = 0.001
INFO:allennlp.common.registrable:instantiating registered subclass bert_adam of <class 'allennlp.training.optimizers.Optimizer'>
WARNING:pytorch_pretrained_bert.optimization:t_total value of -1 results in schedule not being applied
INFO:allennlp.common.params:trainer.num_serialized_models_to_keep = 2
INFO:allennlp.common.params:trainer.keep_serialized_model_every_num_seconds = None
INFO:allennlp.common.params:trainer.model_save_interval = None
INFO:allennlp.common.params:trainer.summary_interval = 100
INFO:allennlp.common.params:trainer.histogram_interval = None
INFO:allennlp.common.params:trainer.should_log_parameter_statistics = True
INFO:allennlp.common.params:trainer.should_log_learning_rate = False
INFO:allennlp.common.params:trainer.log_batch_size_period = None
WARNING:allennlp.training.trainer:You provided a validation dataset but patience was set to None, meaning that early stopping is disabled
INFO:allennlp.training.trainer:Beginning training.
INFO:allennlp.training.trainer:Epoch 0/7
INFO:allennlp.training.trainer:Peak CPU memory usage MB: 3613.788
INFO:allennlp.training.trainer:GPU 0 memory usage MB: 1639
INFO:allennlp.training.trainer:Training
  0%|          | 0/1 [00:00<?, ?it/s]INFO:imojie.dataset_readers.copy_seq2multiseq:Reading instances from lines in file at: data/train/4cr_qpbo_extractions.tsv
loss: 2.0916 ||: : 2it [00:06,  3.01s/it]                     
INFO:allennlp.training.trainer:Validating
  0%|          | 0/1 [00:00<?, ?it/s]INFO:imojie.dataset_readers.copy_seq2multiseq:Reading instances from lines in file at: data/dev/carb/extractions.tsv
carb_auc: 0.0000, carb_f1: 0.0000, carb_sum: 0.0000, loss: 0.0000 ||: : 2it [05:10, 154.74s/it]                     
carb_auc: 0.0000, carb_f1: 0.0000, carb_sum: 0.0000, loss: 0.0000 ||: : 3it [08:05, 164.27s/it]
carb_auc: 0.0000, carb_f1: 0.0000, carb_sum: 0.0000, loss: 0.0000 ||: : 5it [13:20, 160.18s/it]
carb_auc: 0.0000, carb_f1: 0.0000, carb_sum: 0.0000, loss: 0.0000 ||: : 7it [18:22, 155.49s/it]
carb_auc: 0.0000, carb_f1: 0.0000, carb_sum: 0.0000, loss: 0.0000 ||: : 8it [20:59, 155.89s/it]
carb_auc: 0.0000, carb_f1: 0.0000, carb_sum: 0.0000, loss: 0.0000 ||: : 14it [36:21, 149.70s/it]
carb_auc: 0.0000, carb_f1: 0.0000, carb_sum: 0.0000, loss: 0.0000 ||: : 15it [38:51, 149.89s/it]
carb_auc: 0.0000, carb_f1: 0.0000, carb_sum: 0.0000, loss: 0.0000 ||: : 19it [49:52, 162.80s/it]
carb_auc: 0.0000, carb_f1: 0.0000, carb_sum: 0.0000, loss: 0.0000 ||: : 20it [52:50, 158.55s/it]
Traceback (most recent call last):

Running pretrained model on a sentence

How can I use Imojie directly on a file of several sentences?

E.g. can I do something like this:

python script.py --file input_sentences.txt --output output_triples.txt

Update results with XLnet in place of BERT

https://huggingface.co/transformers/model_doc/xlnet.html

How to infer from the pretrained model?

Hi, I'm trying to get predictions in Jupyter Notebook and I can't get around this error. Any help is appreciated!

ERROR:allennlp.data.vocabulary:Namespace: bert
ERROR:allennlp.data.vocabulary:Token: [unused99]

KeyError Traceback (most recent call last)
in
----> 1 predictor.predict_json({'source':'32.7 % of all households were made up of individuals and 15.7 % had someone living alone who was 65 years of age or older .'})

/usr/local/lib/python3.6/dist-packages/allennlp/predictors/predictor.py in predict_json(self, inputs)
63 def predict_json(self, inputs: JsonDict) -> JsonDict:
64 instance = self._json_to_instance(inputs)
---> 65 return self.predict_instance(instance)
66
67 def json_to_labeled_instances(self, inputs: JsonDict) -> List[Instance]:

/usr/local/lib/python3.6/dist-packages/allennlp/predictors/predictor.py in predict_instance(self, instance)
178
179 def predict_instance(self, instance: Instance) -> JsonDict:
--> 180 outputs = self._model.forward_on_instance(instance)
181 return sanitize(outputs)
182

/usr/local/lib/python3.6/dist-packages/allennlp/models/model.py in forward_on_instance(self, instance)
124 torch.Tensors into numpy arrays and remove the batch dimension.
125 """
--> 126 return self.forward_on_instances([instance])[0]
127
128 def forward_on_instances(self,

/usr/local/lib/python3.6/dist-packages/allennlp/models/model.py in forward_on_instances(self, instances)
153 dataset.index_instances(self.vocab)
154 model_input = util.move_to_device(dataset.as_tensor_dict(), cuda_device)
--> 155 outputs = self.decode(self(**model_input))
156
157 instance_separated_output: List[Dict[str, numpy.ndarray]] = [{} for _ in dataset.instances]

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in call(self, *input, **kwargs)
545 result = self._slow_forward(*input, **kwargs)
546 else:
--> 547 result = self.forward(*input, **kwargs)
548 for hook in self._forward_hooks.values():
549 hook_result = hook(self, input, result)

/workdir/Desktop/rel/github/imojie/imojie/models/copy_seq2seq_bahdanu.py in forward(self, source_tokens, source_token_ids, source_to_target, metadata, target_tokens, target_token_ids, optimizer)
380 # not built at init function
381 # so need to introduce another function
--> 382 self.initialize()
383 self._initialized = True
384

/workdir/Desktop/rel/github/imojie/imojie/models/copy_seq2seq_bahdanu.py in initialize(self)
260 self.vocab._oov_token = '[unused99]'
261 self.vocab._padding_token = '[PAD]'
--> 262 self._oov_index = self.vocab.get_token_index(self.vocab._oov_token, self._target_namespace) # pylint: disable=protected-access
263 self._pad_index = self.vocab.get_token_index(self.vocab._padding_token, self._target_namespace) # pylint: disable=protected-access
264

/usr/local/lib/python3.6/dist-packages/allennlp/data/vocabulary.py in get_token_index(self, token, namespace)
635 else:
636 try:
--> 637 return self._token_to_index[namespace][self._oov_token]
638 except KeyError:
639 logger.error('Namespace: %s', namespace)

KeyError: '[unused99]'

This is what I've tried so far.

dataset_reader = CopySeq2SeqNetDatasetReader('bert')

config_file = os.path.join("github","imojie","imojie","configs","imojie.json")
with open(config_file) as f:
model_config = Params(json.load(f))

serialization_dir=os.path.join("github","imojie","models","imojie")
weights = os.path.join("github","imojie","models","imojie","model_state_epoch_7.th")

model = Model.load(config=model_config,serialization_dir=serialization_dir,weights_file=weights)

predictor = MemSeq2SeqPredictor(model=model,dataset_reader=dataset_reader)
predictor.predict_json({'source':'some text.'})

CaRB scores for OpenIE-4 and OpenIE-5

Hello,

Thank you for your effort on your paper and the related work.

Could you indicate me the causes of the difference in score between your work and the CaRB paper.

I've copied your matcher.py file and changed the carb.py file in the original car repo and find that the result is the same as reported by the Carb paper,
and when I replace the linient_tuple_match by your submited linient_tuple_match (indicated by the comment : # SUBMITTED VERSION OF LINIENT TUPLE MATCH), I get instead for OpenIE4 :
AUC: 0.348 Optimal (precision, recall, F1): (0.603, 0.518, 0.557) Zero Conf (precision, recall, F1): (0.591, 0.524, 0.555)

Is it because you changed the output file of OpenIE4 extractions ?

Thank you

Question for baseline implementations

Hi Keshav,

I have run through your baselines: IMoJIE (on OpenIE-4 bootstrapping) and CopyAttn+BERT (on OpenIE-4 bootstrapping). I would like to know how to implement other baselines such as CopyAtten + LSTM, CopyAtten + LSTM + iterative memory? I modified the config file ‘lstm_a’ and ‘lstm_s’ respectively. The training works fine but the inference cannot work.

Also may I know what is coverage and coverage + BERT model?

Thank you a lot,
Best regards,
KC

Is the pretrained model available?

Can you share the pretrained model? I'm trying to use the model to extract information from my own text inputs.

Requirements.txt issues

Hi -

I had to comment out the torch and sentencepiece lines to get it working. For the latter, I get the error

ERROR: Could not find a version that satisfies the requirement sentencepiece==0.1.85 (from -r requirements.txt (line 28)) (from versions: 0.0.0, 0.0.2, 0.0.3, 0.0.4, 0.0.5, 0.0.6, 0.0.7, 0.0.9, 0.1.0, 0.1.1, 0.1.2, 0.1.3, 0.1.83, 0.1.86, 0.1.90, 0.1.91, 0.1.92) ERROR: No matching distribution found for sentencepiece==0.1.85 (from -r requirements.txt (line 28))