bekou/multihead_joint_entity_relation_extraction

Implementation of our papers Joint entity recognition and relation extraction as a multi-head selection problem (Expert Syst. Appl, 2018) and Adversarial training for multi-context joint entity and relation extraction (EMNLP, 2018).

multihead_joint_entity_relation_extraction's Issues

do not support min batch，only one sample one step

the best epoch

Firstly，thank you very much for your great job. I have some questions and hope to get your answer.

1. Train on the training set and evaluate on the dev set to obtain early stopping epoch python3 train_es.py
2. Train on the concatenated (train + dev) set and evaluate on the test set until either (1) the max epochs or (2) the early stopping limit (specified by train_es.py) is exceeded python3 train_eval.py

In the first step, we can get the best epoch based on the dev set. In my opinion, the best epoch may not be the best in the second step, because the training set change(train + dev). Another question, how do you get a stable result? I run many times and the result is different.
Looking forward for your reply, thank you!

数据集

你好，正在学习关系识别，没有数据。请问能否给分享您的数据。我的邮箱是[email protected]

how to do prediction?

thanks your good work, but I can know how can we achieve interactive prediction when input a sentence and output result

getEvaluator of class model of tf_utils.py

in line 18, should the default value of rel_chunk_eval be "boundaries"?

Evaluation: relation F1

Hi,

I am confused about the relation F1 in the paper.
For me, a triplet is correct only if the relation and all the tokens of head and tail are correct,
eg: (B-PERSON √, I-PERSON √, Born_in √, B-LOCATION √, I-LOCATION√), I will count it as correct_triplet += 1

Is it the relation F1 in your EMNLP paper? You said CoNLL04 baseline+AT Relation F1 = 61.95

Thank you

what's the difference between your multihead selection and table filling methods?

the table filling methods in https://www.aclweb.org/anthology/C16-1239:

and your multihead selection:

multihead_joint_entity_relation_extraction/tf_utils.py

Lines 301 to 302 in 9fcabf5

	rel_scores = self.getHeadSelectionScores(rel_input,
	dropout_keep_in_prob=dropout_fcl_rel_keep)

is so similar, I get puzzled for that I can't see any difference between multihead selection and table filling methods.

@bekou

About memory usage

Thank you very much for your great job！
I try to train this model on my data, but i found that it takes up a lot of memory. And as the training continues, the memory usage is slowly increasing. Is this normal? If I want to reduce the memory usage , where should I start?

License

Hello!

Could you please kindly add MIT license to this work?

About vector "left" , "right" and function “broadcasting”

Hi， I don't quite understand the use of the vector "left" and "right" and the function “broadcasting”, could you explain it more clearly？ thanks

About model training speed

Why is the model training speed very slow？ 1M training set, 100k test set, training test takes about a day, and uses 4 GPUs at the same time.

Are these the hyperparameters from the articles ?

Thanks again for your great work and especially for structuring a messy state-of-the-art.

Are the hyperparameters in the configs/ folder the ones you used to obtain scores reported in your articles? In particular, I think you don't mention using a multi-layer BiLSTM in your first article.

I fail to replicate your result without adversarial training on CoNLL04 when running your code.
The gap is quite significant since I obtain 82.2 NER / 58.4 RE against the reported 83.9 / 62.0.
I use train_eval.py with the best epoch from train_es.py.

Do you know what could cause that difference?

[error when trying to change batchsize]

First of all, thank you very much for such a great work.
Hmm, I am trying to run your code, but I found that the batchsize in "build_data.py" is "1", I tried to set it to "20", then I got an error which said "ValueError: setting an array element with a sequence."
Can you help me with that?

The model predicts the relationship of all entities as "N", that is, no relationship.

Because the data labeling strategy according to the multi-head selection model, most of the relationship labels are "N", which leads to the model biasing the prediction of "N". How to solve such a problem?

About ADE data set

Hello, I am a bioinformatics student.

Can I request a dataset of ADE adapted to the input format? Or can I request the open source code that processes these original dataset?

I guarantee that the above data or code is for the laboratory only and does not involve any commercial activities.

Best Regards,

why adding -u after python3?

config.batchsize!=1, raise error

when I set batchsize=16 or other num, raise error"ValueError: setting an array element with a sequence."in
"""
_, val, predicted_ner, actual_ner, predicted_rel, actual_rel, _, m_train = self.sess.run(
[operations.train_step, operations.obj, operations.predicted_op_ner, operations.actual_op_ner, operations.predicted_op_rel, operations.actual_op_rel, operations.score_op_rel,
operations.m_op], feed_dict=x_train)
"""
how can i fix it?

how to do prediction?

thx for your sharing, and i have a question about how to do prediction? thx

Why is the model worse after adding the adversarial training in CoNLL dataset?

我想问下，用自己的数据集替换掉CoNLL04的train,dev,test，为啥运行程序的时候报KeyError的错误

tf_utils.py BUG?

Thanks to the author's hard work, I have two questions about this code and I hope to get your answer.

• Is line 203 redundant?
• Why batch_size is specified as 1?

I found that when constructing model input data, there is only one doc content per input as input, but the model's code seems to support small batch input. If I construct a small batch of input data, the model will work properly.

Negative samples

I'd like to ask how to get the Negative samples? are they set randomly?

triples extraction question?

Hey, can this program be directly used to extract triples for a given raw sentence? It seems that in you test/dev file, the relation between entities must be known first. So it cannot directly deal with a raw sentence.

Saving and Restoring the model

Hello. I am newbie to tensorflow. How do I save the model so that I can use it later? I tried tf.train.Saver() within the session but it says ValueError: No variables to save

Saving the model and Prediction

How to do a prediction after training the model? i.e saving the model and using the saved model for prediction. Thanks!

ACE 04 Train / dev splits

Hello and thank you for your work,

In your article you say that you compute a random dev set of 15% of the data for each fold of ACE04. Could you release the list of documents selected for each fold?
Additionally, the datasets statistics such as number of sentences, mentions and relations could be helpful to ensure that the experimental setting is exactly reproduced.