cambridgeltl/sapbert

[NAACL'21 & ACL'21] SapBERT: Self-alignment pretraining for BERT & XL-BEL: Cross-Lingual Biomedical Entity Linking.

sapbert's Issues

The evaluation data download link doesn't work

It says

Link temporarily disabled
This can happen when the link has been shared or downloaded too many times in a day.

Check again later and we’ll open access to more people.

Statistical significance test

Dear authors,

Could you please share the python script for statistical significance computation? I think you used T-test.

Thank you in advance

Entity Span --> CUI

Hi - what's the simplest way to go from entity span to CUI? It seems the HuggingFace model just gets you the [CLS] hidden state representation and you need to use that to find the nearest neighbor in UMLS. But it wasn't clear how to get that UMLS index

1. ko_1k_test_query_with_context.txt, 2. finetuning, and 3. inference api

Hi, I have a several questions

First, in xl_bel, I could find ko_1k_test_query_with_context.txt. Many sentences consisting the txt file are irrelevant with disease. I want to know why and how you used this txt file.

Second, we are trying to give a input text with Korean and output with English. Mainly dealing medical area will be
rheumatology. Can you tell me the procedure to implement this?

Finally, I asked in huggingface ( cambridgeltl/SapBERT-UMLS-2020AB-all-lang-from-XLMR ) that,
I have question about the output value of inference api in huggingface.
In my assumption,
Do row value related to disease id and column value related to input text?
I want to know the what row and column mean, and if row value is disease id as I thought, where can I find the exact disease name matching to those disease id?

Thanks

dictionary for custom dataset

Hello and thank you for sharing this project.

I want to evaluate this model on my custom dataset. How can I create a dictionary for that?

Details on fine-tuning data

Hello and thanks for sharing this great project.

Regarding fine-tuning of SapBERT, the README states

For finetuning on your customised dataset, generate data in the format of [...]
where entity_name_1 and entity_name_2 are synonym pairs (belonging to the same concept concept_id) sampled from a given labelled dataset.

Are there any examples on how this looks exactly for the datasets (NCBI Disease, Cometa, etc.) used in the evaluation?

requirements.txt doesn't help resolving requirements.

I cannot fulfill the correct requirements for executing your kindly prepared google-colab sheet. Below I summarized my failure.

First cell ERROR:

tensorflow 2.9.2 requires protobuf<3.20,>=3.9.2, but you have protobuf 3.20.3 which is incompatible.
tensorflow 2.9.2 requires tensorboard<2.10,>=2.9, but you have tensorboard 2.11.2 which is incompatible.

I don't know if the above error poses a problem, however, I get another
ERROR (something like):

ModuleNotFoundError: No module named 'torchtext.legacy'.

subsequently I downgrade (stackoverflow recommendation) using !pip install torchtext==0.10.0, giving me
ERROR:
torchvision 0.14.1+cu116 requires torch==1.13.1, but you have torch 1.9.0 which is incompatible.

So I upgrade to 1.13.1.
Then I get

     11 from torchtext.data.utils import RandomShuffler
     12 from .example import Example
---> 13 from torchtext.utils import download_from_url, unicode_csv_reader
     14 
     15 

ImportError: cannot import name 'unicode_csv_reader' from 'torchtext.utils' (/usr/local/lib/python3.8/dist-packages/torchtext/utils.py)

At some point I also get

torchvision 0.14.1+cu116 requires torch==1.13.1, but you have torch 1.7.0 which is incompatible.
torchtext 0.10.0 requires torch==1.9.0, but you have torch 1.7.0 which is incompatible.
torchaudio 0.13.1+cu116 requires torch==1.13.1, but you have torch 1.7.0 which is incompatible.

MedMentions Dictionary file created

Hello Team
Thank you for your great contribution. Can you please brief me on how was the medmentions dictionary file was created to run evaluations.

Thanks
Saranya

mention detection (how to?)

Hi SapBERT authors, this code is so valuable!

With this code, I can link biomedical entities to the UMLS entities. In my case, I want to give a sentence-query, like needed for the evaluations of the unsupervised trained model. I wonder how the mention detection was performed (not described in paper)?

Tokenizer

Is there any information on the tokenizer from HuggingFace?

tokenizer = AutoTokenizer.from_pretrained("cambridgeltl/SapBERT-from-PubMedBERT-fulltext")

I assume it's the same as PubmedBERT, which I presume is using an 'in-domain' vocabulary. Just would love confirmation! thanks