jetrunner/laprador

🦮 Code and pretrained models for Findings of ACL 2022 paper "LaPraDoR: Unsupervised Pretrained Dense Retriever for Zero-Shot Text Retrieval"

Code and pretrained models for Findings of ACL 2022 paper LaPraDoR: Unsupervised Pretrained Dense Retriever for Zero-Shot Text Retrieval.

In this repo, we provide codes for reproducing experiments on BEIR dataset in the zero-shot setting of the paper. For fine-tuning, please follow the instructions in BEIR repo.

Note that due to copyright restrictions, some datasets are not available.

Installation

git clone https://github.com/benchmarkir/beir.git
cp code/exact_search.py beir/beir/retrieval/search/dense/exact_search.py
cp code/bm25.py beir
cp code/laprador.py beir
cd beir
pip install -e .
pip install tensorflow==2.5.0

Download pre-processed BM25 index

# cd ./beir
wget https://github.com/JetRunner/LaPraDoR/releases/download/v0.1/BM25-result.zip
unzip BM25-result.zip

Retrieve with LaPraDoR

# Please refer to https://github.com/beir-cellar/beir#beers-available-datasets for dataset name and split
dataset=trec-covid
split=test
python laprador.py $dataset $split

Use your own dataset (requires Elasticsearch)

To use your own dataset, please process the data to the same format as BEIR and then index the dataset following bm25.py. You will need Elasticsearch to index the corpus.

Install Elasticsearch

Installation

Following this link to download Elasticsearch. Here, we use Ubuntu 18.04 as an example.

sudo apt update
sudo apt install apt-transport-https
sudo mkdir -p /usr/share/man/man1
sudo apt install openjdk-8-jdk
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
sudo sh -c 'echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" > /etc/apt/sources.list.d/elastic-7.x.list'
sudo apt update
sudo apt install elasticsearch

Start Elasticsearch

sudo systemctl enable elasticsearch.service # or sudo service elasticsearch enable
sudo systemctl start elasticsearch.service # or sudo service elasticsearch start

# You can verify that Elasticsearch is running by sending an HTTP request to port 9200 on localhost with the following curl command:
curl -X GET "localhost:9200/"

Citation

@inproceedings{xu2022laprador,
    title={{LaPraDoR}: Unsupervised Pretrained Dense Retriever for Zero-Shot Text Retrieval},
    author={Canwen Xu and Daya Guo and Nan Duan and Julian McAuley},
    booktitle={{ACL} 2022 (Findings)},
    year={2022}
}

laprador's Issues

Will training code be available?

Hello, it's a good work! But I can only find the inference code in your repo. Could you share the training code for reproducibility?

Which instructions for fine-tuning

Hi, thanks for the awesome work!

I saw this note in the README: "For fine-tuning, please follow the instructions in BEIR repo.". I looked through the BEIR repository, and it's not obvious to me which instructions you're referring to. Can you provide a little more insight how we might use the BEIR code for fine-tuning LAPRADOR? thanks.

Ist sie nicht quelloffen, weil die Ergebnisse schwer zu reproduzieren sind?

Ist nicht Open Source, weil die Ergebnisse schwer zu reproduzieren sind? Es ist ratsam, solche Arbeiten mit Ergebnissen ohne konsequente Umsetzung in Zukunft nicht mehr zu veröffentlichen.

When I want to reproduce your project

when I run code "!python laprador.py $dataset $split",
It occurred error "No sentence-transformers model found with name /root/.cache/torch/sentence_transformers/canwenxu_laprador. Creating a new one with MEAN pooling.". Please help me

Cannot load tokenizer from Huggingface/Sentence Transformers

for example, trying to load via transformers

from transformers import AutoModel, AutoTokenizer,
model_name = "canwenxu/laprador"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModel.from_pretrained(model_name)

results in the following error

---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
/var/folders/6f/87t1c2k13mx2sy9rbwt2knrw0000gn/T/ipykernel_24646/643650800.py in <module>
      2 model_name = "canwenxu/laprador"
      3 
----> 4 tokenizer = AutoTokenizer.from_pretrained(model_name)
      5 model = AutoModel.from_pretrained(model_name)

~/Library/Python/3.8/lib/python/site-packages/transformers/models/auto/tokenization_auto.py in from_pretrained(cls, pretrained_model_name_or_path, *inputs, **kwargs)
    485                     f"Tokenizer class {tokenizer_class_candidate} does not exist or is not currently imported."
    486                 )
--> 487             return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
    488 
    489         # Otherwise we have to be creative.

~/Library/Python/3.8/lib/python/site-packages/transformers/tokenization_utils_base.py in from_pretrained(cls, pretrained_model_name_or_path, *init_inputs, **kwargs)
   1742                 logger.info(f"loading file {file_path} from cache at {resolved_vocab_files[file_id]}")
   1743 
-> 1744         return cls._from_pretrained(
   1745             resolved_vocab_files,
   1746             pretrained_model_name_or_path,

~/Library/Python/3.8/lib/python/site-packages/transformers/tokenization_utils_base.py in _from_pretrained(cls, resolved_vocab_files, pretrained_model_name_or_path, init_configuration, use_auth_token, *init_inputs, **kwargs)
   1870         # Instantiate tokenizer.
   1871         try:
-> 1872             tokenizer = cls(*init_inputs, **init_kwargs)
   1873         except OSError:
   1874             raise OSError(

~/Library/Python/3.8/lib/python/site-packages/transformers/models/bert/tokenization_bert_fast.py in __init__(self, vocab_file, tokenizer_file, do_lower_case, unk_token, sep_token, pad_token, cls_token, mask_token, tokenize_chinese_chars, strip_accents, **kwargs)
    175         **kwargs
    176     ):
--> 177         super().__init__(
    178             vocab_file,
    179             tokenizer_file=tokenizer_file,

~/Library/Python/3.8/lib/python/site-packages/transformers/tokenization_utils_fast.py in __init__(self, *args, **kwargs)
    106         elif fast_tokenizer_file is not None and not from_slow:
    107             # We have a serialization from tokenizers which let us directly build the backend
--> 108             fast_tokenizer = TokenizerFast.from_file(fast_tokenizer_file)
    109         elif slow_tokenizer is not None:
    110             # We need to convert a slow tokenizer to build the backend

Exception: No such file or directory (os error 2)

Looks like the issue is from the tokenizer configs