yisunlp/dail

Code for Demonstration Augmentation for Zero-shot In-context Learning (ACL 2024, Findings).

Demonstration Augmentation for Zero-shot In-context Learning

This is the official project repository for our paper Demonstration Augmentation for Zero-shot In-context Learning (ACL 2024, Findings).

Quick start

Environment

You should run the following script to install the dependencies first.

conda create -n DAIL python==3.10
conda activate DAIL
pip install torch==2.1.0 torchvision==0.16.0 torchaudio==2.1.0 --index-url https://download.pytorch.org/whl/cu118
pip install transformers，sentence_transformers,openai

Get the results on MMLU

cd MMLU
python main.py --device {device} --model {model_path} --method {method}
# method should be in: (SelfICL, DAIL, zeroshot, fewshow).

Get the results on BBH

cd BBH
python main.py --device {device} --model {model_path} --method {method} --api_key {api_key}
# method should be in: (SelfICL, DAIL, zeroshot, fewshow).

dail's Issues

Open-source the code for the other baselines

Could you share the code for the other baselines you compared? I would be grateful if you could make it available. I am looking forward to your reply.

A discrepancy between the paper and the code regarding the deletion strategies

For the "FIFO" strategy, the paper says "Delete the samples that entered the bank earlier," but the code keeps "the samples that entered the bank earlier."

The same issue appears with the "diverse" strategy. Here’s the code snippet for reference (see attached screenshot).

Can you please clarify this? Thanks.