This repository contains source code for the paper titled: "Model-Agnostic Gender Debiased Image Captioning" (Accepted to CVPR 2023). [paper]
To set up your environment to run the project, follow these steps:
-
Requirements:
- PyTorch 1.11.0
- Torchvision 0.12.0
- CUDA 11.3
-
Install Dependencies: Run the following command to install the necessary packages:
pip install -r requirements.txt python3 -m spacy download en_core_web_sm
- Download COCO 2014 (train and validation sets) from https://cocodataset.org/.
- Place .pkl file of the output captions of an image captioning model in
Data
. Please refer tooscar_preds.pkl
inData
for the format.
Download the trained model from here and place it in Models
.
To generate debiased captions, run the following command:
python gpt2_generate_debiased_cap.py \
--pred_cap_path Data/oscar_preds.pkl \
--model_path Models/libra_final.pth \
--image_dir path/to/coco/val2014/directory \
--rand_test_ipt_mask True
--rand_mask_rate 0.2
If you use this project in your research or wish to refer to the baseline results published in the paper, please use the following .bib entry.
@inproceedings{hirota2023model,
title={Model-Agnostic Gender Debiased Image Captioning},
author={Hirota, Yusuke and Nakashima, Yuta and Garcia, Noa},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages={15191--15200},
year={2023}
}
- Debiased Caption Generation
- Training (Upcoming feature)
Feel free to contribute to this project or suggest improvements. Your feedback and contributions are greatly appreciated!