qingzhong1/tupleinfonce

TupleInfoNCE ICCV21

TupleInfoNCE

Contrastive Multimodal Fusion with TupleInfoNCE

ICCV 2021

Yunze Liu, Qingnan Fan, Shanghang Zhang, Hao Dong, Thomas Funkhouser, Li Yi*

Official PyTorch code for TupleInfoNCE, a novel contrastive learning objective for multi-modal learning. It contrasts tuples based not only on positive and negative correspondences but also by composing new negative tuples using modalities describing different scenes.

Citation

If you find this code useful for your research, please consider citing:

@inproceedings{liu2021contrastive,
  title={Contrastive multimodal fusion with tupleinfonce},
  author={Liu, Yunze and Fan, Qingnan and Zhang, Shanghang and Dong, Hao and Funkhouser, Thomas and Yi, Li},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  pages={754--763},
  year={2021}
}

Acknowledgement

Our project structure is borrowed from ESANet, so we thank the authors for their wonderful work. We also thank the anonymous reviewers for their constructive feedback.

License

MIT