willyfh/awesome-video-text-datasets

A curated list of video-text datasets in a variety of languages. These datasets can be used for video captioning (video description) or video retrieval.

Awesome Video-Text Datasets

A curated list of video-text datasets in a variety of languages. These datasets can be used for video captioning (video description) or video retrieval. Please feel free to send a pull request to update the list and contribute your changes.

In a survey paper, a list of video-text datasets is often presented. However, even if the reference papers for the datasets are provided, sometimes it can be not easy to find the datasets due to the missing information on where the location of the datasets exactly. Moreover, the existing survey papers commonly only focus on the datasets with monolingual English captions. This repository is made to help researchers in finding video-text datasets for any language, including multilingual datasets.

For each category, the dataset is ordered by the publication year in descending order.

Open Domain

• MSVD-Indonesian [paper][dataset]
  Language: Indonesian | Audio: No | Year: 2023
• ChinaOpen [paper][dataset]
  Language: Chinese, English | Audio: Yes | Year: 2023
• VideoCC [paper][dataset]
  Language: English | Audio: Yes | Year: 2022
• MSR-VTT-Hindi [paper][dataset]
  Language: Hindi | Audio: Yes | Year: 2021
• MSVD-Turkish [paper][dataset]
  Language: English, Turkish | Audio: No | Year: 2021
• VATEX [paper][dataset]
  Language: English, Chinese | Audio: Yes | Year: 2019
• MSR-VTT-it [paper][dataset]
  Language: English, Italian | Audio: Yes | Year: 2019
• MSVD-CN [dataset]
  Language: Chinese | Audio: No | Year: 2018
• ActivityNet Captions [paper][dataset]
  Language: English | Audio: Yes | Year: 2017
• MSR-VTT [paper][dataset]
  Language: English | Audio: Yes | Year: 2016
• TGIF [paper][dataset]
  Language: English | Audio: No | Year: 2016
• MSVD [paper][dataset]
  Language: English | Audio: No | Year: 2011

Movie / TV Show

• TVC [paper][dataset]
  Language: English | Audio: Yes | Year: 2020
• TVR [paper][dataset]
  Language: English | Audio: Yes | Year: 2020
• LSMDC [paper][dataset]
  Language: English | Audio: Yes | Year: 2017
• MPII-MD [paper][dataset]
  Language: English | Audio: Yes | Year: 2015

Cooking

• YouCook2 [paper][dataset]
  Language: English | Audio: Yes | Year: 2018
• YouCook [paper][dataset]
  Language: English | Audio: No | Year: 2013

Instructional

• HowTo100M [paper][dataset]
  Language: English | Audio: Yes | Year: 2019

Indoor

• Charades [paper][dataset]
  Language: English | Audio: Yes | Year: 2016