westlake-repl/microlens

A Large Short-video Recommendation Dataset with Raw Text/Audio/Image/Videos (Talk Invited by DeepMind).

MicroLens: A Content-Driven Micro-Video Recommendation Dataset at Scale

Quick Links: 🗃️Dataset | 📭Citation | 🛠️Code | 🚀Baseline Evaluation | 🤗Video Understanding Meets Recommender Systems | 💡News

Talks & Slides: Invited Talk by Google DeepMind & YouTube & Alipay (Slides)

Dataset

Download links: https://recsys.westlake.edu.cn/MicroLens-50k-Dataset/ and https://recsys.westlake.edu.cn/MicroLens-100k-Dataset/

Email us if you find the link is not available.

News

• 2024/04/15: Our dataset has been added to the MMRec framework, see https://github.com/enoche/MMRec/tree/master/data.

• 2024/04/04: We have provided extracted multi-modal features (text/images/videos) of MicroLens-100k for multimodal recommendation tasks, see https://recsys.westlake.edu.cn/MicroLens-100k-Dataset/extracted_modality_features/.

• 2024/03/01: We have updated the command example for automatically downloading all videos, see https://github.com/westlake-repl/MicroLens/blob/master/Downloader/quick_download.txt.

• 2023/10/21: We also release a subset of our MicroLens with extracted features for multimodal fairness recommendation, which can be downloaded from https://recsys.westlake.edu.cn/MicroLens-Fairness-Dataset/

• 2023/09/28: We have temporarily released MicroLens-50K (50,000 users) and MicroLens-100K (100,000 users) along with their associated multimodal data, including raw text, images, audio, video, and video comments. You can access them through the provided link. To acquire the complete MicroLens dataset, kindly reach out to the corresponding author via email. If you have an innovative idea for building a foundational recommendation model but require a large dataset and computational resources, consider joining our lab as an intern. We can provide access to 100 NVIDIA 80G A100 GPUs and a billion-level dataset of user-video/image/text interactions.

Citation

If you use our dataset, code or find MicroLens useful in your work, please cite our paper as:

@article{ni2023content,
  title={A Content-Driven Micro-Video Recommendation Dataset at Scale},
  author={Ni, Yongxin and Cheng, Yu and Liu, Xiangyan and Fu, Junchen and Li, Youhua and He, Xiangnan and Zhang, Yongfeng and Yuan, Fajie},
  journal={arXiv preprint arXiv:2309.15379},
  year={2023}
}

⚠️ Caution: It's prohibited to privately modify the dataset and then offer secondary downloads. If you've made alterations to the dataset in your work, you are encouraged to open-source the data processing code, so others can benefit from your methods. Or notify us of your new dataset so we can put it on this Github with your paper.

Code

We have released the codes for all algorithms, including VideoRec (which implements all 15 video models in this project), IDRec, and VIDRec. For more details, please refer to the following paths: "Code/VideoRec", "Code/IDRec", and "Code/VIDRec". Each folder contains multiple subfolders, with each subfolder representing the code for a baseline.

Special instructions on VideoRec

In VideoRec, if you wish to switch to a different training mode, please execute the following Python scripts: 'run_id.py', 'run_text.py', 'run_image.py', and 'run_video.py'. For testing, you can use 'run_id_test.py', 'run_text_test.py', 'run_image_test.py', and 'run_video_test.py', respectively. Please see the path "Code/VideoRec/SASRec" for more details.

Before running the training script, please make sure to modify the dataset path, item encoder, pretrained model path, GPU devices, GPU numbers, and hyperparameters. Additionally, remember to specify the best validation checkpoint (e.g., 'epoch-30.pt') before running the test script.

Note that you will need to prepare an LMDB file and specify it in the scripts before running image-based or video-based VideoRec. To assist with this, we have provided a Python script for LMDB generation. Please refer to 'Data Generation/generate_cover_frames_lmdb.py' for more details.

Special instructions on IDRec and VIDRec

In IDRec, see IDRec\process_data.ipynb to process the interaction data. Execute the following Python scripts: 'main.py' under each folder to run the corresponding baselines. The data path, model parameters can be modified by changing the yaml file under each folder.

Environments

python==3.8.12
Pytorch==1.8.0
cudatoolkit==11.1
torchvision==0.9.0
transformers==4.23.1

Baseline_Evaluation

Video_Understanding_Meets_Recommender_Systems

Ad

The laboratory is hiring research assistants, interns, doctoral students, and postdoctoral researchers. Please contact the corresponding author for details.

实验室招聘科研助理，实习生，博士生和博士后，请联系通讯作者。

microlens's Issues

How does the dataset determine whether the user is interacting with the video, playing it in full or playing it for a certain length of time?

Crush

cant download, it crush.

[Issue] Problems Encountered Running `run_video.py`

Description

I have successfully downloaded the MicroLens dataset and am currently running the code provided by the authors. However, I've encountered several issues during the execution.

Issues

1. Clarification Needed on Dataset Selection
   In the file MicroLens/Code/VideoRec/SASRec/run_video.py, there is a line:

   max_video_no = 91717 # 34321 for 10wu, 91717 for 100wu

   I am unsure what "10wu" and "100wu" refer to. Which setting should I use for the MicroLens-100k dataset?

2. Error Using x3d-s Video Model
   When running the code with the x3d-s video model, strictly following the parameters specified in the paper, the following traceback error occurs:

RuntimeError: input image (T: 5 H: 7 W: 7) smaller than kernel size (kT: 13 kH: 5 kW: 5)

The error points to an issue with the pooling operation in the video model. It seems the input image dimensions are too small for the kernel size. How should I adjust the kernel size or the input dimensions?

4. Instability in Metrics When Using video-mae-base Model

Using the video-mae-base as the video model, I observed that the learning metrics Hit10 and nDCG10 fluctuate significantly during training, with a trend of occasionally approaching zero. What might be causing this instability, and how can it be resolved?

Request for Assistance

I would appreciate any insights or recommendations on addressing these issues, especially with the right dataset settings for MicroLens-100k, the handling of input dimensions for the x3d-s model, and strategies for stabilizing training metrics with the video-mae-base model.

Thank you very much!