fesvhtr/cuva

[CVPR 2024] Official repository of the paper "Uncovering What, Why and How: A Comprehensive Benchmark for Causation Understanding of Video Anomaly"

Uncovering What, Why and How: A Comprehensive Benchmark for Causation Understanding of Video Anomaly

The official repo for Uncovering What, Why and How: A Comprehensive Benchmark for Causation Understanding of Video Anomaly [CVPR2024].
This repository is still under maintenance. The code for the partial ablation experiment on A-Guardian is still being organized.
If you have any questions please contact [email protected].

Introduction

We present a comprehensive benchmark for Causation Understanding of Video Anomaly (CUVA). We also introduce MMEval, a novel evaluation metric designed to better align with human preferences for CUVA. Then we propose a novel prompt-based method that can serve as a baseline approach for the challenging CUVA.

Get Start

git clone https://github.com/fesvhtr/CUVA.git
conda create -n cuva python=3.8
pip install -r requirements.txt
conda activate cuva

CUVA Benchmark

CUVA Dataset

Please download the dataset from hf. There are 4 zip files and 1 json file in the dataset, unzip them and put them in the data folder.

Inference with Video-ChatGPT + A-Guardian

export PYTHONPATH="./:$PYTHONPATH"
cd /CUVA/Models/Video-ChatGPT/video_chatgpt/CUVA
./inference_CUVA.sh

Classic Evaluation

Refer to repo QA-Eval

git clone https://github.com/fesvhtr/QA-Eval
python eval.py

Evaluation with MMEval

export PYTHONPATH="./:$PYTHONPATH"
cd /CUVA/Models/Video-ChatGPT/video_chatgpt/CUVA
./mmEval_demo.sh

Multiple reasoning and evaluation

Modify and run CUVA.py and mmEval.py in the CUVA folder.

Acknowledgement

Sincere thanks to Video-chatGPT, VideoChat, mPlug, Otter, VideoLLaMA, Univtg and others for their excellent work.

Cite

If you find our work useful for your research, please consider citing:

@misc{du2024uncovering,
      title={Uncovering What, Why and How: A Comprehensive Benchmark for Causation Understanding of Video Anomaly}, 
      author={Hang Du and Sicheng Zhang and Binzhu Xie and Guoshun Nan and Jiayang Zhang and Junrui Xu and Hangyu Liu and Sicong Leng and Jiangming Liu and Hehe Fan and Dajiu Huang and Jing Feng and Linli Chen and Can Zhang and Xuhuan Li and Hao Zhang and Jianhang Chen and Qimei Cui and Xiaofeng Tao},
      year={2024},
      eprint={2405.00181},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

License

CUVA is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License (CC BY-NC-SA 4.0).

cuva's Issues

Filter out the Traffi Accidents, Traffic Violations samples, and Pedestrain Incidents

Thank you very much for your amazing work!

I wonder given the cuva_gt_cleaned.json and cuva_test.json, how can I locate the samples with only Traffi Accidents, Traffic Violations samples, and Pedestrain Incidents? Are there any labels marking this?

Thank you very much for your support in adcanve!

Best,
Zhenghao

About Voting_new.json

Hi Hang Du, can you tell me what is Voting_new.json used for?

Thanks!