detecting-invisible-people's Introduction

Detecting Invisible People

[ICCV 2021 Paper] [Website]

Tarasha Khurana, Achal Dave, Deva Ramanan

Introduction

This repository contains code for Detecting Invisible People. We extend the original DeepSORT algorithm to localize people even while they are completely occluded in a video. See the arXiv preprint for more information.

Dependencies

Create a conda environment with the given environment.yml file.

conda env create -f environment.yml

Preprocessing

The code expects the directory structure of your dataset in the MOT Challenge data format, which is approximately like the following:

MOT17/
-- train/
---- seq_01/
------ img1/
------ img1Depth/
------ gt/
------ det/
...
-- test/
---- seq_02/
------ img1/
------ img1Depth/
------ det/

The folder img1Depth stores the normalized disparity in .npy format. See Note. Originally, the paper runs the method on depth given by the MegaDepth depth estimator.

Given the above folder structure, generate the appearance features for your detections as described in the DeepSORT repository.

Running the method

The script run_forecast_filtering.sh will run the method with hyperparameters used in the paper. It will produce output .txt files in the MOT Challenge submission format. The bashscript has support for computing the metrics, but this has not been verified. Run the bashscript like the following:

bash run_forecast_filtering.sh experimentName

Note that in order to speed up code release, the dataset, preprocessed detections and output file paths are hardcoded in the files and will have to be manually changed.

Citing Detecting Invisible People

If you find this code useful in your research, please consider citing the following paper:

@inproceedings{khurana2021detecting,
  title={{Detecting Invisible People}},
  author={Khurana, Tarasha and Dave, Achal and Ramanan, Deva},
  booktitle={{IEEE/CVF International Conference on Computer Vision (ICCV)}},
  year={2021}
}

Warning

This is only the starter code that has not been cleaned for release. It currently only has verified support for running the method described in Detecting Invisible People, with the output tracks written in the MOT Challenge submission format. Although Top-k metric's code has been provided, this codebase does not guarantee support for the metric yet.

The hope is that you are able to benchmark this method for your CVPR 2022 submission and compute your own metrics on the method's output. If the method code does not work, please open an issue.

Note

Although it is easy to run any monocular depth estimator and store their output (usually given as disparity) in an .npy file, I have added a script in tools/demo_images.py which can save the .npy files for you. Note that this script should be run after setting up the MegaDepth codebase and copying this file to its root directory. I will likely also release my own depth maps for the MOT17 dataset over the Halloween weekend.

If you try to run the metrics, I have given my groundtruth JSON (as expected by pycocotools).

detecting-invisible-people's People

Contributors

Stargazers

Watchers

detecting-invisible-people's Issues

Some questions about the project

I apologize for my previous rudeness, but I do have some issues that I would like to discuss rationally with you. as follows.
First, I think the code in the paper that is open sourced for others to study should at least be available or error-free, not just use github as our own code repository.

There are a lot of code writing errors in this project.
1.Where is the definition or introduction of the function get_mask_from_bbox() in line 83 of the tracker.py file in the deep_sort folder?
2.What’s the meaning of the parameter “tn=self.tn” of the function update() in line 95?
3.Where is the definition or call of the function compute_mean_depth_from_mask() in line 434?
4.Where is the definition of the variable only_extrapolate in line 161 of the tracker_mask.py file?
5.Does the function project() require a parameter called temporal_noise in line 301 of the kalman_filter.py file?
6.and so on
Please double check and update the code carefully when you have time.
Also, I have a question, are the results stated in the paper based on this project??

Second, the project is not coding standard.
1.There are a lot of unused imports in the project, such as “import ast” in detection.py, cv2 in kalman_filter.py, and so on.
2.In addition, in the kalman_filter.py file, the type of tn was originally int, later in the line 92 tn became a dict? Shouldn't the correct way to write line 80 be “if not tn”?

Finally,I previous said that the necessary comments are missing in the code, like
1.what exactly is the meaning of tn mentioned above? Whether it is a glyph or a dict with specific meaning is not given in the code.
2.In line 132 of Kalman_filter.py, the function project() requires 5 parameters. Why are there no corresponding annotations for the other 3 parameters except the mean and covariance parameters that were originally in the deep_sort project?
There are many such cases, please check carefully when you have time, and add this part of the note if you can.

Detecting Invisible People is a funny stuff, I wanted to peruse the code for this project, but it was kind of bad, and reading it made me so irritable, hence i raise yesterday's issue. I think not only we must learn to be a rational person, also we should be rational and rigorous with our own research and works. The above are some of the questions I have raised. I am looking forward to your reply.

tarashakhurana / detecting-invisible-people Goto Github PK