youtubevos/masktrackrcnn

MaskTrackRCNN for video instance segmentation based on mmdetection

MaskTrackRCNN for video instance segmentation

Introduction

This repo serves as the official code release of the MaskTrackRCNN model for video instance segmentation described in the tech report:

@article{ Yang2019vis,
  author = {Linjie Yang and Yuchen Fan and Ning Xu},
  title = {Video instance segmentation},
  journal = {CoRR},
  volume = {abs/1905.04804},
  year = {2019},
  url = {https://arxiv.org/abs/1905.04804}
}

In this work, a new task video instance segmentation is presented. Video instance segmentation extends the image instance segmentation task from the image domain to the video domain. The new problem aims at simultaneous detection, segmentation and tracking of object instances in videos. YouTubeVIS, a new dataset tailored for this task is collected based on the current largest video object segmentation dataset YouTubeVOS. Sample annotations of a video clip can be seen below. We also proposed an algorithm to jointly detect, segment, and track object instances in a video, named MaskTrackRCNN. A tracking head is added to the original MaskRCNN model to match objects across frames. An overview of the algorithm is shown below.

Installation

This repo is built based on mmdetection commit hash f3a939f. Please refer to INSTALL.md to install the library. You also need to install a customized COCO API for YouTubeVIS dataset. You can use following commands to create conda env with all dependencies.

conda create -n MaskTrackRCNN -y
conda activate MaskTrackRCNN
conda install -c pytorch pytorch=0.4.1 torchvision cuda92 -y
conda install -c conda-forge cudatoolkit-dev=9.2 opencv -y
conda install cython pillow=7 -y
pip install git+https://github.com/youtubevos/cocoapi.git#"egg=pycocotools&subdirectory=PythonAPI"
bash compile.sh
pip install .

You may also need to follow #1 to load MSCOCO pretrained models.

Model training and evaluation

Our model is based on MaskRCNN-resnet50-FPN. The model is trained end-to-end on YouTubeVIS based on a MSCOCO pretrained checkpoint (link).

Training

1. Download YouTubeVIS from here.
2. Symlink the train/validation dataset to $MMDETECTION/data folder. Put COCO-style annotations under $MMDETECTION/data/annotations.

mmdetection
├── mmdet
├── tools
├── configs
├── data
│   ├── train
│   ├── val
│   ├── annotations
│   │   ├── instances_train_sub.json
│   │   ├── instances_val_sub.json

3. Run python3 tools/train.py configs/masktrack_rcnn_r50_fpn_1x_youtubevos.py to train the model. For reference to arguments such as learning rate and model parameters, please refer to configs/masktrack_rcnn_r50_fpn_1x_youtubevos.py

Evaluation

Our pretrained model is available for download at Google Drive. Run the following command to evaluate the model on YouTubeVIS.

python3 tools/test_video.py configs/masktrack_rcnn_r50_fpn_1x_youtubevos.py [MODEL_PATH] --out [OUTPUT_PATH] --eval segm

A json file containing the predicted result will be generated as OUTPUT_PATH.json. YouTubeVIS currently only allows evaluation on the codalab server. Please upload the generated result to codalab server to see actual performances.

License

This project is released under the Apache 2.0 license.

Contact

If you have any questions regarding the repo, please contact Linjie Yang ([email protected]) or create an issue.

masktrackrcnn's Issues

mAP of MaskTrackRCNN tested on codalab server

I use the pretrained model epoch_12.pth from Google Drive, and then evaluate the results on the codalab server, but i got mAP: 0.248, however, it should be 0.3 as reported in the paper. Any suggestion? Thank you!

Import YTVOS

Hi, thank you for sharing good works.

I tried this command, but could not solved.
pip install git+https://github.com/youtubevos/cocoapi.git#"egg=pycocotools&subdirectory=PythonAPI"

Could you advise how to handle this??

(MaskTrackRCNN) user@user:/sdata1/workspace/MaskTrackRCNN$ python3 tools/test_video.py configs/masktrack_rcnn_r50_fpn_1x_youtubevos.py models/epoch_12.pth --out output --eval segm
Traceback (most recent call last):
File "tools/test_video.py", line 8, in
from mmdet import datasets
File "/home/user/anaconda3/envs/MaskTrackRCNN/lib/python3.6/site-packages/mmdet/datasets/init.py", line 1, in
from .custom import CustomDataset
File "/home/user/anaconda3/envs/MaskTrackRCNN/lib/python3.6/site-packages/mmdet/datasets/custom.py", line 11, in
from .extra_aug import ExtraAugmentation
File "/home/user/anaconda3/envs/MaskTrackRCNN/lib/python3.6/site-packages/mmdet/datasets/extra_aug.py", line 5, in
from mmdet.core.evaluation.bbox_overlaps import bbox_overlaps
File "/home/user/anaconda3/envs/MaskTrackRCNN/lib/python3.6/site-packages/mmdet/core/init.py", line 5, in
from .evaluation import * # noqa: F401, F403
File "/home/user/anaconda3/envs/MaskTrackRCNN/lib/python3.6/site-packages/mmdet/core/evaluation/init.py", line 4, in
from .coco_utils import coco_eval, fast_eval_recall, results2json, results2json_videoseg, ytvos_eval
File "/home/user/anaconda3/envs/MaskTrackRCNN/lib/python3.6/site-packages/mmdet/core/evaluation/coco_utils.py", line 5, in
from pycocotools.ytvos import YTVOS
ModuleNotFoundError: No module named 'pycocotools.ytvos'

-

[bug fix] wrong bbox coordinates in evaluation

Background

The previous evaluation code has a bug that it uses unscaled bbox coordinates to extract features for identity matching. It results in lower accuracy when tracking head does not use additional cues for matching. When additional cues (IoU, category, detection score) are used, the performance does not change much.

Changes

commit 4e92e86

Tests

Tested with the current model

Misc

Due to this bug, some results in the arXiv paper is also changed. We will update the results in the arXiv paper shortly. Sorry for any inconvenience caused by this issue.

When training the model, a RuntimeError arised

The json style is inconsistent with MSCOCO?

Please refer to youtubevos/cocoapi#1 (comment)

How can I find original youtub video used in vis or vos?And If I want to reproduce your algorithm I have to generate COCO-style annotations by myself,right?

As mention above

Error during test

Hello, thanks for releasing the code! I have been having trouble initializing this code, I have performed the installation as mentioned on the README.md. But I encountered the following error:
`Traceback (most recent call last):
File "tools/test_video.py", line 4, in
import mmcv
File "/home/daniel/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/mmcv/init.py", line 4, in
from .fileio import *
File "/home/daniel/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/mmcv/fileio/init.py", line 4, in
from .io import dump, load, register_handler
File "/home/daniel/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/mmcv/fileio/io.py", line 4, in
from ..utils import is_list_of, is_str
File "/home/daniel/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/mmcv/utils/init.py", line 29, in
from .parrots_wrapper import (CUDA_HOME, BuildExtension, CppExtension,
File "/home/daniel/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/mmcv/utils/parrots_wrapper.py", line 79, in
_BatchNorm, InstanceNorm, SyncBatchNorm = _get_norm()
File "/home/daniel/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/mmcv/utils/parrots_wrapper.py", line 71, in get_norm
SyncBatchNorm = torch.nn.SyncBatchNorm
AttributeError: module 'torch.nn' has no attribute 'SyncBatchNorm'

Thank you for your attention!
`

ModuleNotFoundError: No module named 'pycocotools.ytvos'

hi,when I run the command "python tools/train.py configs/masktrack_rcnn_r50_fpn_1x_youtubevos.py", it returned an error which was "ModuleNotFoundError: No module named 'pycocotools.ytvos'". It seems like the 'pycocotools.ytvos' is your modified and new added part. Could you please tell me how to solve it?

Dataset organization

Hello, thanks for releasing this work! I did not understand how to organize the dataset into the form that you exposed on the README.md. Could you please explain more?

MaskTrackRCNN on KITTI MOTS

Hi,
Has anybody tried this software on http://www.cvlibs.net/datasets/kitti/eval_mots.php?
Do you have results?

The baseline MAP can be achieved using the default setting?

Hi, Linjie.
under the default setting, I train a model using a single GPU. But it seems to be difficult to achieve 30.3 MAP. Besides, I note that the learning rate is 0.05 in your paper, while 0.005 in your code?
Thanks

Roi pulling not recognized!

I get the following error when running evaluation:

Traceback (most recent call last):
File "./tools/test_video.py", line 8, in
from mmdet import datasets
File "/home/bigboi/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/mmdet/datasets/init.py", line 1, in
from .custom import CustomDataset
File "/home/bigboi/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/mmdet/datasets/custom.py", line 11, in
from .extra_aug import ExtraAugmentation
File "/home/bigboi/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/mmdet/datasets/extra_aug.py", line 5, in
from mmdet.core.evaluation.bbox_overlaps import bbox_overlaps
File "/home/bigboi/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/mmdet/core/init.py", line 6, in
from .post_processing import * # noqa: F401, F403
File "/home/bigboi/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/mmdet/core/post_processing/init.py", line 1, in
from .bbox_nms import multiclass_nms
File "/home/bigboi/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/mmdet/core/post_processing/bbox_nms.py", line 3, in
from mmdet.ops.nms import nms_wrapper
File "/home/bigboi/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/mmdet/ops/init.py", line 2, in
from .roi_align import RoIAlign, roi_align
File "/home/bigboi/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/mmdet/ops/roi_align/init.py", line 1, in
from .functions.roi_align import roi_align
File "/home/bigboi/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/mmdet/ops/roi_align/functions/roi_align.py", line 3, in
from .. import roi_align_cuda
ImportError: cannot import name 'roi_align_cuda' from 'mmdet.ops.roi_align' (/home/bigboi/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/mmdet/ops/roi_align/init.py)

About the evaluation in the codalab

Hi, when I try to upload the json file to the codalab( https://competitions.codalab.org/competitions/20128#participate-submit_results), the website gives a piece of error information "Invalid file type (application/json)." Is it normal? How can I evaluate the model? Thank you !

RuntimeError

Hi, I'm train MaskTrackRCNN following the readme,. However, an error occured.

Traceback (most recent call last):
File "/home/ubuntu/.pycharm_helpers/pydev/pydevd.py", line 2060, in
main()
File "/home/ubuntu/.pycharm_helpers/pydev/pydevd.py", line 2054, in main
globals = debugger.run(setup['file'], None, None, is_module)
File "/home/ubuntu/.pycharm_helpers/pydev/pydevd.py", line 1405, in run
return self._exec(is_module, entry_point_fn, module_name, file, globals, locals)
File "/home/ubuntu/.pycharm_helpers/pydev/pydevd.py", line 1412, in _exec
pydev_imports.execfile(file, globals, locals) # execute the script
File "/home/ubuntu/.pycharm_helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "/home/ubuntu/code/MaskTrackRCNN/tools/train.py", line 99, in
main()
File "/home/ubuntu/code/MaskTrackRCNN/tools/train.py", line 92, in main
logger=logger)
File "/home/ubuntu/code/MaskTrackRCNN/mmdet/apis/train.py", line 59, in train_detector
_non_dist_train(model, dataset, cfg, validate=validate)
File "/home/ubuntu/code/MaskTrackRCNN/mmdet/apis/train.py", line 123, in _non_dist_train
runner.run(data_loaders, cfg.workflow, cfg.total_epochs)
File "/home/ubuntu/Softwares/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/mmcv/runner/runner.py", line 358, in run
epoch_runner(data_loaders[i], **kwargs)
File "/home/ubuntu/Softwares/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/mmcv/runner/runner.py", line 271, in train
self.call_hook('after_train_iter')
File "/home/ubuntu/Softwares/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/mmcv/runner/runner.py", line 229, in call_hook
getattr(hook, fn_name)(self)
File "/home/ubuntu/Softwares/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/mmcv/runner/hooks/optimizer.py", line 17, in after_train_iter
runner.outputs['loss'].backward()
File "/home/ubuntu/Softwares/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/torch/tensor.py", line 118, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File "/home/ubuntu/Softwares/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/torch/autograd/init.py", line 93, in backward
allow_unreachable=True) # allow_unreachable flag
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [148, 256, 14, 14]], which is output 0 of IndexPutBackward, is at version 5; expected version 4 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).

with:
cuda:9.2
pytorch:1.2.0

Could you please give me any suggestion?

The evaluation serve and the baseline results.

Hi, thank you for sharing this good benchmark task and the baseline code.
I have some questions when reproducing the baseline.
I can not find the information about the batch size in your paper.
You have mentioned that you use four GPU for training. Do you use 4*8 as the training batchsize?

I have trained a model using your default settings with one single GPU. Now I want to evaluate the results. However, the link https://competitions.codalab.org/competitions/23215 does not work.
I found a submit server here: https://competitions.codalab.org/competitions/20128#participate-submit_results
After running the test_video.py, I have got an output.pkl and an output.json.
It also prints the information like:
"Starting evaluate segm
Annotations does not exist"
Is it a normal one?
When I try to upload the json, the website gives a piece of error information "Invalid file type (application/json)."

Can you help me with this?

-

Segmentation fault

As is showed below, my training ends with Segmentation fault, Could someone tell me why ？

(MaskTrackRCNN) [lijie@yq01-gpu-255-126-19-00 MaskTrackRCNN-master]$ python3 tools/train.py configs/masktrack_rcnn_r50_fpn_1x_youtubevos.py
2019-12-17 21:52:08,555 - INFO - Distributed training: False
2019-12-17 21:52:09,311 - INFO - load model from: modelzoo://resnet50
2019-12-17 21:52:09,625 - WARNING - unexpected key in source state_dict: fc.weight, fc.bias

missing keys in source state_dict: layer4.0.downsample.1.num_batches_tracked, layer3.4.bn2.num_batches_tracked, layer2.1.bn3.num_batches_tracked, layer1.0.bn1.num_batches_tracked, layer1.1.bn1.num_batches_tracked, layer2.3.bn1.num_batches_tracked, layer3.5.bn2.num_batches_tracked, layer2.0.bn3.num_batches_tracked, layer3.3.bn3.num_batches_tracked, layer3.2.bn3.num_batches_tracked, layer3.4.bn3.num_batches_tracked, layer2.2.bn1.num_batches_tracked, layer3.2.bn2.num_batches_tracked, layer3.1.bn2.num_batches_tracked, layer1.0.downsample.1.num_batches_tracked, bn1.num_batches_tracked, layer4.2.bn1.num_batches_tracked, layer2.1.bn1.num_batches_tracked, layer3.0.bn3.num_batches_tracked, layer1.2.bn3.num_batches_tracked, layer3.4.bn1.num_batches_tracked, layer1.0.bn2.num_batches_tracked, layer2.2.bn2.num_batches_tracked, layer2.0.bn1.num_batches_tracked, layer2.1.bn2.num_batches_tracked, layer4.0.bn2.num_batches_tracked, layer2.0.bn2.num_batches_tracked, layer3.5.bn3.num_batches_tracked, layer3.3.bn2.num_batches_tracked, layer1.0.bn3.num_batches_tracked, layer4.1.bn3.num_batches_tracked, layer3.1.bn1.num_batches_tracked, layer3.5.bn1.num_batches_tracked, layer1.1.bn2.num_batches_tracked, layer4.0.bn3.num_batches_tracked, layer3.3.bn1.num_batches_tracked, layer4.1.bn2.num_batches_tracked, layer2.3.bn2.num_batches_tracked, layer4.1.bn1.num_batches_tracked, layer2.3.bn3.num_batches_tracked, layer3.1.bn3.num_batches_tracked, layer2.0.downsample.1.num_batches_tracked, layer3.0.bn2.num_batches_tracked, layer4.2.bn2.num_batches_tracked, layer1.2.bn2.num_batches_tracked, layer4.0.bn1.num_batches_tracked, layer1.1.bn3.num_batches_tracked, layer3.2.bn1.num_batches_tracked, layer3.0.downsample.1.num_batches_tracked, layer2.2.bn3.num_batches_tracked, layer1.2.bn1.num_batches_tracked, layer3.0.bn1.num_batches_tracked, layer4.2.bn3.num_batches_tracked

loading annotations into memory...
Done (t=14.82s)
creating index...
index created!
2019-12-17 21:53:59,679 - INFO - load checkpoint from https://s3.ap-northeast-2.amazonaws.com/open-mmlab/mmdetection/models/mask_rcnn_r50_fpn_1x_20181010-069fa190.pth
2019-12-17 21:53:59,879 - WARNING - missing keys in source state_dict: track_head.fcs.0.bias, track_head.fcs.1.bias, track_head.fcs.1.weight, track_head.fcs.0.weight

2019-12-17 21:53:59,900 - INFO - Start running, host: lijie@yq01-gpu-255-126-19-00, work_dir: /home/lijie/MaskTrackRCNN-master/work_dirs/masktrack_rcnn_r50_fpn_1x_youtubevos
2019-12-17 21:53:59,900 - INFO - workflow: [('train', 1)], max: 12 epochs
Segmentation fault

Where does instances_train_sub.json locate

we could only download train.json or meta.json, where can we find instances_train_sub.json?
It would be really appreciated if someone could give me some hint, Thanks ahead!

No module named 'mmdet.ops.nms.gpu_nms'

When I run 'python3 tools/train.py configs/masktrack_rcnn_r50_fpn_1x_youtubevos.py', an error occured as follow:
File "tools/train.py", line 7, in
from mmdet.datasets import get_dataset
File "/home/lin/python_work/competition/OVIS/MaskTrackRCNN-master/mmdet/datasets/init.py", line 1, in
from .custom import CustomDataset
File "/home/lin/python_work/competition/OVIS/MaskTrackRCNN-master/mmdet/datasets/custom.py", line 11, in
from .extra_aug import ExtraAugmentation
File "/home/lin/python_work/competition/OVIS/MaskTrackRCNN-master/mmdet/datasets/extra_aug.py", line 5, in
from mmdet.core.evaluation.bbox_overlaps import bbox_overlaps
File "/home/lin/python_work/competition/OVIS/MaskTrackRCNN-master/mmdet/core/init.py", line 6, in
from .post_processing import * # noqa: F401, F403
File "/home/lin/python_work/competition/OVIS/MaskTrackRCNN-master/mmdet/core/post_processing/init.py", line 1, in
from .bbox_nms import multiclass_nms
File "/home/lin/python_work/competition/OVIS/MaskTrackRCNN-master/mmdet/core/post_processing/bbox_nms.py", line 3, in
from mmdet.ops.nms import nms_wrapper
File "/home/lin/python_work/competition/OVIS/MaskTrackRCNN-master/mmdet/ops/init.py", line 1, in
from .nms import nms, soft_nms
File "/home/lin/python_work/competition/OVIS/MaskTrackRCNN-master/mmdet/ops/nms/init.py", line 1, in
from .nms_wrapper import nms, soft_nms
File "/home/lin/python_work/competition/OVIS/MaskTrackRCNN-master/mmdet/ops/nms/nms_wrapper.py", line 4, in
from .gpu_nms import gpu_nms
ModuleNotFoundError: No module named 'mmdet.ops.nms.gpu_nms'

How to fix this error? Many thanks.

About reproduce MaskRCNN baseline

Hi,
Thank you very much for your inspiring work and sharing your code! I tried your code on my own dataset, but I even got bad results of detection compared to the detection results with mmdetection library. Do your have any idea for the possible reasons? I want to try maskrcnn with your code to find the problem, but I it seem that a track head is always needed. Is there anyway to perform maskrcnn only with your code directly?
Your help is greatly appreciated. Thank you very much.
Regards,
Annabel

error about the loss_match

I'm trying to train the model with youtube-vis dataset following the instructions but got loss_match equals zero at the beginning.
Do you have any idea about this?

Here is the error info:
loss {'loss_rpn_cls': [tensor([0.5429], device='cuda:0', grad_fn=), tensor([0.1198], device='cuda:0', grad_fn=), tensor([0.0242], device='cuda:0', grad_fn=), tensor([0.0027], device='cuda:0', grad_fn=), tensor([0.], device='cuda:0', grad_fn=)], 'loss_rpn_reg': [tensor([0.], device='cuda:0', grad_fn=), tensor([0.], device='cuda:0', grad_fn=), tensor([0.0009], device='cuda:0', grad_fn=), tensor([0.], device='cuda:0', grad_fn=), tensor([0.], device='cuda:0', grad_fn=)], 'loss_cls': tensor([3.6361], device='cuda:0', grad_fn=), 'acc': tensor([1.4925], device='cuda:0'), 'loss_reg': tensor([4.3604e-06], device='cuda:0', grad_fn=), 'loss_match': 0.0, 'loss_mask': tensor([0.7678], device='cuda:0', grad_fn=)}
Traceback (most recent call last):
File "tools/train.py", line 93, in
main()
File "tools/train.py", line 89, in main
logger=logger)
File "/data/qianyu/detection/MaskTrackRCNN/mmdet/apis/train.py", line 60, in train_detector
_non_dist_train(model, dataset, cfg, validate=validate)
File "/data/qianyu/detection/MaskTrackRCNN/mmdet/apis/train.py", line 122, in _non_dist_train
runner.run(data_loaders, cfg.workflow, cfg.total_epochs)
File "/home/qianyu/anaconda3/envs/p3t4/lib/python3.7/site-packages/mmcv/runner/runner.py", line 358, in run
epoch_runner(data_loaders[i], **kwargs)
File "/home/qianyu/anaconda3/envs/p3t4/lib/python3.7/site-packages/mmcv/runner/runner.py", line 264, in train
self.model, data_batch, train_mode=True, **kwargs)
File "/data/qianyu/detection/MaskTrackRCNN/mmdet/apis/train.py", line 39, in batch_processor
loss, log_vars = parse_losses(losses)
File "/data/qianyu/detection/MaskTrackRCNN/mmdet/apis/train.py", line 25, in parse_losses
'{} is not a tensor or list of tensors'.format(loss_name))
TypeError: loss_match is not a tensor or list of tensors

Code for Summit to Server

Would you mind to release your code for change results.pkl to JSON file for evaluation?

Evaluation code

Hello,

Thanks for your nice code again.
I am curious about when the evaluation code would be available.

Thanks,

ImportError: cannot import name 'parallel_test'

when I use "python3 tools/test_video.py configs/masktrack_rcnn_r50_fpn_1x_youtubevos.py ./epoch_12.pth --out ./output --eval segm" to run the code. I got this error, but I have already installed 'mmcv' package.

I have no idea about this, could you please tell me what should I do?

[Compile Error] mmdetection compile fails

building 'nms_cuda' extension
gcc -pthread -B /home/miruware/anaconda3/envs/py36_2/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/miruware/anaconda3/envs/py36_2/lib/python3.7/site-packages/torch/lib/include -I/home/miruware/anaconda3/envs/py36_2/lib/python3.7/site-packages/torch/lib/include/TH -I/home/miruware/anaconda3/envs/py36_2/lib/python3.7/site-packages/torch/lib/include/THC -I/usr/local/cuda/include -I/home/miruware/anaconda3/envs/py36_2/include/python3.7m -c src/nms_cuda.cpp -o build/temp.linux-x86_64-3.7/src/nms_cuda.o -DTORCH_EXTENSION_NAME=nms_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
src/nms_cuda.cpp:2:29: fatal error: torch/extension.h: No such file or directory
compilation terminated.
error: command 'gcc' failed with exit status 1
Building dcn...
running build_ext
building 'deform_conv_cuda' extension
creating build
creating build/temp.linux-x86_64-3.7
creating build/temp.linux-x86_64-3.7/src
gcc -pthread -B /home/miruware/anaconda3/envs/py36_2/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/miruware/anaconda3/envs/py36_2/lib/python3.7/site-packages/torch/lib/include -I/home/miruware/anaconda3/envs/py36_2/lib/python3.7/site-packages/torch/lib/include/TH -I/home/miruware/anaconda3/envs/py36_2/lib/python3.7/site-packages/torch/lib/include/THC -I/usr/local/cuda/include -I/home/miruware/anaconda3/envs/py36_2/include/python3.7m -c src/deform_conv_cuda.cpp -o build/temp.linux-x86_64-3.7/src/deform_conv_cuda.o -DTORCH_EXTENSION_NAME=deform_conv_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
src/deform_conv_cuda.cpp:4:29: fatal error: torch/extension.h: No such file or directory
compilation terminated.
error: command 'gcc' failed with exit status 1
Building sigmoid focal loss op...
running build_ext
building 'sigmoid_focal_loss_cuda' extension
creating build
creating build/temp.linux-x86_64-3.7
creating build/temp.linux-x86_64-3.7/src
gcc -pthread -B /home/miruware/anaconda3/envs/py36_2/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/miruware/anaconda3/envs/py36_2/lib/python3.7/site-packages/torch/lib/include -I/home/miruware/anaconda3/envs/py36_2/lib/python3.7/site-packages/torch/lib/include/TH -I/home/miruware/anaconda3/envs/py36_2/lib/python3.7/site-packages/torch/lib/include/THC -I/usr/local/cuda/include -I/home/miruware/anaconda3/envs/py36_2/include/python3.7m -c src/sigmoid_focal_loss.cpp -o build/temp.linux-x86_64-3.7/src/sigmoid_focal_loss.o -DTORCH_EXTENSION_NAME=sigmoid_focal_loss_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
src/sigmoid_focal_loss.cpp:3:29: fatal error: torch/extension.h: No such file or directory
compilation terminated.
error: command 'gcc' failed with exit status 1
Building masked conv op...
running build_ext
building 'masked_conv2d_cuda' extension
creating build
creating build/temp.linux-x86_64-3.7
creating build/temp.linux-x86_64-3.7/src
gcc -pthread -B /home/miruware/anaconda3/envs/py36_2/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/miruware/anaconda3/envs/py36_2/lib/python3.7/site-packages/torch/lib/include -I/home/miruware/anaconda3/envs/py36_2/lib/python3.7/site-packages/torch/lib/include/TH -I/home/miruware/anaconda3/envs/py36_2/lib/python3.7/site-packages/torch/lib/include/THC -I/usr/local/cuda/include -I/home/miruware/anaconda3/envs/py36_2/include/python3.7m -c src/masked_conv2d_cuda.cpp -o build/temp.linux-x86_64-3.7/src/masked_conv2d_cuda.o -DTORCH_EXTENSION_NAME=masked_conv2d_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
src/masked_conv2d_cuda.cpp:1:29: fatal error: torch/extension.h: No such file or directory
compilation terminated.
error: command 'gcc' failed with exit status 1

I got the compile error when installing the mmdetection library.
The above is a short snippet of the error.

I use
python 3.7.3
pytorch 0.4.1

Could you please give me any suggestion?

Problem about uploading results to evaluation server

Dear author, thanks for your work!
Today I upload the 'valid_submission_sample' to the evaluation server, however, it reports an error like below:

Traceback (most recent call last):
  File "/worker/worker.py", line 302, in run
    root_dir = tempfile.mkdtemp(dir=temp_dir)
  File "/usr/local/lib/python2.7/tempfile.py", line 339, in mkdtemp
    _os.mkdir(file, 0700)
OSError: [Errno 30] Read-only file system: '/tmp/codalab/tmpcP1K7y'

Could you have a look about this?
Thanks in advance!

The pretained weights of Mask R-CNN is not available.

The pretained weights of Mask R-CNN is not available.
Could you provide a link to download the weights.

Ability to implement in Google colab

I tried very hard to run your code on Google colab . But I did not succeed.
Is it possible for you to implement this code in Google colab and publish it?

Question 2: Does your code run in Python 3.7?

Running test_video.py does not evaluate?

Hi thanks for the nice code framework.
For evaluation, when I run the tools/test_video.py, it does not properly run the evaluation even if I follow the setup described in the readme.

This is the outcome after finishing running test_video.py

writing results to ./data/rst/testouput.pkl
Starting evaluate segm
Annotations does not exist

If there is anything I missed, it seems that it'd be nice to update the readme for evaluation information. Any comment would be appreciated.
Thanks.

SyncBatchNorm

I used the mmdetection from f3a939f
It seems like torch 0.4.1 don't support the SynBatchNorm Libraray which is in the mmdetection part.

Can I ask the how to handle SyncBatchNorm.

AttributeError: module 'torch.nn' has no attribute 'SyncBatchNorm'

https://github.com/open-mmlab/mmdetection/blob/8732ed96a85163fc4fb6d36b52ff3c3d25172611/tools/fuse_conv_bn.py#L32

open-mmlab#684

python3 tools/test_video.py configs/masktrack_rcnn_r50_fpn_1x_youtubevos.py models/epoch_12.pth --out output --eval segm

Traceback (most recent call last):
File "tools/test_video.py", line 4, in
import mmcv
File "/home/user/anaconda3/envs/MaskTrackRCNN/lib/python3.6/site-packages/mmcv/init.py", line 4, in
from .fileio import *
File "/home/user/anaconda3/envs/MaskTrackRCNN/lib/python3.6/site-packages/mmcv/fileio/init.py", line 4, in
from .io import dump, load, register_handler
File "/home/user/anaconda3/envs/MaskTrackRCNN/lib/python3.6/site-packages/mmcv/fileio/io.py", line 4, in
from ..utils import is_list_of, is_str
File "/home/user/anaconda3/envs/MaskTrackRCNN/lib/python3.6/site-packages/mmcv/utils/init.py", line 8, in
from .parrots_wrapper import (CUDA_HOME, BuildExtension, CppExtension,
File "/home/user/anaconda3/envs/MaskTrackRCNN/lib/python3.6/site-packages/mmcv/utils/parrots_wrapper.py", line 67, in
_BatchNorm, InstanceNorm, SyncBatchNorm = _get_norm()
File "/home/user/anaconda3/envs/MaskTrackRCNN/lib/python3.6/site-packages/mmcv/utils/parrots_wrapper.py", line 60, in get_norm
SyncBatchNorm = torch.nn.SyncBatchNorm
AttributeError: module 'torch.nn' has no attribute 'SyncBatchNorm'

What is the number for the segmentation category in annotation?

[139447, 42, 678, 42, 671, 62, 658, 62, 656, 72, 648, 72, 642, 80, 640, 80, 636, 87, 633, 87, 631, 94, 626, 94, 619, 106, 614, 106, 614, 121, 599, 121, 598, 128, 592, 128, 592, 129, 591, 129, 591, 132, 588, 132, 588, 134, 586, 134, 586, 136, 584, 136, 584, 138, 582, 138, 582, 140, 580, 140, 580, 140, 580, 140, 580, 140, 580, 140, 580, 140, 580, 140, 580, 141, 579, 141, 579, 141, 579, 141, 579, 141, 579, 141, 579, 141, 579, 141, 579, 141, 579, 138, 582, 138, 582, 133, 587, 133, 587, 129, 591, 129, 591, 127, 593, 127, 593, 125, 595, 125, 595, 121, 599, 121, 599, 119, 601, 119, 601, 113, 607, 113, 607, 105, 615, 105, 615, 97, 623, 97, 623, 45, 675, 45, 675, 44, 676, 44, 677, 43, 676, 44, 676, 44, 676, 44, 676, 44, 676, 44, 676, 44, 676, 44, 676, 44, 676, 44, 677, 43, 677, 43, 677, 43, 677, 43, 678, 42, 678, 43, 677, 43, 677, 45, 676, 44, 676, 44, 676, 44, 676, 44, 676, 44, 676, 46, 674, 46, 674, 47, 673, 47, 673, 48, 672, 48, 672, 50, 670, 50, 670, 53, 667, 53, 667, 53, 667, 53, 667, 55, 665, 55, 665, 60, 660, 60, 660, 61, 660, 60, 660, 62, 658, 62, 658, 62, 658, 62, 658, 63, 657, 63, 657, 64, 656, 64, 656, 64, 656, 64, 656, 65, 655, 65, 655, 65, 655, 65, 655, 66, 654, 66, 654, 67, 653, 67, 653, 67, 653, 67, 653, 67, 653, 67, 653, 67, 653, 67, 653, 69, 651, 69, 651, 69, 651, 69, 651, 69, 653, 67, 653, 68, 652, 68, 652, 68, 652, 68, 652, 68, 652, 68, 652, 69, 651, 69, 651, 69, 651, 69, 651, 70, 651, 69, 651, 70, 650, 70, 650, 72, 648, 72, 648, 73, 647, 73, 647, 73, 647, 73, 647, 73, 648, 72, 648, 72, 648, 72, 648, 73, 647, 73, 647, 75, 646, 74, 646, 74, 646, 74, 646, 75, 645, 75, 645, 77, 643, 77, 643, 78, 643, 77, 643, 77, 645, 75, 645, 76, 644, 76, 644, 76, 644, 76, 644, 76, 645, 75, 645, 76, 644, 76, 644, 80, 640, 80, 640, 92, 628, 92, 628, 94, 626, 94, 626, 96, 624, 96, 624, 99, 622, 98, 622, 100, 621, 99, 621, 100, 620, 100, 620, 103, 617, 103, 617, 103, 617, 103, 617, 105, 615, 105, 615, 108, 612, 108, 612, 108, 612, 108, 612, 111, 609, 111, 609, 113, 607, 113, 607, 116, 604, 116, 604, 117, 603, 117, 603, 120, 600, 120, 600, 123, 597, 123, 597, 125, 595, 125, 595, 127, 593, 127, 593, 130, 590, 130, 590, 131, 589, 131, 589, 132, 588, 132, 588, 136, 584, 136, 584, 137, 583, 137, 583, 138, 582, 138, 582, 139, 581, 139, 581, 142, 578, 142, 578, 143, 577, 143, 577, 145, 575, 145, 575, 146, 574, 146, 574, 147, 573, 147, 573, 150, 570, 150, 570, 151, 569, 151, 569, 152, 568, 152, 568, 157, 563, 157, 563, 166, 554, 166, 553, 171, 549, 171, 549, 178, 542, 178, 542, 182, 538, 182, 537, 187, 533, 187, 533, 196, 524, 196, 524, 201, 519, 201, 518, 206, 514, 206, 512, 210, 510, 210, 510, 213, 507, 213, 507, 213, 507, 213, 506, 214, 506, 214, 505, 215, 505, 215, 505, 215, 505, 215, 504, 216, 504, 216, 504, 216, 504, 216, 504, 216, 504, 216, 504, 216, 504, 216, 501, 219, 501, 219, 498, 222, 498, 222, 498, 223, 497, 223, 495, 225, 495, 225, 495, 225, 495, 225, 493, 227, 493, 227, 491, 229, 491, 229, 490, 230, 490, 230, 489, 232, 488, 232, 479, 241, 479, 241, 478, 242, 478, 242, 462, 258, 462, 258, 456, 264, 456, 264, 445, 275, 445, 275, 440, 280, 440, 280, 438, 282, 438, 282, 437, 283, 437, 283, 433, 287, 433, 287, 427, 293, 427, 293, 417, 303, 417, 303, 412, 308, 412, 308, 408, 312, 408, 312, 404, 316, 404, 316, 399, 321, 399, 321, 377, 343, 377, 343, 364, 356, 364, 356, 353, 367, 353, 367, 345, 375, 345, 375, 339, 382, 338, 382, 325, 395, 325, 395, 295, 425, 295, 425, 293, 427, 293, 427, 289, 431, 289, 431, 266, 454, 266, 454, 258, 462, 258, 462, 254, 466, 254, 466, 248, 472, 248, 472, 244, 476, 244, 476, 242, 478, 242, 478, 240, 480, 240, 480, 237, 483, 237, 483, 236, 484, 236, 484, 235, 485, 235, 485, 234, 486, 234, 486, 230, 490, 230, 490, 227, 493, 227, 493, 225, 495, 225, 495, 223, 497, 223, 497, 222, 498, 222, 498, 218, 502, 218, 502, 215, 505, 215, 505, 211, 509, 211, 509, 210, 510, 210, 510, 208, 512, 208, 512, 206, 514, 206, 514, 203, 517, 203, 517, 199, 521, 199, 521, 196, 524, 196, 524, 193, 527, 193, 527, 191, 529, 191, 529, 189, 531, 189, 531, 185, 535, 185, 535, 182, 538, 182, 538, 180, 540, 180, 540, 179, 541, 179, 541, 177, 543, 177, 543, 177, 543, 177, 543, 175, 545, 175, 545, 175, 545, 175, 544, 174, 546, 174, 546, 171, 549, 171, 549, 171, 549, 171, 549, 168, 552, 168, 552, 168, 552, 168, 552, 168, 552, 167, 553, 167, 553, 167, 553, 167, 553, 165, 555, 165, 555, 163, 557, 163, 557, 161, 559, 161, 559, 161, 559, 161, 559, 160, 560, 160, 560, 158, 562, 158, 562, 158, 562, 158, 562, 157, 563, 157, 563, 155, 565, 155, 565, 155, 565, 155, 565, 153, 567, 153, 567, 153, 567, 153, 567, 153, 567, 153, 567, 152, 568, 152, 568, 152, 568, 152, 567, 153, 567, 153, 567, 153, 567, 153, 567, 152, 568, 152, 568, 152, 568, 152, 568, 152, 568, 152, 568, 152, 568, 152, 568, 152, 570, 150, 570, 150, 570, 150, 570, 150, 571, 149, 571, 149, 573, 147, 573, 147, 574, 146, 574, 146, 575, 145, 575, 145, 576, 144, 576, 144, 576, 146, 574, 146, 575, 145, 575, 145, 575, 145, 575, 145, 577, 143, 577, 143, 578, 143, 577, 143, 578, 142, 578, 142, 578, 142, 578, 142, 578, 142, 578, 142, 578, 145, 575, 145, 577, 143, 577, 143, 577, 144, 576, 144, 577, 144, 576, 144, 578, 143, 577, 143, 578, 142, 578, 142, 579, 142, 578, 142, 579, 143, 577, 143, 578, 144, 576, 144, 578, 142, 578, 142, 578, 144, 576, 144, 577, 146, 574, 146, 574, 147, 573, 147, 574, 147, 573, 147, 574, 147, 573, 147, 574, 146, 574, 146, 575, 146, 574, 146, 574, 148, 572, 148, 574, 146, 574, 146, 574, 149, 571, 149, 573, 147, 573, 147, 573, 148, 572, 148, 573, 148, 572, 148, 572, 148, 572, 148, 573, 150, 570, 150, 572, 149, 571, 149, 571, 150, 570, 150, 571, 150, 570, 150, 570, 150, 570, 150, 570, 154, 566, 154, 566, 154, 566, 154, 566, 155, 565, 155, 565, 156, 564, 156, 564, 157, 563, 157, 563, 162, 558, 162, 558, 165, 555, 165, 555, 167, 553, 167, 553, 168, 552, 168, 552, 170, 550, 170, 550, 173, 547, 173, 547, 175, 545, 175, 545, 176, 544, 176, 544, 179, 541, 179, 541, 182, 538, 182, 538, 183, 537, 183, 537, 186, 534, 186, 534, 188, 532, 188, 532, 192, 528, 192, 528, 193, 527, 193, 527, 200, 520, 200, 520, 202, 518, 202, 518, 205, 515, 205, 515, 205, 515, 205, 515, 205, 515, 205, 515, 206, 514, 206, 514, 207, 513, 207, 513, 208, 512, 208, 512, 209, 511, 209, 511, 209, 511, 209, 511, 211, 509, 211, 509, 212, 508, 212, 508, 212, 508, 212, 508, 214, 506, 214, 506, 214, 506, 214, 506, 215, 505, 215, 505, 215, 505, 215, 505, 215, 505, 215, 505, 215, 505, 215, 505, 217, 503, 217, 503, 218, 502, 218, 502, 218, 502, 218, 502, 219, 501, 219, 501, 219, 501, 219, 501, 219, 501, 219, 501, 219, 501, 219, 501, 221, 499, 221, 499, 221, 499, 221, 499, 221, 499, 221, 499, 221, 499, 221, 499, 221, 499, 221, 499, 222, 498, 222, 498, 222, 498, 222, 498, 222, 498, 222, 498, 224, 496, 224, 496, 224, 496, 224, 496, 224, 496, 224, 496, 224, 496, 224, 496, 224, 496, 224, 496, 224, 496, 224, 496, 226, 494, 226, 494, 226, 494, 226, 494, 226, 494, 226, 494, 227, 493, 227, 493, 228, 492, 228, 492, 228, 492, 228, 492, 228, 492, 228, 492, 228, 492, 228, 492, 230, 490, 230, 490, 230, 490, 230, 490, 230, 490, 230, 490, 231, 489, 231, 489, 232, 488, 232, 488, 232, 488, 232, 488, 234, 486, 234, 486, 234, 486, 234, 486, 234, 486, 234, 486, 236, 484, 236, 484, 237, 483, 237, 483, 237, 483, 237, 483, 238, 482, 238, 482, 240, 480, 240, 480, 240, 480, 240, 480, 241, 479, 241, 479, 244, 476, 244, 476, 244, 476, 244, 476, 245, 475, 245, 475, 246, 474, 246, 474, 249, 471, 249, 471, 249, 471, 249, 471, 252, 468, 252, 468, 253, 467, 253, 467, 253, 467, 253, 467, 256, 464, 256, 464, 257, 463, 257, 463, 258, 462, 258, 462, 262, 458, 262, 458, 264, 456, 264, 456, 265, 455, 265, 455, 273, 447, 273, 447, 278, 442, 278, 442, 280, 440, 280, 440, 284, 436, 284, 436, 289, 431, 289, 431, 290, 430, 290, 430, 293, 427, 293, 427, 296, 424, 296, 424, 303, 417, 303, 417, 305, 415, 305, 415, 309, 411, 309, 411, 314, 406, 314, 406, 324, 396, 324, 396, 337, 383, 337, 383, 361, 359, 361, 359, 381, 339, 396, 324, 403, 317, 403, 317, 409, 311, 409, 311, 420, 300, 420, 300, 425, 295, 425, 295, 429, 291, 429, 291, 435, 285, 435, 285, 449, 271, 449, 271, 459, 261, 459, 261, 467, 253, 467, 253, 477, 243, 477, 243, 481, 239, 481, 239, 485, 235, 485, 235, 490, 230, 490, 230, 503, 217, 503, 217, 523, 197, 523, 197, 533, 178, 542, 178, 599, 121, 599, 121, 620, 86, 634, 86, 638, 73, 647, 73, 173552]

Hi, I try to resolve the puzzle of the annotation for segmentation. I am wondering what these numbers stand for? I don't think this is polygon for the detected instance. Especially, the first and last number in the list. Please kindly answer my post. Thank you!

how to prepare our own dataset with the format as yours?

I would like to apply your method on my own dataset, however, I don't know how to generate the dataset.

TypeError: optimizer must be a dict of torch.optim.Optimizers, but optimizer["type"] is a <class 'str'>

When I started training the model, the errors are as follows:

Traceback (most recent call last):

File "tools/train.py", line 90, in

main()

File "tools/train.py", line 86, in main

logger=logger)

File "/home/zhangsai/.conda/envs/VIS/lib/python3.6/site-packages/mmdet/apis/train.py", line 59, in train_ detector

_ non_ dist_ train(model, dataset, cfg, validate=validate)

File "/home/zhangsai/.conda/envs/VIS/lib/python3.6/site-packages/mmdet/apis/train.py", line 113, in _ non_ dist_ train

cfg.log_ level)

File "/home/zhangsai/.conda/envs/VIS/lib/python3.6/site-packages/mmcv/runner/epoch_based_runner.py", line 187, in __ init__

super().__ init__ (*args, **kwargs)

File "/home/zhangsai/.conda/envs/VIS/lib/python3.6/site-packages/mmcv/runner/base_runner.py", line 84, in __ init__

f'optimizer must be a dict of torch.optim.Optimizers, '

TypeError: optimizer must be a dict of torch.optim.Optimizers, but optimizer["type"] is a <class 'str'>
Hope to get your reply！

Can the vis dataset extract only the mask of a specific class such as person?

Is the format of annotation exactly the same as the coco dataset? @linjieyangsc
Can you give me some references?

How to change the baseline to support training per video?

Hello, thanks for your new task and baseline. I have read some papers from the top ranks in VIS competition, and I have found that most of them had improved the tracking part or post processed after image instance segmentation model. In contrast, I'd like to try utilizing spatial-temporal feature across frames, such as 3D CNN, feature aggregation, etc. But I have encounted some problems in the programming.
Is it necessary to change the baseline(mmdetection) to support training multiple frames per video firstly, otherwise I have no idea to input multiple frames and aggregate features simultaneously, and resolve the scale inconsistency across videos.
Thanks for reading.

Transfer learning

Hi, thanks for sharing this project, it is awesome!
Do you plan to extend it to allow for easier transfer learning, like Mask RCNN?
That would be a killer feature!

Mask-RCNN-baseline results

Hi! Thanks for share your codes and I have some question about your paper. In your paper, I didn't find per frame mask-rcnn baseline results? Is it in table 2 row 1??? (16.2 AP) or it is Mask-Track-RCNN baseline on this dataset?

Using mmcv to load model with unmatched weight dimensions

When using mmcv to load pytorch model with unmatched weight dimensions, it will raise an exception. Make the following change to load such a model.

# [python_lib_path]/site-packages/mmcv/runner/checkpoint.py
# L53, change to print
            print('While copying the parameter named {}, '
                  'whose dimensions in the model are {} and '
                  'whose dimensions in the checkpoint are {}.'
                  .format(name, own_state[name].size(),
                          param.size()))

Something is wrong in your YTVOSDataset code

In the _parse_ann_info function, when fetching gt_mask_polys, you only consider the case that your mask is polygen format, donnot consider when it is the uncompressed RLE format, in which case segm is a dict not a list.
This bug does not cause impact, since the getitem function does not return mask_polys, it returns masks.

There are many problems with the annotation

How to solve the version mismatch of MMdetection 2.0?

Like mention above ! Thanks !

The segementation of instances_val_sub.json

I found that there was no segmentation information in the label of YouTubevis's Valid label.json

val annotations

Hi,
Are you going to release the annotations for the validation set?
Thanks.

The checkpoint loading error

While the current model detection head has 41 channels, the checkpoint detection head has 81 channels. This mismatch produces a runtime error when loading the checkpoint.
What should I do?

Mask size doesn't match

Hi. When I want to evaluate the results generated using your pretrained model on the codalab, the output log is like

creating index...
index created!
Running per image evaluation...
Evaluate annotation type segm
Mask sizes in video 4 and category 1 may not match!
Mask sizes in video 5 and category 23 may not match!
Mask sizes in video 6 and category 20 may not match!
Mask sizes in video 7 and category 1 may not match!
Mask sizes in video 8 and category 17 may not match!
Mask sizes in video 9 and category 19 may not match!
Mask sizes in video 11 and category 11 may not match!
Mask sizes in video 12 and category 1 may not match!
Mask sizes in video 14 and category 10 may not match!
Mask sizes in video 14 and category 10 may not match!

When I check I notice that in valid_submission_sample.json the mask size seems to be [720, 1280] whie in the results.json generated by the pretrained.pth you give the size is [360, 640]. I am wondering why.

Thank you very much.

ror_align_cuda

when i train the model, there is an error:
ImportError: cannot import name 'roi_align_cuda' from 'mmdet.ops.roi_align'
can you tell me how to fix the error?

How to transfer youtube-vis to coco-style

Has anyone successfully run this project,I want to consult some questions.Thank you very much.

-

Reproducibility issue

Hi, @linjieyangsc

First of all, thank you for sharing this great work.
I have a simple question regarding reproducibility of the model.

Q) I trained the model provided here with default settings in the configuration file on 4 GPUs.
Even though I trained multiple times, the results only reached under 30 mAP (28.0 ~ 29.7).
To achieve an accuracy reported in the paper (i.e., 30.2 mAP), what should I do?