youtubevos / masktrackrcnn Goto Github PK
View Code? Open in Web Editor NEWThis project forked from open-mmlab/mmdetection
MaskTrackRCNN for video instance segmentation based on mmdetection
License: Apache License 2.0
This project forked from open-mmlab/mmdetection
MaskTrackRCNN for video instance segmentation based on mmdetection
License: Apache License 2.0
While the current model detection head has 41 channels, the checkpoint detection head has 81 channels. This mismatch produces a runtime error when loading the checkpoint.
What should I do?
I use the pretrained model epoch_12.pth from Google Drive, and then evaluate the results on the codalab server, but i got mAP: 0.248, however, it should be 0.3 as reported in the paper. Any suggestion? Thank you!
When using mmcv to load pytorch model with unmatched weight dimensions, it will raise an exception. Make the following change to load such a model.
# [python_lib_path]/site-packages/mmcv/runner/checkpoint.py
# L53, change to print
print('While copying the parameter named {}, '
'whose dimensions in the model are {} and '
'whose dimensions in the checkpoint are {}.'
.format(name, own_state[name].size(),
param.size()))
Hi,
Thank you very much for your inspiring work and sharing your code! I tried your code on my own dataset, but I even got bad results of detection compared to the detection results with mmdetection library. Do your have any idea for the possible reasons? I want to try maskrcnn with your code to find the problem, but I it seem that a track head is always needed. Is there anyway to perform maskrcnn only with your code directly?
Your help is greatly appreciated. Thank you very much.
Regards,
Annabel
hi,when I run the command "python tools/train.py configs/masktrack_rcnn_r50_fpn_1x_youtubevos.py", it returned an error which was "ModuleNotFoundError: No module named 'pycocotools.ytvos'". It seems like the 'pycocotools.ytvos' is your modified and new added part. Could you please tell me how to solve it?
As is showed below, my training ends with Segmentation fault, Could someone tell me why ?
(MaskTrackRCNN) [lijie@yq01-gpu-255-126-19-00 MaskTrackRCNN-master]$ python3 tools/train.py configs/masktrack_rcnn_r50_fpn_1x_youtubevos.py
2019-12-17 21:52:08,555 - INFO - Distributed training: False
2019-12-17 21:52:09,311 - INFO - load model from: modelzoo://resnet50
2019-12-17 21:52:09,625 - WARNING - unexpected key in source state_dict: fc.weight, fc.bias
missing keys in source state_dict: layer4.0.downsample.1.num_batches_tracked, layer3.4.bn2.num_batches_tracked, layer2.1.bn3.num_batches_tracked, layer1.0.bn1.num_batches_tracked, layer1.1.bn1.num_batches_tracked, layer2.3.bn1.num_batches_tracked, layer3.5.bn2.num_batches_tracked, layer2.0.bn3.num_batches_tracked, layer3.3.bn3.num_batches_tracked, layer3.2.bn3.num_batches_tracked, layer3.4.bn3.num_batches_tracked, layer2.2.bn1.num_batches_tracked, layer3.2.bn2.num_batches_tracked, layer3.1.bn2.num_batches_tracked, layer1.0.downsample.1.num_batches_tracked, bn1.num_batches_tracked, layer4.2.bn1.num_batches_tracked, layer2.1.bn1.num_batches_tracked, layer3.0.bn3.num_batches_tracked, layer1.2.bn3.num_batches_tracked, layer3.4.bn1.num_batches_tracked, layer1.0.bn2.num_batches_tracked, layer2.2.bn2.num_batches_tracked, layer2.0.bn1.num_batches_tracked, layer2.1.bn2.num_batches_tracked, layer4.0.bn2.num_batches_tracked, layer2.0.bn2.num_batches_tracked, layer3.5.bn3.num_batches_tracked, layer3.3.bn2.num_batches_tracked, layer1.0.bn3.num_batches_tracked, layer4.1.bn3.num_batches_tracked, layer3.1.bn1.num_batches_tracked, layer3.5.bn1.num_batches_tracked, layer1.1.bn2.num_batches_tracked, layer4.0.bn3.num_batches_tracked, layer3.3.bn1.num_batches_tracked, layer4.1.bn2.num_batches_tracked, layer2.3.bn2.num_batches_tracked, layer4.1.bn1.num_batches_tracked, layer2.3.bn3.num_batches_tracked, layer3.1.bn3.num_batches_tracked, layer2.0.downsample.1.num_batches_tracked, layer3.0.bn2.num_batches_tracked, layer4.2.bn2.num_batches_tracked, layer1.2.bn2.num_batches_tracked, layer4.0.bn1.num_batches_tracked, layer1.1.bn3.num_batches_tracked, layer3.2.bn1.num_batches_tracked, layer3.0.downsample.1.num_batches_tracked, layer2.2.bn3.num_batches_tracked, layer1.2.bn1.num_batches_tracked, layer3.0.bn1.num_batches_tracked, layer4.2.bn3.num_batches_tracked
loading annotations into memory...
Done (t=14.82s)
creating index...
index created!
2019-12-17 21:53:59,679 - INFO - load checkpoint from https://s3.ap-northeast-2.amazonaws.com/open-mmlab/mmdetection/models/mask_rcnn_r50_fpn_1x_20181010-069fa190.pth
2019-12-17 21:53:59,879 - WARNING - missing keys in source state_dict: track_head.fcs.0.bias, track_head.fcs.1.bias, track_head.fcs.1.weight, track_head.fcs.0.weight
2019-12-17 21:53:59,900 - INFO - Start running, host: lijie@yq01-gpu-255-126-19-00, work_dir: /home/lijie/MaskTrackRCNN-master/work_dirs/masktrack_rcnn_r50_fpn_1x_youtubevos
2019-12-17 21:53:59,900 - INFO - workflow: [('train', 1)], max: 12 epochs
Segmentation fault
Hello, thanks for your new task and baseline. I have read some papers from the top ranks in VIS competition, and I have found that most of them had improved the tracking part or post processed after image instance segmentation model. In contrast, I'd like to try utilizing spatial-temporal feature across frames, such as 3D CNN, feature aggregation, etc. But I have encounted some problems in the programming.
Is it necessary to change the baseline(mmdetection) to support training multiple frames per video firstly, otherwise I have no idea to input multiple frames and aggregate features simultaneously, and resolve the scale inconsistency across videos.
Thanks for reading.
Hi, @linjieyangsc
First of all, thank you for sharing this great work.
I have a simple question regarding reproducibility of the model.
Q) I trained the model provided here with default settings in the configuration file on 4 GPUs.
Even though I trained multiple times, the results only reached under 30 mAP (28.0 ~ 29.7).
To achieve an accuracy reported in the paper (i.e., 30.2 mAP), what should I do?
As mention above
Hi! Thanks for share your codes and I have some question about your paper. In your paper, I didn't find per frame mask-rcnn baseline results? Is it in table 2 row 1??? (16.2 AP) or it is Mask-Track-RCNN baseline on this dataset?
Hi. When I want to evaluate the results generated using your pretrained model on the codalab, the output log is like
creating index...
index created!
Running per image evaluation...
Evaluate annotation type segm
Mask sizes in video 4 and category 1 may not match!
Mask sizes in video 5 and category 23 may not match!
Mask sizes in video 6 and category 20 may not match!
Mask sizes in video 7 and category 1 may not match!
Mask sizes in video 8 and category 17 may not match!
Mask sizes in video 9 and category 19 may not match!
Mask sizes in video 11 and category 11 may not match!
Mask sizes in video 12 and category 1 may not match!
Mask sizes in video 14 and category 10 may not match!
Mask sizes in video 14 and category 10 may not match!
When I check I notice that in valid_submission_sample.json the mask size seems to be [720, 1280] whie in the results.json generated by the pretrained.pth you give the size is [360, 640]. I am wondering why.
Thank you very much.
I'm trying to train the model with youtube-vis dataset following the instructions but got loss_match equals zero at the beginning.
Do you have any idea about this?
Here is the error info:
loss {'loss_rpn_cls': [tensor([0.5429], device='cuda:0', grad_fn=), tensor([0.1198], device='cuda:0', grad_fn=), tensor([0.0242], device='cuda:0', grad_fn=), tensor([0.0027], device='cuda:0', grad_fn=), tensor([0.], device='cuda:0', grad_fn=)], 'loss_rpn_reg': [tensor([0.], device='cuda:0', grad_fn=), tensor([0.], device='cuda:0', grad_fn=), tensor([0.0009], device='cuda:0', grad_fn=), tensor([0.], device='cuda:0', grad_fn=), tensor([0.], device='cuda:0', grad_fn=)], 'loss_cls': tensor([3.6361], device='cuda:0', grad_fn=), 'acc': tensor([1.4925], device='cuda:0'), 'loss_reg': tensor([4.3604e-06], device='cuda:0', grad_fn=), 'loss_match': 0.0, 'loss_mask': tensor([0.7678], device='cuda:0', grad_fn=)}
Traceback (most recent call last):
File "tools/train.py", line 93, in
main()
File "tools/train.py", line 89, in main
logger=logger)
File "/data/qianyu/detection/MaskTrackRCNN/mmdet/apis/train.py", line 60, in train_detector
_non_dist_train(model, dataset, cfg, validate=validate)
File "/data/qianyu/detection/MaskTrackRCNN/mmdet/apis/train.py", line 122, in _non_dist_train
runner.run(data_loaders, cfg.workflow, cfg.total_epochs)
File "/home/qianyu/anaconda3/envs/p3t4/lib/python3.7/site-packages/mmcv/runner/runner.py", line 358, in run
epoch_runner(data_loaders[i], **kwargs)
File "/home/qianyu/anaconda3/envs/p3t4/lib/python3.7/site-packages/mmcv/runner/runner.py", line 264, in train
self.model, data_batch, train_mode=True, **kwargs)
File "/data/qianyu/detection/MaskTrackRCNN/mmdet/apis/train.py", line 39, in batch_processor
loss, log_vars = parse_losses(losses)
File "/data/qianyu/detection/MaskTrackRCNN/mmdet/apis/train.py", line 25, in parse_losses
'{} is not a tensor or list of tensors'.format(loss_name))
TypeError: loss_match is not a tensor or list of tensors
Would you mind to release your code for change results.pkl to JSON file for evaluation?
I used the mmdetection from f3a939f
It seems like torch 0.4.1 don't support the SynBatchNorm Libraray which is in the mmdetection part.
Can I ask the how to handle SyncBatchNorm.
AttributeError: module 'torch.nn' has no attribute 'SyncBatchNorm'
python3 tools/test_video.py configs/masktrack_rcnn_r50_fpn_1x_youtubevos.py models/epoch_12.pth --out output --eval segm
Traceback (most recent call last):
File "tools/test_video.py", line 4, in
import mmcv
File "/home/user/anaconda3/envs/MaskTrackRCNN/lib/python3.6/site-packages/mmcv/init.py", line 4, in
from .fileio import *
File "/home/user/anaconda3/envs/MaskTrackRCNN/lib/python3.6/site-packages/mmcv/fileio/init.py", line 4, in
from .io import dump, load, register_handler
File "/home/user/anaconda3/envs/MaskTrackRCNN/lib/python3.6/site-packages/mmcv/fileio/io.py", line 4, in
from ..utils import is_list_of, is_str
File "/home/user/anaconda3/envs/MaskTrackRCNN/lib/python3.6/site-packages/mmcv/utils/init.py", line 8, in
from .parrots_wrapper import (CUDA_HOME, BuildExtension, CppExtension,
File "/home/user/anaconda3/envs/MaskTrackRCNN/lib/python3.6/site-packages/mmcv/utils/parrots_wrapper.py", line 67, in
_BatchNorm, InstanceNorm, SyncBatchNorm = _get_norm()
File "/home/user/anaconda3/envs/MaskTrackRCNN/lib/python3.6/site-packages/mmcv/utils/parrots_wrapper.py", line 60, in get_norm
SyncBatchNorm = torch.nn.SyncBatchNorm
AttributeError: module 'torch.nn' has no attribute 'SyncBatchNorm'
The pretained weights of Mask R-CNN is not available.
Could you provide a link to download the weights.
Dear author, thanks for your work!
Today I upload the 'valid_submission_sample' to the evaluation server, however, it reports an error like below:
Traceback (most recent call last):
File "/worker/worker.py", line 302, in run
root_dir = tempfile.mkdtemp(dir=temp_dir)
File "/usr/local/lib/python2.7/tempfile.py", line 339, in mkdtemp
_os.mkdir(file, 0700)
OSError: [Errno 30] Read-only file system: '/tmp/codalab/tmpcP1K7y'
Could you have a look about this?
Thanks in advance!
Has anyone successfully run this project,I want to consult some questions.Thank you very much.
Hi,
Has anybody tried this software on http://www.cvlibs.net/datasets/kitti/eval_mots.php?
Do you have results?
Hello, thanks for releasing this work! I did not understand how to organize the dataset into the form that you exposed on the README.md. Could you please explain more?
I would like to apply your method on my own dataset, however, I don't know how to generate the dataset.
we could only download train.json or meta.json, where can we find instances_train_sub.json?
It would be really appreciated if someone could give me some hint, Thanks ahead!
Is the format of annotation exactly the same as the coco dataset? @linjieyangsc
Can you give me some references?
I tried very hard to run your code on Google colab . But I did not succeed.
Is it possible for you to implement this code in Google colab and publish it?
Question 2: Does your code run in Python 3.7?
I found that there was no segmentation information in the label of YouTubevis's Valid label.json
Hi, thank you for sharing this good benchmark task and the baseline code.
I have some questions when reproducing the baseline.
I can not find the information about the batch size in your paper.
You have mentioned that you use four GPU for training. Do you use 4*8 as the training batchsize?
I have trained a model using your default settings with one single GPU. Now I want to evaluate the results. However, the link https://competitions.codalab.org/competitions/23215 does not work.
I found a submit server here: https://competitions.codalab.org/competitions/20128#participate-submit_results
After running the test_video.py, I have got an output.pkl and an output.json.
It also prints the information like:
"Starting evaluate segm
Annotations does not exist"
Is it a normal one?
When I try to upload the json, the website gives a piece of error information "Invalid file type (application/json)."
Can you help me with this?
when i train the model, there is an error:
ImportError: cannot import name 'roi_align_cuda' from 'mmdet.ops.roi_align'
can you tell me how to fix the error?
Like mention above ! Thanks !
when I use "python3 tools/test_video.py configs/masktrack_rcnn_r50_fpn_1x_youtubevos.py ./epoch_12.pth --out ./output --eval segm" to run the code. I got this error, but I have already installed 'mmcv' package.
I have no idea about this, could you please tell me what should I do?
The previous evaluation code has a bug that it uses unscaled bbox coordinates to extract features for identity matching. It results in lower accuracy when tracking head does not use additional cues for matching. When additional cues (IoU, category, detection score) are used, the performance does not change much.
commit 4e92e86
Tested with the current model
Due to this bug, some results in the arXiv paper is also changed. We will update the results in the arXiv paper shortly. Sorry for any inconvenience caused by this issue.
building 'nms_cuda' extension
gcc -pthread -B /home/miruware/anaconda3/envs/py36_2/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/miruware/anaconda3/envs/py36_2/lib/python3.7/site-packages/torch/lib/include -I/home/miruware/anaconda3/envs/py36_2/lib/python3.7/site-packages/torch/lib/include/TH -I/home/miruware/anaconda3/envs/py36_2/lib/python3.7/site-packages/torch/lib/include/THC -I/usr/local/cuda/include -I/home/miruware/anaconda3/envs/py36_2/include/python3.7m -c src/nms_cuda.cpp -o build/temp.linux-x86_64-3.7/src/nms_cuda.o -DTORCH_EXTENSION_NAME=nms_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
src/nms_cuda.cpp:2:29: fatal error: torch/extension.h: No such file or directory
compilation terminated.
error: command 'gcc' failed with exit status 1
Building dcn...
running build_ext
building 'deform_conv_cuda' extension
creating build
creating build/temp.linux-x86_64-3.7
creating build/temp.linux-x86_64-3.7/src
gcc -pthread -B /home/miruware/anaconda3/envs/py36_2/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/miruware/anaconda3/envs/py36_2/lib/python3.7/site-packages/torch/lib/include -I/home/miruware/anaconda3/envs/py36_2/lib/python3.7/site-packages/torch/lib/include/TH -I/home/miruware/anaconda3/envs/py36_2/lib/python3.7/site-packages/torch/lib/include/THC -I/usr/local/cuda/include -I/home/miruware/anaconda3/envs/py36_2/include/python3.7m -c src/deform_conv_cuda.cpp -o build/temp.linux-x86_64-3.7/src/deform_conv_cuda.o -DTORCH_EXTENSION_NAME=deform_conv_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
src/deform_conv_cuda.cpp:4:29: fatal error: torch/extension.h: No such file or directory
compilation terminated.
error: command 'gcc' failed with exit status 1
Building sigmoid focal loss op...
running build_ext
building 'sigmoid_focal_loss_cuda' extension
creating build
creating build/temp.linux-x86_64-3.7
creating build/temp.linux-x86_64-3.7/src
gcc -pthread -B /home/miruware/anaconda3/envs/py36_2/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/miruware/anaconda3/envs/py36_2/lib/python3.7/site-packages/torch/lib/include -I/home/miruware/anaconda3/envs/py36_2/lib/python3.7/site-packages/torch/lib/include/TH -I/home/miruware/anaconda3/envs/py36_2/lib/python3.7/site-packages/torch/lib/include/THC -I/usr/local/cuda/include -I/home/miruware/anaconda3/envs/py36_2/include/python3.7m -c src/sigmoid_focal_loss.cpp -o build/temp.linux-x86_64-3.7/src/sigmoid_focal_loss.o -DTORCH_EXTENSION_NAME=sigmoid_focal_loss_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
src/sigmoid_focal_loss.cpp:3:29: fatal error: torch/extension.h: No such file or directory
compilation terminated.
error: command 'gcc' failed with exit status 1
Building masked conv op...
running build_ext
building 'masked_conv2d_cuda' extension
creating build
creating build/temp.linux-x86_64-3.7
creating build/temp.linux-x86_64-3.7/src
gcc -pthread -B /home/miruware/anaconda3/envs/py36_2/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/miruware/anaconda3/envs/py36_2/lib/python3.7/site-packages/torch/lib/include -I/home/miruware/anaconda3/envs/py36_2/lib/python3.7/site-packages/torch/lib/include/TH -I/home/miruware/anaconda3/envs/py36_2/lib/python3.7/site-packages/torch/lib/include/THC -I/usr/local/cuda/include -I/home/miruware/anaconda3/envs/py36_2/include/python3.7m -c src/masked_conv2d_cuda.cpp -o build/temp.linux-x86_64-3.7/src/masked_conv2d_cuda.o -DTORCH_EXTENSION_NAME=masked_conv2d_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
src/masked_conv2d_cuda.cpp:1:29: fatal error: torch/extension.h: No such file or directory
compilation terminated.
error: command 'gcc' failed with exit status 1
I got the compile error when installing the mmdetection library.
The above is a short snippet of the error.
I use
python 3.7.3
pytorch 0.4.1
Could you please give me any suggestion?
Hi, when I try to upload the json file to the codalab( https://competitions.codalab.org/competitions/20128#participate-submit_results), the website gives a piece of error information "Invalid file type (application/json)." Is it normal? How can I evaluate the model? Thank you !
Hi, thank you for sharing good works.
I tried this command, but could not solved.
pip install git+https://github.com/youtubevos/cocoapi.git#"egg=pycocotools&subdirectory=PythonAPI"
Could you advise how to handle this??
(MaskTrackRCNN) user@user:/sdata1/workspace/MaskTrackRCNN$ python3 tools/test_video.py configs/masktrack_rcnn_r50_fpn_1x_youtubevos.py models/epoch_12.pth --out output --eval segm
Traceback (most recent call last):
File "tools/test_video.py", line 8, in
from mmdet import datasets
File "/home/user/anaconda3/envs/MaskTrackRCNN/lib/python3.6/site-packages/mmdet/datasets/init.py", line 1, in
from .custom import CustomDataset
File "/home/user/anaconda3/envs/MaskTrackRCNN/lib/python3.6/site-packages/mmdet/datasets/custom.py", line 11, in
from .extra_aug import ExtraAugmentation
File "/home/user/anaconda3/envs/MaskTrackRCNN/lib/python3.6/site-packages/mmdet/datasets/extra_aug.py", line 5, in
from mmdet.core.evaluation.bbox_overlaps import bbox_overlaps
File "/home/user/anaconda3/envs/MaskTrackRCNN/lib/python3.6/site-packages/mmdet/core/init.py", line 5, in
from .evaluation import * # noqa: F401, F403
File "/home/user/anaconda3/envs/MaskTrackRCNN/lib/python3.6/site-packages/mmdet/core/evaluation/init.py", line 4, in
from .coco_utils import coco_eval, fast_eval_recall, results2json, results2json_videoseg, ytvos_eval
File "/home/user/anaconda3/envs/MaskTrackRCNN/lib/python3.6/site-packages/mmdet/core/evaluation/coco_utils.py", line 5, in
from pycocotools.ytvos import YTVOS
ModuleNotFoundError: No module named 'pycocotools.ytvos'
In the _parse_ann_info function, when fetching gt_mask_polys, you only consider the case that your mask is polygen format, donnot consider when it is the uncompressed RLE format, in which case segm is a dict not a list.
This bug does not cause impact, since the getitem function does not return mask_polys, it returns masks.
When I run 'python3 tools/train.py configs/masktrack_rcnn_r50_fpn_1x_youtubevos.py', an error occured as follow:
File "tools/train.py", line 7, in
from mmdet.datasets import get_dataset
File "/home/lin/python_work/competition/OVIS/MaskTrackRCNN-master/mmdet/datasets/init.py", line 1, in
from .custom import CustomDataset
File "/home/lin/python_work/competition/OVIS/MaskTrackRCNN-master/mmdet/datasets/custom.py", line 11, in
from .extra_aug import ExtraAugmentation
File "/home/lin/python_work/competition/OVIS/MaskTrackRCNN-master/mmdet/datasets/extra_aug.py", line 5, in
from mmdet.core.evaluation.bbox_overlaps import bbox_overlaps
File "/home/lin/python_work/competition/OVIS/MaskTrackRCNN-master/mmdet/core/init.py", line 6, in
from .post_processing import * # noqa: F401, F403
File "/home/lin/python_work/competition/OVIS/MaskTrackRCNN-master/mmdet/core/post_processing/init.py", line 1, in
from .bbox_nms import multiclass_nms
File "/home/lin/python_work/competition/OVIS/MaskTrackRCNN-master/mmdet/core/post_processing/bbox_nms.py", line 3, in
from mmdet.ops.nms import nms_wrapper
File "/home/lin/python_work/competition/OVIS/MaskTrackRCNN-master/mmdet/ops/init.py", line 1, in
from .nms import nms, soft_nms
File "/home/lin/python_work/competition/OVIS/MaskTrackRCNN-master/mmdet/ops/nms/init.py", line 1, in
from .nms_wrapper import nms, soft_nms
File "/home/lin/python_work/competition/OVIS/MaskTrackRCNN-master/mmdet/ops/nms/nms_wrapper.py", line 4, in
from .gpu_nms import gpu_nms
ModuleNotFoundError: No module named 'mmdet.ops.nms.gpu_nms'
How to fix this error? Many thanks.
Hi,
Are you going to release the annotations for the validation set?
Thanks.
I get the following error when running evaluation:
Traceback (most recent call last):
File "./tools/test_video.py", line 8, in
from mmdet import datasets
File "/home/bigboi/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/mmdet/datasets/init.py", line 1, in
from .custom import CustomDataset
File "/home/bigboi/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/mmdet/datasets/custom.py", line 11, in
from .extra_aug import ExtraAugmentation
File "/home/bigboi/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/mmdet/datasets/extra_aug.py", line 5, in
from mmdet.core.evaluation.bbox_overlaps import bbox_overlaps
File "/home/bigboi/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/mmdet/core/init.py", line 6, in
from .post_processing import * # noqa: F401, F403
File "/home/bigboi/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/mmdet/core/post_processing/init.py", line 1, in
from .bbox_nms import multiclass_nms
File "/home/bigboi/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/mmdet/core/post_processing/bbox_nms.py", line 3, in
from mmdet.ops.nms import nms_wrapper
File "/home/bigboi/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/mmdet/ops/init.py", line 2, in
from .roi_align import RoIAlign, roi_align
File "/home/bigboi/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/mmdet/ops/roi_align/init.py", line 1, in
from .functions.roi_align import roi_align
File "/home/bigboi/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/mmdet/ops/roi_align/functions/roi_align.py", line 3, in
from .. import roi_align_cuda
ImportError: cannot import name 'roi_align_cuda' from 'mmdet.ops.roi_align' (/home/bigboi/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/mmdet/ops/roi_align/init.py)
Hi thanks for the nice code framework.
For evaluation, when I run the tools/test_video.py, it does not properly run the evaluation even if I follow the setup described in the readme.
This is the outcome after finishing running test_video.py
writing results to ./data/rst/testouput.pkl
Starting evaluate segm
Annotations does not exist
If there is anything I missed, it seems that it'd be nice to update the readme for evaluation information. Any comment would be appreciated.
Thanks.
Hi, I'm train MaskTrackRCNN following the readme,. However, an error occured.
Traceback (most recent call last):
File "/home/ubuntu/.pycharm_helpers/pydev/pydevd.py", line 2060, in
main()
File "/home/ubuntu/.pycharm_helpers/pydev/pydevd.py", line 2054, in main
globals = debugger.run(setup['file'], None, None, is_module)
File "/home/ubuntu/.pycharm_helpers/pydev/pydevd.py", line 1405, in run
return self._exec(is_module, entry_point_fn, module_name, file, globals, locals)
File "/home/ubuntu/.pycharm_helpers/pydev/pydevd.py", line 1412, in _exec
pydev_imports.execfile(file, globals, locals) # execute the script
File "/home/ubuntu/.pycharm_helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "/home/ubuntu/code/MaskTrackRCNN/tools/train.py", line 99, in
main()
File "/home/ubuntu/code/MaskTrackRCNN/tools/train.py", line 92, in main
logger=logger)
File "/home/ubuntu/code/MaskTrackRCNN/mmdet/apis/train.py", line 59, in train_detector
_non_dist_train(model, dataset, cfg, validate=validate)
File "/home/ubuntu/code/MaskTrackRCNN/mmdet/apis/train.py", line 123, in _non_dist_train
runner.run(data_loaders, cfg.workflow, cfg.total_epochs)
File "/home/ubuntu/Softwares/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/mmcv/runner/runner.py", line 358, in run
epoch_runner(data_loaders[i], **kwargs)
File "/home/ubuntu/Softwares/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/mmcv/runner/runner.py", line 271, in train
self.call_hook('after_train_iter')
File "/home/ubuntu/Softwares/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/mmcv/runner/runner.py", line 229, in call_hook
getattr(hook, fn_name)(self)
File "/home/ubuntu/Softwares/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/mmcv/runner/hooks/optimizer.py", line 17, in after_train_iter
runner.outputs['loss'].backward()
File "/home/ubuntu/Softwares/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/torch/tensor.py", line 118, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File "/home/ubuntu/Softwares/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/torch/autograd/init.py", line 93, in backward
allow_unreachable=True) # allow_unreachable flag
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [148, 256, 14, 14]], which is output 0 of IndexPutBackward, is at version 5; expected version 4 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).
with:
cuda:9.2
pytorch:1.2.0
Could you please give me any suggestion?
Hi, thanks for sharing this project, it is awesome!
Do you plan to extend it to allow for easier transfer learning, like Mask RCNN?
That would be a killer feature!
Hello,
Thanks for your nice code again.
I am curious about when the evaluation code would be available.
Thanks,
Hi, Linjie.
under the default setting, I train a model using a single GPU. But it seems to be difficult to achieve 30.3 MAP. Besides, I note that the learning rate is 0.05 in your paper, while 0.005 in your code?
Thanks
Please refer to youtubevos/cocoapi#1 (comment)
When I started training the model, the errors are as follows:
Traceback (most recent call last):
File "tools/train.py", line 90, in
main()
File "tools/train.py", line 86, in main
logger=logger)
File "/home/zhangsai/.conda/envs/VIS/lib/python3.6/site-packages/mmdet/apis/train.py", line 59, in train_ detector
_ non_ dist_ train(model, dataset, cfg, validate=validate)
File "/home/zhangsai/.conda/envs/VIS/lib/python3.6/site-packages/mmdet/apis/train.py", line 113, in _ non_ dist_ train
cfg.log_ level)
File "/home/zhangsai/.conda/envs/VIS/lib/python3.6/site-packages/mmcv/runner/epoch_based_runner.py", line 187, in __ init__
super().__ init__ (*args, **kwargs)
File "/home/zhangsai/.conda/envs/VIS/lib/python3.6/site-packages/mmcv/runner/base_runner.py", line 84, in __ init__
f'optimizer must be a dict of torch.optim.Optimizers, '
TypeError: optimizer must be a dict of torch.optim.Optimizers, but optimizer["type"] is a <class 'str'>
Hope to get your reply!
[139447, 42, 678, 42, 671, 62, 658, 62, 656, 72, 648, 72, 642, 80, 640, 80, 636, 87, 633, 87, 631, 94, 626, 94, 619, 106, 614, 106, 614, 121, 599, 121, 598, 128, 592, 128, 592, 129, 591, 129, 591, 132, 588, 132, 588, 134, 586, 134, 586, 136, 584, 136, 584, 138, 582, 138, 582, 140, 580, 140, 580, 140, 580, 140, 580, 140, 580, 140, 580, 140, 580, 140, 580, 141, 579, 141, 579, 141, 579, 141, 579, 141, 579, 141, 579, 141, 579, 141, 579, 141, 579, 138, 582, 138, 582, 133, 587, 133, 587, 129, 591, 129, 591, 127, 593, 127, 593, 125, 595, 125, 595, 121, 599, 121, 599, 119, 601, 119, 601, 113, 607, 113, 607, 105, 615, 105, 615, 97, 623, 97, 623, 45, 675, 45, 675, 44, 676, 44, 677, 43, 676, 44, 676, 44, 676, 44, 676, 44, 676, 44, 676, 44, 676, 44, 676, 44, 676, 44, 677, 43, 677, 43, 677, 43, 677, 43, 678, 42, 678, 43, 677, 43, 677, 45, 676, 44, 676, 44, 676, 44, 676, 44, 676, 44, 676, 46, 674, 46, 674, 47, 673, 47, 673, 48, 672, 48, 672, 50, 670, 50, 670, 53, 667, 53, 667, 53, 667, 53, 667, 55, 665, 55, 665, 60, 660, 60, 660, 61, 660, 60, 660, 62, 658, 62, 658, 62, 658, 62, 658, 63, 657, 63, 657, 64, 656, 64, 656, 64, 656, 64, 656, 65, 655, 65, 655, 65, 655, 65, 655, 66, 654, 66, 654, 67, 653, 67, 653, 67, 653, 67, 653, 67, 653, 67, 653, 67, 653, 67, 653, 69, 651, 69, 651, 69, 651, 69, 651, 69, 653, 67, 653, 68, 652, 68, 652, 68, 652, 68, 652, 68, 652, 68, 652, 69, 651, 69, 651, 69, 651, 69, 651, 70, 651, 69, 651, 70, 650, 70, 650, 72, 648, 72, 648, 73, 647, 73, 647, 73, 647, 73, 647, 73, 648, 72, 648, 72, 648, 72, 648, 73, 647, 73, 647, 75, 646, 74, 646, 74, 646, 74, 646, 75, 645, 75, 645, 77, 643, 77, 643, 78, 643, 77, 643, 77, 645, 75, 645, 76, 644, 76, 644, 76, 644, 76, 644, 76, 645, 75, 645, 76, 644, 76, 644, 80, 640, 80, 640, 92, 628, 92, 628, 94, 626, 94, 626, 96, 624, 96, 624, 99, 622, 98, 622, 100, 621, 99, 621, 100, 620, 100, 620, 103, 617, 103, 617, 103, 617, 103, 617, 105, 615, 105, 615, 108, 612, 108, 612, 108, 612, 108, 612, 111, 609, 111, 609, 113, 607, 113, 607, 116, 604, 116, 604, 117, 603, 117, 603, 120, 600, 120, 600, 123, 597, 123, 597, 125, 595, 125, 595, 127, 593, 127, 593, 130, 590, 130, 590, 131, 589, 131, 589, 132, 588, 132, 588, 136, 584, 136, 584, 137, 583, 137, 583, 138, 582, 138, 582, 139, 581, 139, 581, 142, 578, 142, 578, 143, 577, 143, 577, 145, 575, 145, 575, 146, 574, 146, 574, 147, 573, 147, 573, 150, 570, 150, 570, 151, 569, 151, 569, 152, 568, 152, 568, 157, 563, 157, 563, 166, 554, 166, 553, 171, 549, 171, 549, 178, 542, 178, 542, 182, 538, 182, 537, 187, 533, 187, 533, 196, 524, 196, 524, 201, 519, 201, 518, 206, 514, 206, 512, 210, 510, 210, 510, 213, 507, 213, 507, 213, 507, 213, 506, 214, 506, 214, 505, 215, 505, 215, 505, 215, 505, 215, 504, 216, 504, 216, 504, 216, 504, 216, 504, 216, 504, 216, 504, 216, 504, 216, 501, 219, 501, 219, 498, 222, 498, 222, 498, 223, 497, 223, 495, 225, 495, 225, 495, 225, 495, 225, 493, 227, 493, 227, 491, 229, 491, 229, 490, 230, 490, 230, 489, 232, 488, 232, 479, 241, 479, 241, 478, 242, 478, 242, 462, 258, 462, 258, 456, 264, 456, 264, 445, 275, 445, 275, 440, 280, 440, 280, 438, 282, 438, 282, 437, 283, 437, 283, 433, 287, 433, 287, 427, 293, 427, 293, 417, 303, 417, 303, 412, 308, 412, 308, 408, 312, 408, 312, 404, 316, 404, 316, 399, 321, 399, 321, 377, 343, 377, 343, 364, 356, 364, 356, 353, 367, 353, 367, 345, 375, 345, 375, 339, 382, 338, 382, 325, 395, 325, 395, 295, 425, 295, 425, 293, 427, 293, 427, 289, 431, 289, 431, 266, 454, 266, 454, 258, 462, 258, 462, 254, 466, 254, 466, 248, 472, 248, 472, 244, 476, 244, 476, 242, 478, 242, 478, 240, 480, 240, 480, 237, 483, 237, 483, 236, 484, 236, 484, 235, 485, 235, 485, 234, 486, 234, 486, 230, 490, 230, 490, 227, 493, 227, 493, 225, 495, 225, 495, 223, 497, 223, 497, 222, 498, 222, 498, 218, 502, 218, 502, 215, 505, 215, 505, 211, 509, 211, 509, 210, 510, 210, 510, 208, 512, 208, 512, 206, 514, 206, 514, 203, 517, 203, 517, 199, 521, 199, 521, 196, 524, 196, 524, 193, 527, 193, 527, 191, 529, 191, 529, 189, 531, 189, 531, 185, 535, 185, 535, 182, 538, 182, 538, 180, 540, 180, 540, 179, 541, 179, 541, 177, 543, 177, 543, 177, 543, 177, 543, 175, 545, 175, 545, 175, 545, 175, 544, 174, 546, 174, 546, 171, 549, 171, 549, 171, 549, 171, 549, 168, 552, 168, 552, 168, 552, 168, 552, 168, 552, 167, 553, 167, 553, 167, 553, 167, 553, 165, 555, 165, 555, 163, 557, 163, 557, 161, 559, 161, 559, 161, 559, 161, 559, 160, 560, 160, 560, 158, 562, 158, 562, 158, 562, 158, 562, 157, 563, 157, 563, 155, 565, 155, 565, 155, 565, 155, 565, 153, 567, 153, 567, 153, 567, 153, 567, 153, 567, 153, 567, 152, 568, 152, 568, 152, 568, 152, 567, 153, 567, 153, 567, 153, 567, 153, 567, 152, 568, 152, 568, 152, 568, 152, 568, 152, 568, 152, 568, 152, 568, 152, 568, 152, 570, 150, 570, 150, 570, 150, 570, 150, 571, 149, 571, 149, 573, 147, 573, 147, 574, 146, 574, 146, 575, 145, 575, 145, 576, 144, 576, 144, 576, 146, 574, 146, 575, 145, 575, 145, 575, 145, 575, 145, 577, 143, 577, 143, 578, 143, 577, 143, 578, 142, 578, 142, 578, 142, 578, 142, 578, 142, 578, 142, 578, 145, 575, 145, 577, 143, 577, 143, 577, 144, 576, 144, 577, 144, 576, 144, 578, 143, 577, 143, 578, 142, 578, 142, 579, 142, 578, 142, 579, 143, 577, 143, 578, 144, 576, 144, 578, 142, 578, 142, 578, 144, 576, 144, 577, 146, 574, 146, 574, 147, 573, 147, 574, 147, 573, 147, 574, 147, 573, 147, 574, 146, 574, 146, 575, 146, 574, 146, 574, 148, 572, 148, 574, 146, 574, 146, 574, 149, 571, 149, 573, 147, 573, 147, 573, 148, 572, 148, 573, 148, 572, 148, 572, 148, 572, 148, 573, 150, 570, 150, 572, 149, 571, 149, 571, 150, 570, 150, 571, 150, 570, 150, 570, 150, 570, 150, 570, 154, 566, 154, 566, 154, 566, 154, 566, 155, 565, 155, 565, 156, 564, 156, 564, 157, 563, 157, 563, 162, 558, 162, 558, 165, 555, 165, 555, 167, 553, 167, 553, 168, 552, 168, 552, 170, 550, 170, 550, 173, 547, 173, 547, 175, 545, 175, 545, 176, 544, 176, 544, 179, 541, 179, 541, 182, 538, 182, 538, 183, 537, 183, 537, 186, 534, 186, 534, 188, 532, 188, 532, 192, 528, 192, 528, 193, 527, 193, 527, 200, 520, 200, 520, 202, 518, 202, 518, 205, 515, 205, 515, 205, 515, 205, 515, 205, 515, 205, 515, 206, 514, 206, 514, 207, 513, 207, 513, 208, 512, 208, 512, 209, 511, 209, 511, 209, 511, 209, 511, 211, 509, 211, 509, 212, 508, 212, 508, 212, 508, 212, 508, 214, 506, 214, 506, 214, 506, 214, 506, 215, 505, 215, 505, 215, 505, 215, 505, 215, 505, 215, 505, 215, 505, 215, 505, 217, 503, 217, 503, 218, 502, 218, 502, 218, 502, 218, 502, 219, 501, 219, 501, 219, 501, 219, 501, 219, 501, 219, 501, 219, 501, 219, 501, 221, 499, 221, 499, 221, 499, 221, 499, 221, 499, 221, 499, 221, 499, 221, 499, 221, 499, 221, 499, 222, 498, 222, 498, 222, 498, 222, 498, 222, 498, 222, 498, 224, 496, 224, 496, 224, 496, 224, 496, 224, 496, 224, 496, 224, 496, 224, 496, 224, 496, 224, 496, 224, 496, 224, 496, 226, 494, 226, 494, 226, 494, 226, 494, 226, 494, 226, 494, 227, 493, 227, 493, 228, 492, 228, 492, 228, 492, 228, 492, 228, 492, 228, 492, 228, 492, 228, 492, 230, 490, 230, 490, 230, 490, 230, 490, 230, 490, 230, 490, 231, 489, 231, 489, 232, 488, 232, 488, 232, 488, 232, 488, 234, 486, 234, 486, 234, 486, 234, 486, 234, 486, 234, 486, 236, 484, 236, 484, 237, 483, 237, 483, 237, 483, 237, 483, 238, 482, 238, 482, 240, 480, 240, 480, 240, 480, 240, 480, 241, 479, 241, 479, 244, 476, 244, 476, 244, 476, 244, 476, 245, 475, 245, 475, 246, 474, 246, 474, 249, 471, 249, 471, 249, 471, 249, 471, 252, 468, 252, 468, 253, 467, 253, 467, 253, 467, 253, 467, 256, 464, 256, 464, 257, 463, 257, 463, 258, 462, 258, 462, 262, 458, 262, 458, 264, 456, 264, 456, 265, 455, 265, 455, 273, 447, 273, 447, 278, 442, 278, 442, 280, 440, 280, 440, 284, 436, 284, 436, 289, 431, 289, 431, 290, 430, 290, 430, 293, 427, 293, 427, 296, 424, 296, 424, 303, 417, 303, 417, 305, 415, 305, 415, 309, 411, 309, 411, 314, 406, 314, 406, 324, 396, 324, 396, 337, 383, 337, 383, 361, 359, 361, 359, 381, 339, 396, 324, 403, 317, 403, 317, 409, 311, 409, 311, 420, 300, 420, 300, 425, 295, 425, 295, 429, 291, 429, 291, 435, 285, 435, 285, 449, 271, 449, 271, 459, 261, 459, 261, 467, 253, 467, 253, 477, 243, 477, 243, 481, 239, 481, 239, 485, 235, 485, 235, 490, 230, 490, 230, 503, 217, 503, 217, 523, 197, 523, 197, 533, 178, 542, 178, 599, 121, 599, 121, 620, 86, 634, 86, 638, 73, 647, 73, 173552]
Hi, I try to resolve the puzzle of the annotation for segmentation. I am wondering what these numbers stand for? I don't think this is polygon for the detected instance. Especially, the first and last number in the list. Please kindly answer my post. Thank you!
Hello, thanks for releasing the code! I have been having trouble initializing this code, I have performed the installation as mentioned on the README.md. But I encountered the following error:
`Traceback (most recent call last):
File "tools/test_video.py", line 4, in
import mmcv
File "/home/daniel/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/mmcv/init.py", line 4, in
from .fileio import *
File "/home/daniel/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/mmcv/fileio/init.py", line 4, in
from .io import dump, load, register_handler
File "/home/daniel/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/mmcv/fileio/io.py", line 4, in
from ..utils import is_list_of, is_str
File "/home/daniel/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/mmcv/utils/init.py", line 29, in
from .parrots_wrapper import (CUDA_HOME, BuildExtension, CppExtension,
File "/home/daniel/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/mmcv/utils/parrots_wrapper.py", line 79, in
_BatchNorm, InstanceNorm, SyncBatchNorm = _get_norm()
File "/home/daniel/anaconda3/envs/MaskTrackRCNN/lib/python3.7/site-packages/mmcv/utils/parrots_wrapper.py", line 71, in get_norm
SyncBatchNorm = torch.nn.SyncBatchNorm
AttributeError: module 'torch.nn' has no attribute 'SyncBatchNorm'
Thank you for your attention!
`
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.