hding2455 / dsc Goto Github PK

released code for CVPR2021: Deeply Shape-guided Cascade for Instance Segmentation

License: Other

Python 82.75% C++ 7.67% Cuda 9.47% Shell 0.12%

dsc's Introduction

Deeply Shape-guided Cascade for Instance Segmentation

This repo hosts the code for implementing the DSC algorithms for instance segmentation.

Deeply Shape-guided Cascade for Instance Segmentation,
Hao Ding, Siyuan Qiao, Alan Yuille, Wei Shen
In: Proc. IEEE Conference Computer Vision Pattern Recognition(CVPR), 2021
arXiv preprint (arXiv 1911.11263)

Highlights

Imposing shape guidance: DSC improved the original HTC baseline by imposing shape guidance both explicitly and implicitly.
State-of-the-art performance: Our best single model based on ResNeXt-101-64x4d + FPN with deformable convolutions achieves 51.8% in box AP and 45.5% in mask AP on COCO test-dev (without multi-scale testing)
Better improvement for huddled instances: With shape guidance, DSC's improvement is more significant on images with more huddled instances.

Installation

This implementation is based on mmdetection(v2.1.0). Please refer to INSTALL.md GET_STARTED.md for installation and dataset preparation. A list of installed packages information from anaconda is provided in conda_list.txt

Model Zoo

We provide the following trained models on COCO

Model	Backbone	Multi-scale training	Lr schd	box AP	Mask AP	config	Model
F-DSC	R50-FPN	No	1x	44.5	39.5	config	download
DSC	R50-FPN	No	1x	45.0	39.7	config	download
DSC	R50-FPN	No	20e	45.8	40.1	config	download
DSC	R101-FPN	No	20e	46.6	40.7	config	download
DSC	X101-32x4d-FPN	No	20e	48.0	41.9	config	download
DSC	X101-64x4d-DCN-FPN	Yes	20e	51.4	44.9	config	download

Usage

Train with multiple GPUs

./tools/dist_train.sh ${CONFIG_FILE} ${GPU_NUM}

Example: 
./tools/dist_train.sh configs/dsc/fast_dsc_r50_fpn_1x_coco.py  8

Train with single GPU

python tools/train.py ${CONFIG_FILE}

Example:
python tools/train.py configs/dsc/fast_dsc_r50_fpn_1x_coco.py

Testing

# multi-gpu testing
./tools/dist_test.sh ${CONFIG_FILE} ${CHECKPOINT_FILE} ${GPU_NUM} --out  ${OUTPUT_FILE} --eval bbox segm

Example: 
./tools/dist_test.sh configs/dsc/fast_dsc_r50_fpn_1x_coco.py  f_dsc_r50_fpn_1x_coco.pth 8 --out results.pkl --eval bbox segm

# single-gpu testing
python tools/test_ins.py ${CONFIG_FILE} ${CHECKPOINT_FILE} --show --out  ${OUTPUT_FILE} --eval segm

Example: 
python tools/test_ins.py configs/dsc/fast_dsc_r50_fpn_1x_coco.py  f_dsc_r50_fpn_1x_coco.pth --out  results.pkl --eval bbox segm

Citations

Please consider citing our papers in your publications if this repo helps you.

@inproceedings{ding2021dsc,
  title     =  {Deeply Shape-guided Cascade for Instance Segmentation},
  author    =  {Ding, Hao and Qiao, Siyuan and Yuille, Alan and Shen, Wei},
  booktitle =  {IEEE Conf. Comput. Vis. Pattern Recog. (CVPR)},
  year      =  {2021}
}

License

For academic use, this project is licensed under the 2-clause BSD License - see the LICENSE file for details. For commercial use, please contact Hao Ding (email) and Wei Shen.

dsc's People

Contributors

Stargazers

Watchers

Forkers

eshasadia

dsc's Issues

Can only PNG format pictures be used in the dataset

Is there a demo that can be quickly used for instance segmentation of pictures or videos?

FileNotFoundError: [Errno 2] No such file or directory: 'custom_data/train_persimmon/DSC01274.png'

FileNotFoundError: [Errno 2] No such file or directory: 'custom_data/train_persimmon/DSC01274.png'
The pictures in my JSON file are saved in JPG format. Why can't I find the pictures in PNG format？

gt semantic error

le "/opt/conda/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/kaggle/working/mmdetection/mmdet/datasets/custom.py", line 194, in getitem
data = self.prepare_train_img(idx)
File "/kaggle/working/mmdetection/mmdet/datasets/custom.py", line 217, in prepare_train_img
return self.pipeline(results)
File "/kaggle/working/mmdetection/mmdet/datasets/pipelines/compose.py", line 40, in call
data = t(data)
File "/kaggle/working/mmdetection/mmdet/datasets/pipelines/formating.py", line 318, in call
data[key] = results[key]
KeyError: 'gt_semantic_seg'

Can i use it to test other dataset (eg. signle image)?

An error occurred when I changed the variable "stage", I want to investigate the effect of each stage on the final mask.

KeyError: 'DSC is not in the models registry'

evaluating model on Cityscapes

Hello, I started training the model using the Cityscapes dataset (I have also converted the annotations to COCO style) and I wanted to run the test.py file. I have used both cityscapes and bbox & segm as eval args, both together and separately, but I keep obtaining the following results:
With cityscapes arg:

###############################
what : AP AP_50%
###############################
person : nan nan
rider : nan nan
car : nan nan
truck : nan nan
bus : nan nan
train : nan nan
motorcycle : nan nan
bicycle : nan nan
average : nan nan

With bbox (and segm) args:

Evaluating bbox...
Loading and preparing results...
DONE (t=1.14s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type bbox
DONE (t=2.55s).
Accumulating evaluation results...
DONE (t=0.68s).
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = -1.000
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = -1.000
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = -1.000
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = -1.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = -1.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = -1.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000

However, while training, at the end of every epoch, after performing the eval, I get normal looking results, eg. 0.390 AP.
This is the command that I run:
python tools/test.py configs/dsc/dsc_r50_fpn_1x_cityscapes.py work_dirs/dsc_r50_fpn_1x_cityscapes/epoch_8.pth --out results_city_ep8.pkl --eval cityscapes bbox segm
Are there other arguments that I should add to the command? Thanks!

About .pkl file visualization

Hello, I have two questions to ask. I use "python tools/test_ins.py configs/dsc/fast_dsc_r50_fpn_1x_coco.py f_dsc_r50_fpn_1x_coco.pth --out results.pkl --eval bbox segm
" to test the COCO dataset, but the generated .pkl file, what should I do to achieve its visualization?
I am going to do the GPU training data set "python tools/train.py configs/dsc/fast_dsc_r50_fpn_1x_coco.py". Will this directly generate the .pth model.
Thank you for your contribution to the published paper

bbox feature' s channels are 256,however the mask‘s channels are 81,if they do multiplication,they will report an error

设置0.0标志位时会报错， The shape of the mask [2006] at index 0 does not match the shape of the indexed tensor [512, 4] at index 0

bbox_pred_for_regress = torch.zeros_like(bbox_results['bbox_pred'])
bbox_pred_for_regress[:, :] = bbox_results['bbox_pred'][:, :]
bbox_pred_for_regress[real_gt_flags] = 0.0

No of classes or semantic head

How should i set classes for semantic head.
For instance segmentation i have got three classes. and in coco bbox head its 80

but semantic head its 183 ,dont know what to set there .please help
is it 2,(0,1) or 3+1= bg+3 foreground

Multiple Issue..Difficult to use this

@hding2455
I was unable to run this repo,even with the current version that you use here.
Issue 1

mmdet.version was missing , i resolved it
issue 2 , CONWS already registered, i resolved it
unresolvable cannot import .cornor_pool_ext from corner_pool.py

With new version there is difference between file of yours and SamplingResults of version after your version of MMDET

Please suggest how to run.. I tried all possible bits but finally got stuck here.

DSC not in registry

M not able to register the DSC to mm detect.
I copy all the files that are needed for dsc.We cannot copy whole repo or else it might break some thing in latest version. So I am copying whatever got changed wrt to framework. let me know which step is still missing

!mkdir /kaggle/working/mmdetection/configs/dsc
!cp  /kaggle/input/dsc-configs/dsc_r50_fpn_1x_coco.py  /kaggle/working/mmdetection/configs/dsc
!cp /kaggle/input/dsc-configs/dsc_x101_32x4d_fpn_20e_coco.py  /kaggle/working/mmdetection/configs/dsc
!cp /kaggle/input/dsc-configs/dsc.py /kaggle/working/mmdetection/mmdet/models/detectors/
!cp /kaggle/input/dsc-configs/dsc_roi_head.py /kaggle/working/mmdetection/mmdet/models/roi_heads/
!cp /kaggle/input/dsc-configs/roi_heads_init.py /kaggle/working/mmdetection/mmdet/models/roi_heads/__init__.py

!cp ../input/dsc-configs/dsc_mask_head.py /kaggle/working/mmdetection/mmdet/models/roi_heads/mask_heads/

!cp /kaggle/input/dsc-configs/mask_head_init.py /kaggle/working/mmdetection/mmdet/models/roi_heads/mask_heads/__init__.py
!cp ../input/dsc-configs/dsc_bbox_head.py /kaggle/working/mmdetection/mmdet/models/roi_heads/bbox_heads/

!cp /kaggle/input/dsc-configs/bbox_heads_init.py /kaggle/working/mmdetection/mmdet/models/roi_heads/bbox_heads/__init__.py

Urgent:DSCMaskHead Num classes issue

I set all num classes in heads as per my problem but still
i get an error as
Thenum_classes(80) in DSCMaskHead of MMDataParallel does not matches the length ofCLASSES 3) in CocoDataset

for head in cfg.model.roi_head.bbox_head:
head.num_classes = 3

cfg.model.roi_head.mask_head.num_classes=3