tianzhi0549/py-faster-rcnn-resnet-imagenet

Faster RCNN with ResNet for ImageNet Detection

Faster RCNN with ResNet for ImageNet Detection

Based on Faster RCNN, the repository aims to reproduce the ImageNet Detection results in ResNet paper (Deep Residual Learning for Image Recognition).

So far, it achieves mAP 52.2% (vs. 60.5% in the paper) on val2 of ImageNet 2015 Detection dataset without the use of Box refinement, Global context and Multi-scale testing.

If you have any questions about this code, please create an issue.

What has been done

• Add ImageNet Detection imdb and roidb
• Use ResNet-101
• Multi-gpus training and testing
• Fix some problems when training Faster RCNN over ImageNet

What has NOT been done

• Box refinement
• Global context
• Multi-scale testing

If you are interested in these, please feel free to submit PRs.

How to use this code

If you find any directory in the following steps does not exist, please create it by yourself.

Test with trained models

• Clone this repository and set up the environment as described in the original README.md of Faster RCNN

• Download the models I trained. Populate them into output/faster_rcnn_alt_opt/imagenet_2015_trainval1_woextra.

  resnet-101_rpn_stage1_iter_320000.caffemodel

  resnet-101_fast_rcnn_stage1_iter_320000.caffemodel

• Generate RPN proposals

  python tools/rpn_generate.py --def models/imagenet/ResNet-101/faster_rcnn_alt_opt/rpn_test.pt --net output/faster_rcnn_alt_opt/imagenet_2015_trainval1_woextra/resnet-101_rpn_stage1_iter_320000.caffemodel --imdb imagenet_2015_val2 --gpu 0 1 2 3

• Use Fast RCNN to detect objects with these proposals generated by RPN

  ./tools/test_net.py --gpu 0 1 2 3 --def models/imagenet/ResNet-101/fast_rcnn/test.prototxt --net output/faster_rcnn_alt_opt/imagenet_2015_trainval1_woextra/resnet-101_fast_rcnn_stage1_iter_320000.caffemodel --imdb imagenet_2015_val2 --cfg experiments/cfgs/fast_rcnn.yml --rpn_file output/default/imagenet_2015_val2/resnet-101_rpn_stage1_iter_320000/proposals_test

Once it finishes, you could see the final mAP over val2.

Train it by yourself

Hardware required: 4 or 8 GPU cards are required to train the model because the batch sizes used in the original paper are 8 for RPN and 16 for Fast RCNN.

Please note: because the improvement of stage 3 and 4 is minior, only the first two stages are trained. With 8 Titanx GPU cards, it takes about four days to train.

• Download ImageNet Detection 2015 datasets, and unzip it into directory data. It should have this basic structure

  ImageNet2015/Annotations
  ImageNet2015/Data
  ImageNet2015/ImageSets
  # others...

• Run cp trainval1_woextra.txt val2.txt data/ImageNet2015/ImageSets/DET

• Download the pretranied model of ResNet-101 here, and populate it into data/imagenet_models with name ResNet-101.caffemodel

• Run experiments/scripts/faster_rcnn_alt_opt.sh 0,1,2,3,4,5,6,7 ResNet-101 imagenet to train. 0,1,2,3,4,5,6,7 is GPU ids. Please replace it if you use other GPUs.

py-faster-rcnn-resnet-imagenet's Issues

About the layers to keep fixed when fine-tuning?

From the ResNet-101 proto file, I notice that you keep the parameters fixed before conv4_x when training Faster-RCNN.
Is this also what the authors did? I didn't find relative description in the original ResNet paper.

The VGG16 model of the original Faster-RCNN implementation freeze the layers before conv3_x, which is also how the authors done in R-FCN.

It's interesting to evaluate about the affect of different number of layers to freeze for fine tuning.

Prototxt VGG16 works for Resnet?

Hi, I am a little confused about the sample test command line:
the prototxt passed with --def models/imagenet/ResNet-101/fast_rcnn/test.prototxt starts with
name: "VGG_ILSVRC_16_layers"
while the model is resnet-101_fast_rcnn_stage1_iter_320000.caffemodel.
I also noticed there is no layer with 1000 outputs, the number of labels of imagenet competition, while the file models/imagenet/ResNet-101/ResNet-101-deploy.prototxt has it and starts with name: "ResNet-101".

What am I missing?