rbgirshick/deeppyramid

Deep feature pyramids for various computer vision algorithms (DPMs, pyramid R-CNN, etc.)

DeepPyramid

DeepPyramid is a simple toolkit for building feature pyramids from deep convolutional networks. The DeepPyramid data structure is nearly identical to the HOG feature pyramid created by the featpyramid.m function in the voc-dpm code.

References

This code was used in our tech report about the relationship between deformable part models and convolutional networks.

@article{girshick14dpdpm,
    author    = {Ross Girshick and Forrest Iandola and Trevor Darrell and Jitendra Malik},
    title     = {Deformable Part Models are Convolutional Neural Networks},
    journal   = {CoRR},
    year      = {2014},
    volume    = {abs/1409.5403},
    url       = {http://arxiv.org/abs/1409.5403},
    year      = {2014}
}

Installation

0. Prerequisites
1. MATLAB (tested with 2014a on 64-bit Linux)
2. Caffe's prerequisites
3. Install Caffe (this is the most complicated part)
4. Follow the Caffe installation instructions
5. Let's call the place where you installed caffe $CAFFE_ROOT (you can run export CAFFE_ROOT=$(pwd))
6. Important: Make sure to compile the Caffe MATLAB wrapper, which is not built by default: make matcaffe
7. Important: Make sure to run cd $CAFFE_ROOT/data/ilsvrc12 && ./get_ilsvrc_aux.sh to download the ImageNet image mean
8. DeepPyramid has been tested with master and dev at the time of this writing
9. Get DeepPyramid
10. git clone https://github.com/rbgirshick/DeepPyramid.git
11. If you haven't installed R-CNN, you'll need to download its models
12. Copy R-CNN's non-finetuned ImageNet network <rcnnpath>/data/caffe_nets/ilsvrc_2012_train_iter_310k to <deeppyramidpath>/data/caffe_nets/ilsvrc_2012_train_iter_310k (or just create a symlink).

Usage

1. Run matlab from inside the DeepPyramid code directory
2. Add the matcaffe mex function to your path (addpath /path/to/caffe/matlab/caffe)
3. Run the demo demo_deep_pyramid

Uses

DeepPyramid can be used for implementing DPMs on deep convolutional network features, rather than HOG features. It can also be used whenever you need a dense multiscale pyramid of image features.

Caveats

The implementation is designed to be simple and as a result is very inefficient. There are a variety of ways to speed it up, and they will be done in the future. For now, it takes about 0.5 to 0.6 seconds to compute a feature pyramid on an NVIDIA Titan GPU, which is acceptable.

deeppyramid's Issues

Performance Drop

Did anyone attempt to replicate the results in the paper? Just learning a no-parts single component model on pascal with a sliding window SVM gives me about 33% mAP, as opposed to the papers' 39% mAP (before addition of multiple aspect ratio models). Anyone tried this or can Ross share why this might happen/code for training?

Thank you!

run 'demo_deep_pyramid.m', get error using caffel.....,

Error using caffe
Expected 3 arguments, got 2
Error in init_cnn_model (line 60)
caffe('init', cnn.definition_file, cnn.binary_file);
Error in demo_deep_pyramid (line 43)
cnn = init_cnn_model('use_gpu', USE_GPU, 'use_caffe', USE_CAFFE);

However I could run dp-dpm succeed, with matlab2015,caffe-0.999,Tesla K40

About the Distance Transform Pooling Layer

Hi Ross,

This code only computes the feature pyramid. Would you please provide the Distance transform pooling layer (I guess you implemented a new Caffe layer) as a reference? Thank you.

How to reproduce the result of this paper?

Hi, rbg, i've got the DeepPyramid and voc-release5, then i extracted feature pyramids from deepPyramid and store it. but when i follow the DPM structures, i fail it. In your experiment, how to combine them.

Is there an easy way to use this code in order to add more scales to the feature pyramid?

For instance, not just up to 3.4 upsampling, but even more (a factor of 6 resize up, etc...)

Thank you!

what function is DPM-CNN

what function is DPM-CNN? I can't find it.

how to set t batch_size

at demo_deep_pyramid.m line56, "deep_pyramid" seems do not accept 3 arguments, please check

using caffe for DeepPyramid unknown command init

Dear autor
When I run the demo of DeepPyramid,the file of demo_deep_pyramid.m, I get the error of unknown command "init",I chiose of caffe version of 0.999,but still error .my platform is ubuntu 14.04+latest version caffe+matlab2014a,I complie caffe success,but when I run the demo_deep_pyramid.m,the demo had error .please help me ,I do not know why .plese juse help ,plese!