weichengkuo/deepbox

Code release for DeepBox paper in ICCV 2015

DeepBox: Learning Objectness with Convolutional Networks

Created by Weicheng Kuo at UC Berkeley

Introduction

Fast DeepBox is a bounding box proposal re-ranker using ConvNets. It produces state-of-the-art bounding box proposal within 0.5s using a light-weight 4-layer network. Experiments on both PASCAL and COCO showed that DeepBox performs significantly better than Edge boxes in terms of Area under Curve and that the gain carries over to detection mAP. This implementation is based on Ross's Fast-RCNN codebase, thereby written in Python and C++/Caffe.

DeepBox was initially described in an arXiv paper and later published at ICCV 2015.

License

Fast DeepBox is released under the MIT License (refer to the LICENSE file for details).

Citing Fast DeepBox

If you find Fast DeepBox useful in your research, please consider citing:

@inproceedings{KuoICCV15DeepBox,
    Author = {Weicheng Kuo, Bharath Hariharan, Jitendra Malik},
    Title = {DeepBox:Learning Objectness with Convolutional Networks},
    Booktitle = {International Conference on Computer Vision ({ICCV})},
    Year = {2015}
}

Contents

1. Requirements: software
2. Requirements: hardware
3. Basic installation
4. Demo
5. Beyond the demo: training and testing
6. Usage
7. Extra downloads

Requirements: software

1. Requirements for Caffe and pycaffe (see: Caffe installation instructions)

Note: Caffe doesn't have to be built with support for Python layers!

You can download my Makefile.config for reference. 2. Python packages you might not have: cython, python-opencv, easydict 3. MATLAB (required for running COCO evaluation)

Requirements: hardware

1. A good GPU (e.g., Titan, K20, K40, ...) with at least 3G of memory suffices

Installation (sufficient for the demo)

1. Clone the Fast DeepBox repository

git clone https://github.com/weichengkuo/DeepBox.git

2. We'll call the directory that you cloned Fast DeepBox into FDBOX_ROOT

3. Build the Cython modules

   cd $FDBOX_ROOT/src
   make

4. Build Caffe and pycaffe

   cd $FDBOX_ROOT/caffe-fast-rcnn
   # Now follow the Caffe installation instructions here:
   #   http://caffe.berkeleyvision.org/installation.html
   
   # If you're experienced with Caffe and have all of the requirements installed
   # and your Makefile.config in place, then simply do:
   make -j8 && make pycaffe

5. Download pre-computed Fast DeepBox models

   cd $FDBOX_ROOT/output/default
   ./scripts/fetch_fast_dbox_models.sh

   This will populate the $FDBOX_ROOT/output/default folder with coco_train2014,which contains a variety of models. These include multiscale/single-scale Fast DeepBox models and a multiscale sliding window model for training comparison.

Demo

After successfully completing basic installation, you'll be ready to run the demo.

Python

To run the demo

cd $FDBOX_ROOT
python ./tools/demo.py

By default, visualization only shows top five proposals per image and their scores. Users can set the number of proposals they want to visualize by passing in --numboxes [Number of proposals] argument. There is another demo mode on full COCO dataset that user can choose by passing in --demo 0 argument. This allows users to visualize Fast DeepBox proposals on any frame of users' choice in COCO validation set, but requires users to download pre-computed Edge boxes proposals and install COCO dataset in full.

Beyond the basic demo: installation for full COCO demo, training and testing models

Here are the steps to set up full functionalities of DeeBox package.

1. Download pre-computed Edge boxes proposals

   cd $FDBOX_ROOT/data
   ./scripts/fetch_edge_box_data.sh

   The Edge boxes proposals are pre-computed in order to reduce installation requirements. This step is necessary for demo.

2. Download COCO Matlab data

   cd $FDBOX_ROOT/data
   ./scripts/fetch_coco_matlab_data.sh

   This step downloads the Matlab COCO image ordering with which Edge boxes proposals are computed. Ground truth boxes would also be downloaded to enable evaluation.

3. Set up Microsoft COCO directory by

   cd $FDBOX_ROOT/data
   ln -s PATH/TO/YOUR/COCO ./MSCOCO

   If you haven't installed COCO on your machine yet, you can follow the instructions on the following github page to download and compile all the data. -MSCOCO

4. Download pre-trained ImageNet models

   Pretrained ImageNet model for Alex net can be downloaded to initialize the DeepBox network training.

   cd $FDBOX_ROOT
   ./data/scripts/fetch_imagenet_models.sh

   Alternatively, users can initialize the training with our multiscale sliding window model in

   $FDBOX_ROOT/output/default/coco_train2014/fast-dbox-slidwindow-multiscale.caffemodel

Usage

Train a Fast DeepBox proposer:

python ./tools/train_net.py (Show all training options)
python ./tools/train_net.py --gpu 0 (Train on GPU 0)

Train output is written underneath $FDBOX_ROOT/output/default/coco_train2014.

Test a Fast DeepBox proposer:

python ./tools/test_net.py (Show all testing options)
python ./tools/test_net.py --gpu 0 (Test on GPU 0)
python ./tools/test_net.py --imdb coco_test-dev2015 (Test on Test-dev split)  

Test output is written underneath $FDBOX_ROOT/output/default/coco_val2014 by default. If you test on test-dev, it will be underneath coco_test-dev2015 instead.

Full Demo a pre-trained Fast DeepBox proposer on COCO val set

python ./tools/demo.py (Show all demo options)
python ./tools/demo.py --demo 0 --frame 0 --numboxes 5 (Demo on frame 0 of COCO val set showing top 5 proposals)

Evaluate the test result. This part is written in Matlab. First launch your Matlab in $FDBOX_ROOT. Then do the following:

cd ./eval
eval_fast_dbox
plot_fast_dbox(name)

The evaluation output will be printed and the Recall-vs-NumOfProposals plots will be shown.

Extra-downloads

Pre-computed Fast DeepBox proposals on COCO train, val, and test-dev.

cd $FDBOX_ROOT/data
./scripts/fetch_fast_dbox_models.sh

deepbox's Issues

Hi ,I hope you can help me solve a problem,thx

Hi,
I find you do the full object detection of fast Deepbox +fast-rcnn in section 4.8 of your paper, but I don't know how to do to make deepbox as a method of get proposal to finish object detection in fast-rcnn， also ,I don't find the issue in READ ME . I am starting DL no longer,I hope you can help me ,thx！

Training cannot converge when use 5 conv layers

I'm trying to train a model with 5 conv layers from AlexNet model. And I followed src/fast_dbox_config.py, using a yaml file to setting DEDUP_BOXES=0.0625(1/16). Other settings are using default config. Using pretrained model imagenet_models/CaffeNet.v2.caffemodel for 5 conv layers.
But the the loss cannot converge.

Here is my train prototxt

name: "CaffeNet"
#COCO dataset needs to change the bbox regression layer
input: "data"
input_shape {
dim: 1
dim: 3
dim: 140
dim: 140
}
input: "rois"
input_shape {
dim: 1 # to be changed on-the-fly to num ROIs
dim: 5 # [batch ind, x1, y1, x2, y2] zero-based indexing
dim: 1
dim: 1
}
input: "labels"
input_shape {
dim: 1 # to be changed on-the-fly to match num ROIs
dim: 1
dim: 1
dim: 1
}
layer {
name: "conv1"
type: "Convolution"
bottom: "data"
top: "conv1"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 96
kernel_size: 11
stride: 4
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
value: 0
}
}
}
layer {
name: "relu1"
type: "ReLU"
bottom: "conv1"
top: "conv1"
}
layer {
name: "norm1"
type: "LRN"
bottom: "conv1"
top: "norm1"
lrn_param {
local_size: 5
alpha: 0.0001
beta: 0.75
}
}
layer {
name: "pool1"
type: "Pooling"
bottom: "norm1"
top: "pool1"
pooling_param {
pool: MAX
kernel_size: 3
stride: 2
}
}
layer {
name: "conv2"
type: "Convolution"
bottom: "pool1"
top: "conv2"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 256
pad: 2
kernel_size: 5
group: 2
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
value: 0.1
}
}
}
layer {
name: "relu2"
type: "ReLU"
bottom: "conv2"
top: "conv2"
}
layer {
name: "norm2"
type: "LRN"
bottom: "conv2"
top: "norm2"
lrn_param {
local_size: 5
alpha: 0.0001
beta: 0.75
}
}
layer {
name: "pool2"
type: "Pooling"
bottom: "norm2"
top: "pool2"
pooling_param {
pool: MAX
kernel_size: 3
stride: 2
}
}
layer {
name: "conv3"
type: "Convolution"
bottom: "pool2"
top: "conv3"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 384
pad: 1
kernel_size: 3
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
value: 0
}
}
}
layer {
name: "relu3"
type: "ReLU"
bottom: "conv3"
top: "conv3"
}
layer {
name: "conv4"
type: "Convolution"
bottom: "conv3"
top: "conv4"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 384
pad: 1
kernel_size: 3
group: 2
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
value: 0.1
}
}
}
layer {
name: "relu4"
type: "ReLU"
bottom: "conv4"
top: "conv4"
}
layer {
name: "conv5"
type: "Convolution"
bottom: "conv4"
top: "conv5"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 256
pad: 1
kernel_size: 3
group: 2
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
value: 0.1
}
}
}
layer {
name: "relu5"
type: "ReLU"
bottom: "conv5"
top: "conv5"
}
layer {
name: "roi_pool2_0"
type: "ROIPooling"
bottom: "conv5"
bottom: "rois"
top: "pool2_0"
roi_pooling_param {
pooled_w: 6
pooled_h: 6
spatial_scale: 0.125 # 1/16
}
}
layer {
name: "fc3"
type: "InnerProduct"
bottom: "pool2_0"
top: "fc3"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
inner_product_param {
num_output: 1024
}
}
layer {
name: "relu3_0"
type: "ReLU"
bottom: "fc3"
top: "fc3"
}
layer {
name: "drop3"
type: "Dropout"
bottom: "fc3"
top: "fc3"
dropout_param {
dropout_ratio: 0.5
}
}
layer {
name: "cls_score"
type: "InnerProduct"
bottom: "fc3"
top: "cls_score"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
inner_product_param {
num_output: 2
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
value: 0
}
}
}
layer {
name: "loss_cls"
type: "SoftmaxWithLoss"
bottom: "cls_score"
bottom: "labels"
top: "loss_cls"
loss_weight: 1
}

Couldn't able to run the basic demo.

These were the last lines before termination.

---

Process image 0,5864 boxes
F0122 10:44:21.139531 31764 roi_pooling_layer.cpp:70] Check failed: roi_end_w < width_ (150 vs. 149)
*** Check failure stack trace: ***
Aborted (core dumped)

---

Command I am using for execution : python ./tools/demo.py --demo 1
Need suggestions to resolve this issue.

Cannot download edge_box_data.tgz

Hi,

I have a trouble in downloading edge_box_data.tgz.
./scripts/fetch_edge_box_data.sh does not work.
Also,
wget ftp://ftp.cs.berkeley.edu/pub/projects/vision/edge_box_data.tgz
does not work either.
Is there anybody who succeeded in downloading this tarball?

Thank you

caffe-fast-rcnn missing folders and files -> build errors

Hi,

The caffe-fast-rcnn is missing the 'matlab' and 'examples' folder. Also, 'format.hpp' is missing from 'caffe-fast-rcnn/src/caffe/util'. I'm building on Ubuntu 16.04 with a NVIDIA GTX 1080.

It would be great if this could get fixed so that others can more easily install DeepBox. With these changes, I was able to build caffe-fast-rcnn.

Thomas

How to download pre-computed deepbox proposals?

Hi,

It seems we cannot download pre-computed Fast DeepBox proposals on COCO train, val, and test-dev using the following commands:

cd $FDBOX_ROOT/data
./scripts/fetch_fast_dbox_models.sh

We can only download coco_train2014.tgz by using script from ./output/default.

Please have an update about it.

Thank you.

model trained on VOC 2007

In your paper, you say that you evaluated DeepBox on both PASCAL VOC 2007 and COCO 2014 datasets. Would you please also make the model you trained on VOC 2007 trainval set publicly available?

This project can't be got clone now?

Getting results Worse than even Edgebox Proposals

Hi,
I tried your Deepbox implementation with Edgebox proposals and after running it on several images and manually comparing the results of Deepbox to Edgebox, i found that it performed even worse than Edgebox alone. Can you help me where i should improve to get the better result out of it ? What I noticed that scores to those regions are much higher which don't contain objects.

Below you can find image and files containing scores and boxes for Edgebox as well as Deepbox.
example.zip

Cannot download pre-computed Fast DeepBox models

The link to download the model, 'http://www.cs.berkeley.edu/~wckuo/fast-dbox-data/coco_train2014.tgz' is not accessible and gives a moved permanently error. It would be great if you could share the trained models. Thank you.

How to work with my own set of images as input?

I am playing with the ipython notebook. It's picking random image from COCO collection and showing annotations for that particular image. How could I use my own image?