alexgkendall/caffe-segnet

Implementation of SegNet: A Deep Convolutional Encoder-Decoder Architecture for Semantic Pixel-Wise Labelling

Caffe SegNet

This is a modified version of Caffe which supports the SegNet architecture

As described in SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation Vijay Badrinarayanan, Alex Kendall and Roberto Cipolla, PAMI 2017 [http://arxiv.org/abs/1511.00561]

Updated Version:

This version supports cudnn v2 acceleration. @TimoSaemann has a branch supporting a more recent version of Caffe (Dec 2016) with cudnn v5.1: https://github.com/TimoSaemann/caffe-segnet-cudnn5

Getting Started with Example Model and Webcam Demo

If you would just like to try out a pretrained example model, then you can find the model used in the SegNet webdemo and a script to run a live webcam demo here: https://github.com/alexgkendall/SegNet-Tutorial

For a more detailed introduction to this software please see the tutorial here: http://mi.eng.cam.ac.uk/projects/segnet/tutorial.html

Dataset

Prepare a text file of space-separated paths to images (jpegs or pngs) and corresponding label images alternatively e.g. /path/to/im1.png /another/path/to/lab1.png /path/to/im2.png /path/lab2.png ...

Label images must be single channel, with each value from 0 being a separate class. The example net uses an image size of 360 by 480.

Net specification

Example net specification and solver prototext files are given in examples/segnet. To train a model, alter the data path in the data layers in net.prototxt to be your dataset.txt file (as described above).

In the last convolution layer, change num_output to be the number of classes in your dataset.

Training

In solver.prototxt set a path for snapshot_prefix. Then in a terminal run ./build/tools/caffe train -solver ./examples/segnet/solver.prototxt

Publications

If you use this software in your research, please cite our publications:

http://arxiv.org/abs/1511.02680 Alex Kendall, Vijay Badrinarayanan and Roberto Cipolla "Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding." arXiv preprint arXiv:1511.02680, 2015.

http://arxiv.org/abs/1511.00561 Vijay Badrinarayanan, Alex Kendall and Roberto Cipolla "SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation." PAMI, 2017.

License

This extension to the Caffe library is released under a creative commons license which allows for personal and research use only. For a commercial license please contact the authors. You can view a license summary here: http://creativecommons.org/licenses/by-nc/4.0/

caffe-segnet's Issues

Compilation problem

Hello! I receive an error when compiling on ArchLinux. Original caffe from here compiles without problems with same arg for cmake.
My log:

[nikitas@localhost build]$ cmake -DBLAS=open -DCPU_ONLY=ON  ..
-- Boost version: 1.61.0
-- Found the following Boost libraries:
--   system
--   thread
--   chrono
--   date_time
--   atomic
-- Found gflags  (include: /usr/include, library: /usr/lib64/libgflags.so)
-- Found glog    (include: /usr/include, library: /usr/lib64/libglog.so)
-- Found PROTOBUF Compiler: /usr/bin/protoc
-- Found lmdb    (include: /usr/include, library: /usr/lib64/liblmdb.so)
-- Found LevelDB (include: /usr/include, library: /usr/lib64/libleveldb.so)
-- Found Snappy  (include: /usr/include, library: /usr/lib64/libsnappy.so)
-- CUDA is disabled. Building without it...
-- Found CUDA: /opt/cuda (found suitable exact version "7.5")
-- OpenCV found (/usr/local/share/OpenCV)
-- Found OpenBLAS libraries: /usr/lib64/libopenblas.so
-- Found OpenBLAS include: /usr/include
-- Found PythonInterp: /usr/bin/python2.7 (found suitable version "2.7.11", minimum required is "2.7")
-- Found PythonLibs: /usr/lib64/libpython2.7.so (found suitable version "2.7.11", minimum required is "2.7")
-- Found NumPy: /usr/lib/python2.7/site-packages/numpy/core/include (found suitable version "1.11.0", minimum required is "1.7.1")
-- NumPy ver. 1.11.0 found (include: /usr/lib/python2.7/site-packages/numpy/core/include)
-- Boost version: 1.61.0
-- Found the following Boost libraries:
--   python
-- Found Doxygen: /usr/bin/doxygen (found version "1.8.11")
-- Detected Doxygen OUTPUT_DIRECTORY: ./doxygen/
-- Found Git: /usr/bin/git (found version "2.8.2")
--
-- ******************* Caffe Configuration Summary *******************
-- General:
--   Version           :   <TODO> (Caffe doesn't declare its version in headers)
--   Git               :   rc2-334-g6731fbb
--   System            :   Linux
--   C++ compiler      :   /usr/bin/c++
--   Release CXX flags :   -O3 -DNDEBUG -fPIC -Wall -Wno-sign-compare -Wno-uninitialized
--   Debug CXX flags   :   -g -fPIC -Wall -Wno-sign-compare -Wno-uninitialized
--   Build type        :   Release
--
--   BUILD_SHARED_LIBS :   ON
--   BUILD_python      :   ON
--   BUILD_matlab      :   OFF
--   BUILD_docs        :   ON
--   CPU_ONLY          :   ON
--
-- Dependencies:
--   BLAS              :   Yes (open)
--   Boost             :   Yes (ver. 1.61)
--   glog              :   Yes
--   gflags            :   Yes
--   protobuf          :   Yes (ver. 2.6.1)
--   lmdb              :   Yes (ver. 0.9.18)
--   Snappy            :   Yes (ver. 1.1.3)
--   LevelDB           :   Yes (ver. 1.18)
--   OpenCV            :   Yes (ver. 3.1.0)
--   CUDA              :   No
--
-- Python:
--   Interpreter       :   /usr/bin/python2.7 (ver. 2.7.11)
--   Libraries         :   /usr/lib64/libpython2.7.so (ver 2.7.11)
--   NumPy             :   /usr/lib/python2.7/site-packages/numpy/core/include (ver 1.11.0)
--
-- Documentaion:
--   Doxygen           :   /usr/bin/doxygen (1.8.11)
--   config_file       :   /home/nikitas/caffe-segnet/.Doxyfile
--
-- Install:
--   Install path      :   /home/nikitas/caffe-segnet/build/install
--
-- Configuring done
-- Generating done
-- Build files have been written to: /home/nikitas/caffe-segnet/build
[nikitas@localhost build]$ make
[  0%] Running C++/Python protocol buffer compiler on /home/nikitas/caffe-segnet/src/caffe/proto/caffe.proto
Scanning dependencies of target proto
[  1%] Building CXX object src/caffe/CMakeFiles/proto.dir/__/__/include/caffe/proto/caffe.pb.cc.o
[  3%] Linking CXX static library ../../lib/libproto.a
[  3%] Built target proto
Scanning dependencies of target caffe
[  3%] Building CXX object src/caffe/CMakeFiles/caffe.dir/layers/dense_image_data_layer.cpp.o
/home/nikitas/caffe-segnet/src/caffe/layers/dense_image_data_layer.cpp: In member function 'virtual void caffe::DenseImageDataLayer<Dtype>::InternalThreadEntry()':
/home/nikitas/caffe-segnet/src/caffe/layers/dense_image_data_layer.cpp:131:3: error: 'DenseImageDataParameter' was not declared in this scope
   DenseImageDataParameter dense_image_data_param = this->layer_param_.dense_image_data_param();
   ^~~~~~~~~~~~~~~~~~~~~~~
/home/nikitas/caffe-segnet/src/caffe/layers/dense_image_data_layer.cpp:132:26: error: 'dense_image_data_param' was not declared in this scope
   const int batch_size = dense_image_data_param.batch_size();
                          ^~~~~~~~~~~~~~~~~~~~~~
/home/nikitas/caffe-segnet/src/caffe/layers/dense_image_data_layer.cpp: In instantiation of 'void caffe::DenseImageDataLayer<Dtype>::DataLayerSetUp(const std::vector<caffe::Blob<Dtype>*>&, const std::vector<caffe::Blob<Dtype>*>&) [with Dtype = float]':
/home/nikitas/caffe-segnet/src/caffe/layers/dense_image_data_layer.cpp:222:1:   required from here
/home/nikitas/caffe-segnet/src/caffe/layers/dense_image_data_layer.cpp:26:45: error: 'class caffe::LayerParameter' has no member named 'dense_image_data_param'; did you mean 'has_image_data_param'?
   const int new_height = this->layer_param_.dense_image_data_param().new_height();
                          ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
/home/nikitas/caffe-segnet/src/caffe/layers/dense_image_data_layer.cpp:27:45: error: 'class caffe::LayerParameter' has no member named 'dense_image_data_param'; did you mean 'has_image_data_param'?
   const int new_width  = this->layer_param_.dense_image_data_param().new_width();
                          ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
/home/nikitas/caffe-segnet/src/caffe/layers/dense_image_data_layer.cpp:28:46: error: 'class caffe::LayerParameter' has no member named 'dense_image_data_param'; did you mean 'has_image_data_param'?
   const int crop_height = this->layer_param_.dense_image_data_param().crop_height();
                           ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
/home/nikitas/caffe-segnet/src/caffe/layers/dense_image_data_layer.cpp:29:46: error: 'class caffe::LayerParameter' has no member named 'dense_image_data_param'; did you mean 'has_image_data_param'?
   const int crop_width  = this->layer_param_.dense_image_data_param().crop_width();
                           ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
/home/nikitas/caffe-segnet/src/caffe/layers/dense_image_data_layer.cpp:30:45: error: 'class caffe::LayerParameter' has no member named 'dense_image_data_param'; did you mean 'has_image_data_param'?
   const bool is_color  = this->layer_param_.dense_image_data_param().is_color();
                          ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
/home/nikitas/caffe-segnet/src/caffe/layers/dense_image_data_layer.cpp:31:43: error: 'class caffe::LayerParameter' has no member named 'dense_image_data_param'; did you mean 'has_image_data_param'?
   string root_folder = this->layer_param_.dense_image_data_param().root_folder();
                        ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
/home/nikitas/caffe-segnet/src/caffe/layers/dense_image_data_layer.cpp:40:45: error: 'class caffe::LayerParameter' has no member named 'dense_image_data_param'; did you mean 'has_image_data_param'?
   const string& source = this->layer_param_.dense_image_data_param().source();
                          ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
/home/nikitas/caffe-segnet/src/caffe/layers/dense_image_data_layer.cpp:49:26: error: 'class caffe::LayerParameter' has no member named 'dense_image_data_param'; did you mean 'has_image_data_param'?
   if (this->layer_param_.dense_image_data_param().shuffle()) {
       ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
/home/nikitas/caffe-segnet/src/caffe/layers/dense_image_data_layer.cpp:60:26: error: 'class caffe::LayerParameter' has no member named 'dense_image_data_param'; did you mean 'has_image_data_param'?
   if (this->layer_param_.dense_image_data_param().rand_skip()) {
       ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
/home/nikitas/caffe-segnet/src/caffe/layers/dense_image_data_layer.cpp:62:28: error: 'class caffe::LayerParameter' has no member named 'dense_image_data_param'; did you mean 'has_image_data_param'?
         this->layer_param_.dense_image_data_param().rand_skip();
         ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
/home/nikitas/caffe-segnet/src/caffe/layers/dense_image_data_layer.cpp:83:45: error: 'class caffe::LayerParameter' has no member named 'dense_image_data_param'; did you mean 'has_image_data_param'?
   const int batch_size = this->layer_param_.dense_image_data_param().batch_size();
                          ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
/home/nikitas/caffe-segnet/src/caffe/layers/dense_image_data_layer.cpp: In instantiation of 'void caffe::DenseImageDataLayer<Dtype>::InternalThreadEntry() [with Dtype = float]':
/home/nikitas/caffe-segnet/src/caffe/layers/dense_image_data_layer.cpp:222:1:   required from here
/home/nikitas/caffe-segnet/src/caffe/layers/dense_image_data_layer.cpp:169:28: error: 'class caffe::LayerParameter' has no member named 'dense_image_data_param'; did you mean 'has_image_data_param'?
     if (this->layer_param_.dense_image_data_param().mirror()) {
         ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
/home/nikitas/caffe-segnet/src/caffe/layers/dense_image_data_layer.cpp:211:30: error: 'class caffe::LayerParameter' has no member named 'dense_image_data_param'; did you mean 'has_image_data_param'?
       if (this->layer_param_.dense_image_data_param().shuffle()) {
           ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
/home/nikitas/caffe-segnet/src/caffe/layers/dense_image_data_layer.cpp: In instantiation of 'void caffe::DenseImageDataLayer<Dtype>::DataLayerSetUp(const std::vector<caffe::Blob<Dtype>*>&, const std::vector<caffe::Blob<Dtype>*>&) [with Dtype = double]':
/home/nikitas/caffe-segnet/src/caffe/layers/dense_image_data_layer.cpp:222:1:   required from here
/home/nikitas/caffe-segnet/src/caffe/layers/dense_image_data_layer.cpp:26:45: error: 'class caffe::LayerParameter' has no member named 'dense_image_data_param'; did you mean 'has_image_data_param'?
   const int new_height = this->layer_param_.dense_image_data_param().new_height();
                          ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
/home/nikitas/caffe-segnet/src/caffe/layers/dense_image_data_layer.cpp:27:45: error: 'class caffe::LayerParameter' has no member named 'dense_image_data_param'; did you mean 'has_image_data_param'?
   const int new_width  = this->layer_param_.dense_image_data_param().new_width();
                          ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
/home/nikitas/caffe-segnet/src/caffe/layers/dense_image_data_layer.cpp:28:46: error: 'class caffe::LayerParameter' has no member named 'dense_image_data_param'; did you mean 'has_image_data_param'?
   const int crop_height = this->layer_param_.dense_image_data_param().crop_height();
                           ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
/home/nikitas/caffe-segnet/src/caffe/layers/dense_image_data_layer.cpp:29:46: error: 'class caffe::LayerParameter' has no member named 'dense_image_data_param'; did you mean 'has_image_data_param'?
   const int crop_width  = this->layer_param_.dense_image_data_param().crop_width();
                           ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
/home/nikitas/caffe-segnet/src/caffe/layers/dense_image_data_layer.cpp:30:45: error: 'class caffe::LayerParameter' has no member named 'dense_image_data_param'; did you mean 'has_image_data_param'?
   const bool is_color  = this->layer_param_.dense_image_data_param().is_color();
                          ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
/home/nikitas/caffe-segnet/src/caffe/layers/dense_image_data_layer.cpp:31:43: error: 'class caffe::LayerParameter' has no member named 'dense_image_data_param'; did you mean 'has_image_data_param'?
   string root_folder = this->layer_param_.dense_image_data_param().root_folder();
                        ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
/home/nikitas/caffe-segnet/src/caffe/layers/dense_image_data_layer.cpp:40:45: error: 'class caffe::LayerParameter' has no member named 'dense_image_data_param'; did you mean 'has_image_data_param'?
   const string& source = this->layer_param_.dense_image_data_param().source();
                          ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
/home/nikitas/caffe-segnet/src/caffe/layers/dense_image_data_layer.cpp:49:26: error: 'class caffe::LayerParameter' has no member named 'dense_image_data_param'; did you mean 'has_image_data_param'?
   if (this->layer_param_.dense_image_data_param().shuffle()) {
       ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
/home/nikitas/caffe-segnet/src/caffe/layers/dense_image_data_layer.cpp:60:26: error: 'class caffe::LayerParameter' has no member named 'dense_image_data_param'; did you mean 'has_image_data_param'?
   if (this->layer_param_.dense_image_data_param().rand_skip()) {
       ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
/home/nikitas/caffe-segnet/src/caffe/layers/dense_image_data_layer.cpp:62:28: error: 'class caffe::LayerParameter' has no member named 'dense_image_data_param'; did you mean 'has_image_data_param'?
         this->layer_param_.dense_image_data_param().rand_skip();
         ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
/home/nikitas/caffe-segnet/src/caffe/layers/dense_image_data_layer.cpp:83:45: error: 'class caffe::LayerParameter' has no member named 'dense_image_data_param'; did you mean 'has_image_data_param'?
   const int batch_size = this->layer_param_.dense_image_data_param().batch_size();
                          ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
/home/nikitas/caffe-segnet/src/caffe/layers/dense_image_data_layer.cpp: In instantiation of 'void caffe::DenseImageDataLayer<Dtype>::InternalThreadEntry() [with Dtype = double]':
/home/nikitas/caffe-segnet/src/caffe/layers/dense_image_data_layer.cpp:222:1:   required from here
/home/nikitas/caffe-segnet/src/caffe/layers/dense_image_data_layer.cpp:169:28: error: 'class caffe::LayerParameter' has no member named 'dense_image_data_param'; did you mean 'has_image_data_param'?
     if (this->layer_param_.dense_image_data_param().mirror()) {
         ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
/home/nikitas/caffe-segnet/src/caffe/layers/dense_image_data_layer.cpp:211:30: error: 'class caffe::LayerParameter' has no member named 'dense_image_data_param'; did you mean 'has_image_data_param'?
       if (this->layer_param_.dense_image_data_param().shuffle()) {
           ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
src/caffe/CMakeFiles/caffe.dir/build.make:62: recipe for target 'src/caffe/CMakeFiles/caffe.dir/layers/dense_image_data_layer.cpp.o' failed
make[2]: *** [src/caffe/CMakeFiles/caffe.dir/layers/dense_image_data_layer.cpp.o] Error 1
CMakeFiles/Makefile2:240: recipe for target 'src/caffe/CMakeFiles/caffe.dir/all' failed
make[1]: *** [src/caffe/CMakeFiles/caffe.dir/all] Error 2
Makefile:127: recipe for target 'all' failed
make: *** [all] Error 2

making class weight for custom DB

I am establishing the process for making class weight.
First, I had checked default DB(CamVid) and default weights in segnet_train.prototxt
As I counted law data pixels of default DB, below data table was shown.

class, pixels, files
0 10682767 366
1 14750079 365
2 623349 366
3 20076880 367
4 2845085 349
5 6166762 319
6 743859 351
7 714595 173
8 3719877 360
9 405385 317
10 184967 201
11 2503995 367

and following your paper, calculated weight. left is default weight, right is my result.
f(class) = frequency(class) / (image_count(class) * 480*360)
weight(class) = median of f(class)) / f(class)

0 0.2595 0.256527308
1 0.1826 0.185282667
2 4.564 4.396287575
3 0.1417 0.136869322
4 0.9051 0.918473131
5 0.3826 0.387319864
6 9.6446 3.533074291
7 1.8418 1.812685267
8 0.6823 0.724619798
9 6.2478 5.855012981
10 7.3614 8.136508447
11 1.097409921

almost weight values seems similar between them
but only class #6 has big different. Could you confirm my raw data?

Also, I have another question.
If I want to ignore class#5, how can I do it? your reference ignores #11.
so it seems that you described only 0-10 on prototxt file.
but if middle position class should be ignored how can we describe weights?
and multi classes can be ignored? If it is possible, can you advice the way?
BR

Problem with Bayesian SegNet Execution

I have successfully created models for SegNet and run the tests and they work properly.

I am currently working on running the Bayesian SegNet models: however, I am unable to due to
errors that occur at runtime.

I have models created for the bayesian cases, then I try to execute /SegNet/Scripts/test_bayesian_segnet.py I receive the following error.

ExceptionTraceback (most recent call last)
in ()
12 input_image = np.repeat(input_image, input_shape[0], axis=0)
13
---> 14 out = net.forward_all(data=input_image)

Exception: Input blob arguments do not match net inputs.

Are there any solutions based on this error that anyone can suggest to get this running correctly?

Thanks.

Encountered a compiling problem

I met the following problem while compiling, but didn't get much useful information after google it. Can anyone tell me what might be the source of these errors?

/home/ted/SegNet/caffe-segnet/src/caffe/layers/cudnn_conv_layer.cu(81): error: argument of type "cudnnAddMode_t" is incompatible with parameter of type "const void *"
          detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Forward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=float]" 
(157): here

/home/ted/SegNet/caffe-segnet/src/caffe/layers/cudnn_conv_layer.cu(81): error: argument of type "const void *" is incompatible with parameter of type "cudnnTensorDescriptor_t"
          detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Forward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=float]" 
(157): here

/home/ted/SegNet/caffe-segnet/src/caffe/layers/cudnn_conv_layer.cu(81): error: argument of type "const void *" is incompatible with parameter of type "cudnnTensorDescriptor_t"
          detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Forward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=float]" 
(157): here

/home/ted/SegNet/caffe-segnet/src/caffe/layers/cudnn_conv_layer.cu(81): error: too many arguments in function call
          detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Forward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=float]" 
(157): here

/home/ted/SegNet/caffe-segnet/src/caffe/layers/cudnn_conv_layer.cu(125): error: argument of type "const void *" is incompatible with parameter of type "cudnnConvolutionBwdFilterAlgo_t"
          detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Backward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<__nv_bool, std::allocator<__nv_bool>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=float]" 
(157): here

/home/ted/SegNet/caffe-segnet/src/caffe/layers/cudnn_conv_layer.cu(125): error: argument of type "float *" is incompatible with parameter of type "size_t"
          detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Backward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<__nv_bool, std::allocator<__nv_bool>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=float]" 
(157): here

/home/ted/SegNet/caffe-segnet/src/caffe/layers/cudnn_conv_layer.cu(125): error: too few arguments in function call
          detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Backward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<__nv_bool, std::allocator<__nv_bool>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=float]" 
(157): here

/home/ted/SegNet/caffe-segnet/src/caffe/layers/cudnn_conv_layer.cu(140): error: argument of type "const void *" is incompatible with parameter of type "cudnnConvolutionBwdDataAlgo_t"
          detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Backward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<__nv_bool, std::allocator<__nv_bool>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=float]" 
(157): here

/home/ted/SegNet/caffe-segnet/src/caffe/layers/cudnn_conv_layer.cu(140): error: argument of type "float *" is incompatible with parameter of type "size_t"
          detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Backward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<__nv_bool, std::allocator<__nv_bool>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=float]" 
(157): here

/home/ted/SegNet/caffe-segnet/src/caffe/layers/cudnn_conv_layer.cu(140): error: too few arguments in function call
          detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Backward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<__nv_bool, std::allocator<__nv_bool>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=float]" 
(157): here

/home/ted/SegNet/caffe-segnet/src/caffe/layers/cudnn_conv_layer.cu(81): error: argument of type "cudnnAddMode_t" is incompatible with parameter of type "const void *"
          detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Forward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=double]" 
(157): here

/home/ted/SegNet/caffe-segnet/src/caffe/layers/cudnn_conv_layer.cu(81): error: argument of type "const void *" is incompatible with parameter of type "cudnnTensorDescriptor_t"
          detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Forward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=double]" 
(157): here

/home/ted/SegNet/caffe-segnet/src/caffe/layers/cudnn_conv_layer.cu(81): error: argument of type "const void *" is incompatible with parameter of type "cudnnTensorDescriptor_t"
          detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Forward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=double]" 
(157): here

/home/ted/SegNet/caffe-segnet/src/caffe/layers/cudnn_conv_layer.cu(81): error: too many arguments in function call
          detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Forward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=double]" 
(157): here

/home/ted/SegNet/caffe-segnet/src/caffe/layers/cudnn_conv_layer.cu(125): error: argument of type "const void *" is incompatible with parameter of type "cudnnConvolutionBwdFilterAlgo_t"
          detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Backward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<__nv_bool, std::allocator<__nv_bool>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=double]" 
(157): here

/home/ted/SegNet/caffe-segnet/src/caffe/layers/cudnn_conv_layer.cu(125): error: argument of type "double *" is incompatible with parameter of type "size_t"
          detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Backward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<__nv_bool, std::allocator<__nv_bool>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=double]" 
(157): here

/home/ted/SegNet/caffe-segnet/src/caffe/layers/cudnn_conv_layer.cu(125): error: too few arguments in function call
          detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Backward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<__nv_bool, std::allocator<__nv_bool>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=double]" 
(157): here

/home/ted/SegNet/caffe-segnet/src/caffe/layers/cudnn_conv_layer.cu(140): error: argument of type "const void *" is incompatible with parameter of type "cudnnConvolutionBwdDataAlgo_t"
          detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Backward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<__nv_bool, std::allocator<__nv_bool>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=double]" 
(157): here

/home/ted/SegNet/caffe-segnet/src/caffe/layers/cudnn_conv_layer.cu(140): error: argument of type "double *" is incompatible with parameter of type "size_t"
          detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Backward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<__nv_bool, std::allocator<__nv_bool>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=double]" 
(157): here

/home/ted/SegNet/caffe-segnet/src/caffe/layers/cudnn_conv_layer.cu(140): error: too few arguments in function call
          detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Backward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<__nv_bool, std::allocator<__nv_bool>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=double]" 
(157): here

20 errors detected in the compilation of "/tmp/tmpxft_0000489a_00000000-7_cudnn_conv_layer.cpp1.ii".
CMake Error at cuda_compile_generated_cudnn_conv_layer.cu.o.cmake:264 (message):
  Error generating file
  /home/ted/SegNet/caffe-segnet/build/src/caffe/CMakeFiles/cuda_compile.dir/layers/./cuda_compile_generated_cudnn_conv_layer.cu.o


make[2]: *** [src/caffe/CMakeFiles/cuda_compile.dir/layers/./cuda_compile_generated_cudnn_conv_layer.cu.o] Error 1
make[1]: *** [src/caffe/CMakeFiles/caffe.dir/all] Error 2
make: *** [all] Error 2

The Bayesian SegNet Result for Camvid Database

Hey @alexgkendall ,when I use your code to calculate the global acc. ,class avg. and the mean IoU of the Bayesian SegNet for Camvid database ,I get the result that 0.85 for the global acc. ,0.70 for the class avg. and 0.52 for the mean IoU, which is different from your paper's result. Can u give me some advice?

On custom data, loss and weights diverge to nan -- faster with lower LR

Thanks very much for sharing your work on SegNet!

I'm interested in using the segnet model with some custom data (aerial imagery; 1000 images and 3 label classes for now), and I'm seeing something strange in training:

Training with base_lr: 0.001, I was seeing training loss = nan. Surprisingly, when I reduced the LR, the loss exploded to nan even faster. Couple of examples:

With base_lr = 1e-4

#Iters Seconds TrainingLoss LearningRate
0    3.022133     1.25783   0.0001
20   58.911155    1.3897    0.0001
40   114.768299   1.18601   0.0001
60   170.620245   1.02438   0.0001
80   226.502911   0.992765  0.0001
100  282.332283   1.05062   0.0001
120  338.179408   1.44935   0.0001
140  394.035362   4.73935   0.0001
160  449.878457   0.997907  0.0001
180  505.701507   0.98053   0.0001
200  561.558644   1.14603   0.0001
220  617.398724   0.936918  0.0001
240  672.909704   nan       0.0001
260  728.026581   nan       0.0001

With base_lr = 1e-6

#Iters Seconds TrainingLoss LearningRate
0   3.033944    1.4484   1e-06
20  58.966689   1.12416  1e-06
40  114.393818  nan      1e-06
60  169.507581  nan      1e-06
80  224.611446  nan      1e-06

I also set debug_info: true, and discovered that the actual weights seem to be diverging, also:

I0408 13:13:13.229012 23306 net.cpp:594]     [Forward] Layer data, top blob data data: 62.8498
I0408 13:13:13.229233 23306 net.cpp:594]     [Forward] Layer data, top blob label data: 2.95128
I0408 13:13:13.229307 23306 net.cpp:594]     [Forward] Layer label_data_1_split, top blob label_data_1_split_0 data: 2.95128
I0408 13:13:13.229379 23306 net.cpp:594]     [Forward] Layer label_data_1_split, top blob label_data_1_split_1 data: 2.95128
I0408 13:13:13.236066 23306 net.cpp:594]     [Forward] Layer conv1_1, top blob conv1_1 data: nan
I0408 13:13:13.236177 23306 net.cpp:604]     [Forward] Layer conv1_1, param blob 0 data: nan
I0408 13:13:13.236227 23306 net.cpp:604]     [Forward] Layer conv1_1, param blob 1 data: nan
I0408 13:13:13.258630 23306 net.cpp:594]     [Forward] Layer conv1_1_bn, top blob conv1_1 data: nan
I0408 13:13:13.258780 23306 net.cpp:604]     [Forward] Layer conv1_1_bn, param blob 0 data: nan
I0408 13:13:13.258839 23306 net.cpp:604]     [Forward] Layer conv1_1_bn, param blob 1 data: nan
I0408 13:13:13.261641 23306 net.cpp:594]     [Forward] Layer relu1_1, top blob conv1_1 data: 0
I0408 13:13:13.304590 23306 net.cpp:594]     [Forward] Layer conv1_2, top blob conv1_2 data: nan
...

Finally, if I set base_lr to 0, the loss diverges, but the weights do not.

I'm going to keep investigating, and will report back here if I resolve it, but I'm a bit stumped. @alexgkendall Do any insights come to mind?

make runtest fails

make all and make test are OK when compiling caffe-segnet.
When make runtest, error occurs:

[ RUN ] DenseImageDataLayerTest/1.TestRead
F0530 23:05:28.438096 8335 data_transformer.cpp:230] Check failed: height <= img_height (360 vs. 0)
*** Check failure stack trace: ***
@ 0x7f590b9e0d7d google::LogMessage::Fail()
@ 0x7f590b9e2c1d google::LogMessage::SendToLog()
@ 0x7f590b9e0998 google::LogMessage::Flush()
@ 0x7f590b9e347e google::LogMessageFatal::~LogMessageFatal()
@ 0x7f5909bf01fb caffe::DataTransformer<>::Transform()
@ 0x7f5909ccb7dc caffe::DenseImageDataLayer<>::InternalThreadEntry()
@ 0x7f5901e7909a (unknown)
@ 0x7f59093776aa start_thread
@ 0x7f59090aceed (unknown)
Makefile:468: recipe for target 'runtest' failed
make: *** [runtest] Aborted (core dumped)

Error also occurs when trainging:

I0530 23:10:23.171794 8530 net.cpp:90] Creating Layer conv3_2
I0530 23:10:23.171803 8530 net.cpp:410] conv3_2 <- conv3_1
I0530 23:10:23.171813 8530 net.cpp:368] conv3_2 -> conv3_2
I0530 23:10:23.171825 8530 net.cpp:120] Setting up conv3_2
@ 0x7f7d2e4b28eb caffe::DataTransformer<>::Transform()
@ 0x7f7d2e58a9bc caffe::DenseImageDataLayer<>::InternalThreadEntry()
@ 0x7f7d2633a09a (unknown)
@ 0x7f7d250766aa start_thread
@ 0x7f7d2d732eed (unknown)
Aborted (core dumped)

Thank you for your help!

Merging with current caffe build

Hey @alexgkendall do you have any advice on merging the segnet version of caffe with the current BVLC caffe build? I am trying to use some of their new features. Thanks!

error in "make runtest" for the current caffe-segnet

Hi, @alexgkendall ,
I had compiled the current caffe-segnet again.
"make all" and "make test" passed, but came across error in "make runtest"
The details can be viewed here:
https://goo.gl/vwW8GT

Although I can run "/SegNet/caffe-segnet/build/tools/caffe train -gpu 0 -solver /SegNet/Models/segnet_solver.prototxt" successfully.

Could this error be ignored or should be fixed?

Thanks~
Milton

opencv v3.1.0

you have the same issue as CRF-RNN: opencv v3 is not recognized. I solved the problem in the same way, by changing the Makefile:

torrvision/crfasrnn#51

This should be useful

Benchmarking with caffe time

Hi,

I wanted to know if it is possible to benchmark layer by layer time for SegNet.

I tried ./build/tools/caffe time -model ../Models/segnet_basic_solver_small.prototxt -gpu 0 -iterations 1

but i get an error
[libprotobuf ERROR google/protobuf/text_format.cc:245] Error parsing text-format caffe.NetParameter: 1:4: Message type "caffe.NetParameter" has no field named "net".

which according to the following post szagoruyko/loadcaffe#5
is there any way to get it sorted?
everything else works fine (training and testing) but I wanted to extract some performance metrics.

Model trained on additional public 3.5K dataset

Thanks for sharing this fantastic structure and code of SegNet. According to your tutorial, I already managed to reproduce your first two result (SegNet-basic and SegNet). However, I would like to see better performance of SegNet (which trained on additional public 3.5K dataset). Could you please share your trained model (.caffemodel file)? Or kindly share the entire 3.5K dataset (with annotation) so that I can train by myself.

Thanks,
Meng-Jiun Chiou

Use CamVid as lmdb format

Hi,

I have a problem when convert CamVid dataset to lmdb format (instead of using list of images like you did). I used the same model and only change the input layer to lmdb backend. The result when using lmdb format is always lower around 5% than using list of image.

Any idea how to narrow down the problem?

Many thanks!

k-channel image dataset

My dataset has 2 classes; with 1000 training images of (5,256,256) also corresponding ground truth data (1,256,256) which is a binary image either 0 or 1 to represent the 2 classes.
Can seg-net be used on k-channel dataset or is limited to RGB ?

Unknown CMake command "caffe_set_caffe_link"

I have this problem during configuration (user:build$ cmake ..)

Thanks for your help!!

-- Found Atlas: /usr/include
-- Found Atlas (include: /usr/include, library: /usr/lib/libatlas.so)
-- Found PythonInterp: /home/user/anaconda2/bin/python2.7 (found suitable version "2.7.11", minimum required is "2.7")
-- Found PythonLibs: /usr/lib/x86_64-linux-gnu/libpython2.7.so (found suitable version "2.7.6", minimum required is "2.7")
-- Found NumPy: /home/user/anaconda2/lib/python2.7/site-packages/numpy/core/include (found suitable version "1.10.4", minimum required is "1.7.1")
-- NumPy ver. 1.10.4 found (include: /home/user/anaconda2/lib/python2.7/site-packages/numpy/core/include)
-- Boost version: 1.54.0
-- Found the following Boost libraries:
-- python
-- Found Doxygen: /usr/bin/doxygen (found version "1.8.6")
CMake Error at CMakeLists.txt:50 (caffe_set_caffe_link):
Unknown CMake command "caffe_set_caffe_link".

About example/net.prototxt

Hi @alexgkendall ! Recently I am reading your segnet framework and I found something strange!

In the caffe_segnet/examples/segnet/net.prototxt, I found that the layer conv1's bottom blob is said to be norm, however, there is no norm blob! I also refer your SegNet-Tutorial. In the Models/segnet_basic_train.prototxt, there is a layer called norm which connect the blobs data and norm.

So is there something wrong or should we correct it ?

Is a 4GB GPU enough?

Hi, @alexgkendall, could I train your SegNet on a 4GB GPU?

Check failed: status == CUBLAS STATUS_SUCCESS (11 vs. 0) CUBLAS STATUS MAPPING_ERROR

Hello!
An example described here is excellent and easily trained http://mi.eng.cam.ac.uk/projects/segnet/tutorial.html
For my task requires segmentation of two classes: the background and subject. Man and background.
But I have a problem with the preparation of annotated images (labels), in its own database. As I understand the network sees only color in the gray scale. I create labels using Adobe photoshop so:
Uploading original JPEG image
Superimposed over the original, without a background image in PNG format. Thus creating selection mask.
The sacrificial layer with the image without background PNG image.
Encodes the image to indexed color (3 colors: void 0 0 0, person 192 128 128, transparency alpha channel). This is necessary in order to while painting flowers no new colors or blur.
I produce fill the selected area with mask the above colors: 192 128 228 subject, background 0 0 0)
Then convert the image to RGB PNG 8 bit / chanel mode.
If you submit the following annotated (labels) to the input image network, it displays the following:
F0606 19:20:16.349822 2152 math_functions.cu:123] Check failed: status == CUBLAS_STATUS_SUCCESS (11 vs. 0) CUBLAS_STATUS_MAPPING_ERROR.
Then I proceed as follows:
Skipping my images via this script, that should solve my problems:

!/usr/bin/env python

import os
import numpy as np
from itertools import izip
from argparse import ArgumentParser
from collections import OrderedDict
from skimage.io import ImageCollection, imsave
from skimage.transform import resize

camvid_colors = OrderedDict([
("Animal", np.array([64, 128, 64], dtype=np.uint8)),
("Archway", np.array([192, 0, 128], dtype=np.uint8)),
("Bicyclist", np.array([0, 128, 192], dtype=np.uint8)),
("Bridge", np.array([0, 128, 64], dtype=np.uint8)),
("Building", np.array([128, 0, 0], dtype=np.uint8)),
("Car", np.array([64, 0, 128], dtype=np.uint8)),
("CartLuggagePram", np.array([64, 0, 192], dtype=np.uint8)),
("Child", np.array([192, 128, 64], dtype=np.uint8)),
("Column_Pole", np.array([192, 192, 128], dtype=np.uint8)),
("Fence", np.array([64, 64, 128], dtype=np.uint8)),
("LaneMkgsDriv", np.array([128, 0, 192], dtype=np.uint8)),
("LaneMkgsNonDriv", np.array([192, 0, 64], dtype=np.uint8)),
("Misc_Text", np.array([128, 128, 64], dtype=np.uint8)),
("MotorcycleScooter", np.array([192, 0, 192], dtype=np.uint8)),
("OtherMoving", np.array([128, 64, 64], dtype=np.uint8)),
("ParkingBlock", np.array([64, 192, 128], dtype=np.uint8)),
("Pedestrian", np.array([64, 64, 0], dtype=np.uint8)),
("Road", np.array([128, 64, 128], dtype=np.uint8)),
("RoadShoulder", np.array([128, 128, 192], dtype=np.uint8)),
("Sidewalk", np.array([0, 0, 192], dtype=np.uint8)),
("SignSymbol", np.array([192, 128, 128], dtype=np.uint8)),
("Sky", np.array([128, 128, 128], dtype=np.uint8)),
("SUVPickupTruck", np.array([64, 128, 192], dtype=np.uint8)),
("TrafficCone", np.array([0, 0, 64], dtype=np.uint8)),
("TrafficLight", np.array([0, 64, 64], dtype=np.uint8)),
("Train", np.array([192, 64, 128], dtype=np.uint8)),
("Tree", np.array([128, 128, 0], dtype=np.uint8)),
("Truck_Bus", np.array([192, 128, 192], dtype=np.uint8)),
("Tunnel", np.array([64, 0, 64], dtype=np.uint8)),
("VegetationMisc", np.array([192, 192, 0], dtype=np.uint8)),
("Wall", np.array([64, 192, 0], dtype=np.uint8)),
("Void", np.array([0, 0, 0], dtype=np.uint8))
])

def convert_label_to_grayscale(im):
out = (np.ones(im.shape[:2]) * 255).astype(np.uint8)
for gray_val, (label, rgb) in enumerate(camvid_colors.items()):
match_pxls = np.where((im == np.asarray(rgb)).sum(-1) == 3)
out[match_pxls] = gray_val
assert (out != 255).all(), "rounding errors or missing classes in camvid_colors"
return out.astype(np.uint8)

def make_parser():
parser = ArgumentParser()
parser.add_argument(
'label_dir',
help="Directory containing all RGB camvid label images as PNGs"
)
parser.add_argument(
'out_dir',
help="""Directory to save grayscale label images.
Output images have same basename as inputs so be careful not to
overwrite original RGB labels""")
return parser

if name == 'main':
parser = make_parser()
args = parser.parse_args()
labs = ImageCollection(os.path.join(args.label_dir, "*"))
os.makedirs(args.out_dir)
for i, (inpath, im) in enumerate(izip(labs.files, labs)):
print i + 1, "of", len(labs)
# resize to caffe-segnet input size and preserve label values
resized_im = (resize(im, (360, 480), order=0) * 255).astype(np.uint8)
out = convert_label_to_grayscale(resized_im)
outpath = os.path.join(args.out_dir, os.path.basename(inpath))
imsave(outpath, out)
Then again I try to start training with the converted images. Again a message:
Check failed: status == CUBLAS_STATUS_SUCCESS (11 vs. 0).
Remarkable is that the marked image after the tool takes the network and trained them: https://github.com/kyamagu/js-segment-annotator
But that's not what I need. I do not want to mark up the image manually, I already have cut without PNG image background. Please help me to understand how to convert colors to segnet took them? Maybe hsv?
Here's a link to my data: https://github.com/Maxfashko/CamVid?files=1

strange results with four-channel images

First, thanks for sharing your code. I'm running into a strange issue that I'm hoping Alex or someone else might be able to offer some insight on.

I'm training a model on a dataset with 3 classes and 4 channels (RGB + near-infrared). For some reason, at inference time, the model is only labeling pixels as class one or two, but it is not labeling any pixels as class 3. If I remove the near infrared channel from the data and retrain, then the model behaves as expected, labeling pixels of all three classes. I'm training the model for at least 100 epochs and I've tried it both with and without median class balancing. The strange thing is, if I train on the four-channel data for a small number of epochs (about 10) then it does label pixels of all three class types (albeit the predictions are fairly noisy still); however, when trying one trained for 20 or more epochs, it is no longer classifying anything as class 3.

I have triple-checked the prototxt files and my script for encoding the four-channel images into lmdb, but I don't see any obvious issues. Any insight about what might be causing this and how to fix it? Thanks!

Using Custom Data

I have trained the SegNet network and have a resulting Caffe model.
I would like to now test it on some of my own data.

Is there a simple way to test the model using a custom data set (I am unable to locate in the code
where the images are being taken from for the testing and display in test_segmentation_camvid.py)

Thanks.

Accuracy layer uncertain per-class behavior

Accuracy layer don't handles the case when class is missed in GT sample. It results to NaN value for per-class accuracy value for this particular class since that total number of class presence in sample (mini-batch) is 0 (then - division by zero):
top[1]->mutable_cpu_data()[i] = accuracies[i] / nums[i];

Caffe already fixed it.

HowTo obtain segmentation output (pixel-wise labeling)

Hi, Alex,

After I train the model on dataset Camvid, and perform testing (label prediction) on a new image using the following command.

./caffe test -model examples/segnet/net.prototxt -weights examples/segnet/traintest_iter_200.caffemodel -gpu 0 -iterations 1

I could get the loss and accuracy output (details here: https://goo.gl/KIa5Vb ), so segmentation output (pixel-wise labeling prediction) for the test image must be computed in someplace.
My question is :
Where could I obtain that segmentation output (pixel-wise labeling)? Or which steps have to be appended?

Thanks in advance!

Milton

per_class_accuracy = -nan in many classes

Hi @alexgkendall

Thanks to your guide! I'm able to train my data and can run the test script to visualize the prediction results. However, there are 2 problems I hope you can give your expert opinion:

1. When do the training, many classes show "per_class_accuracy = -nan", I guess it's normal since my data usually have only 2 or 3 classes in each input image. But does this problem affect the prediction result of our net?
2. Do we have a script/way to summarize the per_class_accuracy when testing? By now test_segmentation_camvid.py file only shows the input/ground truth/prediction images but no statistic over test data available.

Thanks so much!

I1126 12:45:14.784065 13474 solver.cpp:486] Iteration 100, lr = 0.1
I1126 12:45:20.671650 13474 solver.cpp:214] Iteration 120, loss = 0.0285711
I1126 12:45:20.671739 13474 solver.cpp:229] Train net output #0: accuracy = 0.993626
I1126 12:45:20.671751 13474 solver.cpp:229] Train net output #1: loss = 0.0285711 (* 1 = 0.0285711 loss)
I1126 12:45:20.671769 13474 solver.cpp:229] Train net output #2: per_class_accuracy = 0.999469
I1126 12:45:20.671774 13474 solver.cpp:229] Train net output #3: per_class_accuracy = 0.0605096
I1126 12:45:20.671780 13474 solver.cpp:229] Train net output #4: per_class_accuracy = 0
I1126 12:45:20.671785 13474 solver.cpp:229] Train net output #5: per_class_accuracy = -nan
I1126 12:45:20.671790 13474 solver.cpp:229] Train net output #6: per_class_accuracy = -nan
I1126 12:45:20.671795 13474 solver.cpp:229] Train net output #7: per_class_accuracy = -nan
I1126 12:45:20.671810 13474 solver.cpp:229] Train net output #8: per_class_accuracy = -nan
I1126 12:45:20.671815 13474 solver.cpp:229] Train net output #9: per_class_accuracy = -nan
I1126 12:45:20.671823 13474 solver.cpp:486] Iteration 120, lr = 0.1
I1126 12:45:26.587383 13474 solver.cpp:214] Iteration 140, loss = 0.0888415
I1126 12:45:26.587419 13474 solver.cpp:229] Train net output #0: accuracy = 0.976074
I1126 12:45:26.587431 13474 solver.cpp:229] Train net output #1: loss = 0.0888415 (* 1 = 0.0888415 loss)
I1126 12:45:26.587437 13474 solver.cpp:229] Train net output #2: per_class_accuracy = 0.999803
I1126 12:45:26.587443 13474 solver.cpp:229] Train net output #3: per_class_accuracy = 0.12532
I1126 12:45:26.587448 13474 solver.cpp:229] Train net output #4: per_class_accuracy = -nan
I1126 12:45:26.587453 13474 solver.cpp:229] Train net output #5: per_class_accuracy = -nan
I1126 12:45:26.587458 13474 solver.cpp:229] Train net output #6: per_class_accuracy = 0
I1126 12:45:26.587463 13474 solver.cpp:229] Train net output #7: per_class_accuracy = -nan
I1126 12:45:26.587468 13474 solver.cpp:229] Train net output #8: per_class_accuracy = -nan
I1126 12:45:26.587473 13474 solver.cpp:229] Train net output #9: per_class_accuracy = 0.0593645
I1126 12:45:26.587481 13474 solver.cpp:486] Iteration 140, lr = 0.1
I1126 12:45:32.485353 13474 solver.cpp:214] Iteration 160, loss = 0.0526263
I1126 12:45:32.485385 13474 solver.cpp:229] Train net output #0: accuracy = 0.983314
I1126 12:45:32.485396 13474 solver.cpp:229] Train net output #1: loss = 0.0526263 (* 1 = 0.0526263 loss)
I1126 12:45:32.485402 13474 solver.cpp:229] Train net output #2: per_class_accuracy = 0.999622
I1126 12:45:32.485409 13474 solver.cpp:229] Train net output #3: per_class_accuracy = 0.528261
I1126 12:45:32.485414 13474 solver.cpp:229] Train net output #4: per_class_accuracy = -nan
I1126 12:45:32.485419 13474 solver.cpp:229] Train net output #5: per_class_accuracy = -nan
I1126 12:45:32.485424 13474 solver.cpp:229] Train net output #6: per_class_accuracy = 0.0106952
I1126 12:45:32.485429 13474 solver.cpp:229] Train net output #7: per_class_accuracy = -nan
I1126 12:45:32.485433 13474 solver.cpp:229] Train net output #8: per_class_accuracy = -nan
I1126 12:45:32.485438 13474 solver.cpp:229] Train net output #9: per_class_accuracy = 0.0363024
I1126 12:45:32.485445 13474 solver.cpp:486] Iteration 160, lr = 0.1
I1126 12:45:38.385970 13474 solver.cpp:214] Iteration 180, loss = 0.041313
I1126 12:45:38.386006 13474 solver.cpp:229] Train net output #0: accuracy = 0.988395
I1126 12:45:38.386018 13474 solver.cpp:229] Train net output #1: loss = 0.041313 (* 1 = 0.041313 loss)
I1126 12:45:38.386024 13474 solver.cpp:229] Train net output #2: per_class_accuracy = 0.995309
I1126 12:45:38.386030 13474 solver.cpp:229] Train net output #3: per_class_accuracy = 0.00348028
I1126 12:45:38.386036 13474 solver.cpp:229] Train net output #4: per_class_accuracy = 0
I1126 12:45:38.386041 13474 solver.cpp:229] Train net output #5: per_class_accuracy = -nan
I1126 12:45:38.386047 13474 solver.cpp:229] Train net output #6: per_class_accuracy = -nan
I1126 12:45:38.386051 13474 solver.cpp:229] Train net output #7: per_class_accuracy = -nan
I1126 12:45:38.386080 13474 solver.cpp:229] Train net output #8: per_class_accuracy = -nan
I1126 12:45:38.386086 13474 solver.cpp:229] Train net output #9: per_class_accuracy = -nan

Training accuracy for other datasets

hi @alexgkendall I am training the network on a different dataset. For that dataset the training accuracy does not seem to increase beyond 82-83%. After reaching that it tends to fall anywhere below 80%. Even using a learning rate of 10^-4 when 80% accuracy has been reached, does not solve the problem. Please help.

Unclear licensing policy

Hi.

https://github.com/alexgkendall/caffe-segnet/blob/segnet-cleaned/LICENSE
In LICENSE, there are 2 parts. The first one is about copyright and contains some "All rights reserved". The second part is about the license : CC BY 3.0 + a BSD-like license.

https://github.com/alexgkendall/caffe-segnet/blob/segnet-cleaned/README.md#license

This extension to the Caffe library is released under a creative commons license which allows for personal and research use only. You can view a license summary here: http://creativecommons.org/licenses/by/3.0/deed.en_US

In README.md, we can read that the license "allows for personal and research use only". But then, CC BY 3.0 is quoted, and this license (like the BSD-like) does not forbid commercial, state, non profit foundations, etc use.

So the licensing policy is unclear for me. Is it free/libre as described by the Free Software Foundation and Debian?

Regards.

How to specify the validation files during training

Hi @alexgkendall
How can I specify the validation file (val.txt) for validation for the segnet training ? I have found no input as specifying the train.txt file path in Segnet-tutorial-master/Models/segnet_basic_solver.prototxt.
And what's the parameter "test_initialization" mean in the solver.prototxt ?
Thanks a lot.

compute_bn_statistics Process finished with exit code 139

I use LMDB data to train a model successfully. But when I run the modified compute_bn_statistics.py, it crashes after processing a few images (randomly from 1 to 15).

_progress: 1/100
progress: 2/100
progress: 3/100
progress: 4/100
progress: 5/100
progress: 6/100
progress: 7/100

Process finished with exit code 139_
(7 is random sometimes 1, the largest 13 in several tirals)

Why it happens? It goes the same for cpu and gpu. How to fix it?
May it be out-of-memory?

make runtest error - What is my "Compute Capability" ?

While trying to run make runtest, I receive the infamous error

F0216 12:58:41.889211 12098 cudnn_softmax_layer.cpp:19] Check failed: status == CUDNN_STATUS_SUCCESS (6 vs. 0) CUDNN_STATUS_ARCH_MISMATCH
*** Check failure stack trace: ***
@ 0x2b111b85adaa (unknown)
@ 0x2b111b85ace4 (unknown)
@ 0x2b111b85a6e6 (unknown)
@ 0x2b111b85d687 (unknown)
@ 0x2b111d2122e5 caffe::CuDNNSoftmaxLayer<>::LayerSetUp()
@ 0x2b111d23221d caffe::SoftmaxWithLossLayer<>::LayerSetUp()
@ 0x2b111d29184b caffe::Net<>::Init()
@ 0x2b111d293592 caffe::Net<>::Net()
@ 0x6ff069 caffe::NetTest<>::InitNetFromProtoString()
@ 0x6bfd9c caffe::NetTest<>::InitTinyNet()
@ 0x6f3dc1 caffe::NetTest_TestBottomNeedBackward_Test<>::TestBody()
@ 0x7bf963 testing::internal::HandleExceptionsInMethodIfSupported<>()
@ 0x7b65a7 testing::Test::Run()
@ 0x7b664e testing::TestInfo::Run()
@ 0x7b6755 testing::TestCase::Run()
@ 0x7b9a98 testing::internal::UnitTestImpl::RunAllTests()
@ 0x7b9d27 testing::UnitTest::Run()
@ 0x451d4a main
@ 0x2b111de7aec5 (unknown)
@ 0x457e99 (unknown)
@ (nil) (unknown)
make: *** [runtest] Aborted (core dumped)

I found a few conversations and all of them state that cudnn will not work if the compute capability of my GPU is smaller than 3.0.
I have a NVidia Quadro 2000. How do I know what is my compute capability? Couldn't find it

Can something else cause this issue?
BTW, I'm working on a fresh Ubuntu 14.04, and Cuda6.5 required for SegNet. On CPU everything works (but very slowly).

upsample layer problem

Hi @alexgkendall ,

My data have 8 classes, when I change the number of class in the model file, I had this problem. Can you please have a look at my segnet_basic_train.prototxt file (http://pastebin.com/8SFwqtij) and show me what's wrong?

Thanks!

F1125 15:28:05.204437 5212 upsample_layer.cpp:63] Check failed: bottom[0]->height() == bottom[1]->height() (59 vs. 60)
*** Check failure stack trace: ***
@ 0x7f7bf11c3daa (unknown)
@ 0x7f7bf11c3ce4 (unknown)
@ 0x7f7bf11c36e6 (unknown)
@ 0x7f7bf11c6687 (unknown)
@ 0x7f7bf1557af9 caffe::UpsampleLayer<>::Reshape()
@ 0x7f7bf15f3f07 caffe::Net<>::Init()
@ 0x7f7bf15f5cb2 caffe::Net<>::Net()
@ 0x7f7bf14ecb00 caffe::Solver<>::InitTrainNet()
@ 0x7f7bf14edbe3 caffe::Solver<>::Init()
@ 0x7f7bf14eddb6 caffe::Solver<>::Solver()
@ 0x40edf0 caffe::GetSolver<>()
@ 0x408683 train()
@ 0x406c81 main
@ 0x7f7bf06d5ec5 (unknown)
@ 0x40722d (unknown)
@ (nil) (unknown)
Aborted (core dumped)

error in 'make runtest'

Dear all,

I had followed the general installation steps for installing caffe-segnet. But I came across the following error in 'make runtest' step.

[----------] Global test environment tear-down
[==========] 1400 tests from 222 test cases ran. (190603 ms total)
[ PASSED ] 1399 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] NetUpgradeTest.TestUpgradeV1LayerType

1 FAILED TEST
YOU HAVE 2 DISABLED TESTS

Could someone suggest me how to fix it?

Best regards,
Milton

Testing some simple images

I was trying to predict the pixel labels of some simple RGB images (without ground truth annotations) using the pre-trained model (like webdemo does). I put my images in '/home/ted/XMUdataset/KeLi' and generate a test.txt file in '/home/ted/XMUdataset' which looks like:

/home/ted/XMUdataset/KeLi/2001.jpg
/home/ted/XMUdataset/KeLi/2002.jpg
/home/ted/XMUdataset/KeLi/2003.jpg
/home/ted/XMUdataset/KeLi/2004.jpg
/home/ted/XMUdataset/KeLi/2005.jpg
/home/ted/XMUdataset/KeLi/2006.jpg
......

Then I changed the data layer in segnet_inference.prototxt like below:

layer {
  name: "data"
  type: "Data"
  top: "data"
  data_param {
    source: "/home/ted/XMUdataset/test.txt" # Change this to the absolute path to your data file
    batch_size: 1
  }
}

Finally, I edited the test_segmentation_camvid.py to this:

import numpy as np
import matplotlib.pyplot as plt
import os.path
import json
import scipy
import argparse
import math
import pylab
from sklearn.preprocessing import normalize
caffe_root = '/home/ted/SegNet/caffe-segnet/'           # Change this to the absolute directoy to SegNet Caffe
import sys
sys.path.insert(0, caffe_root + 'python')

import caffe

# Import arguments
parser = argparse.ArgumentParser()
parser.add_argument('--model', type=str, required=True)
parser.add_argument('--weights', type=str, required=True)
parser.add_argument('--iter', type=int, required=True)
args = parser.parse_args()

caffe.set_mode_gpu()

net = caffe.Net(args.model,
                args.weights,
                caffe.TEST)


for i in range(0, args.iter):

    net.forward()

    image = net.blobs['data'].data
    predicted = net.blobs['prob'].data
    image = np.squeeze(image[0,:,:,:])
    output = np.squeeze(predicted[0,:,:,:])
    ind = np.argmax(output, axis=0)

    r = ind.copy()
    g = ind.copy()
    b = ind.copy()

    Sky = [128,128,128]
    Building = [128,0,0]
    Pole = [192,192,128]
    Road_marking = [255,69,0]
    Road = [128,64,128]
    Pavement = [60,40,222]
    Tree = [128,128,0]
    SignSymbol = [192,128,128]
    Fence = [64,64,128]
    Car = [64,0,128]
    Pedestrian = [64,64,0]
    Bicyclist = [0,128,192]
    Unlabelled = [0,0,0]

    label_colours = np.array([Sky, Building, Pole, Road, Pavement, Tree, SignSymbol, Fence, Car, Pedestrian, Bicyclist, Unlabelled])
    for l in range(0,11):
        r[ind==l] = label_colours[l,0]
        g[ind==l] = label_colours[l,1]
        b[ind==l] = label_colours[l,2]

    rgb = np.zeros((ind.shape[0], ind.shape[1], 3))
    rgb[:,:,0] = r/255.0
    rgb[:,:,1] = g/255.0
    rgb[:,:,2] = b/255.0

    image = image/255.0

    image = np.transpose(image, (1,2,0))
    output = np.transpose(output, (1,2,0))
    image = image[:,:,(2,1,0)]


    #scipy.misc.toimage(rgb, cmin=0.0, cmax=255).save(IMAGE_FILE+'_segnet.png')

    plt.figure()
    plt.imshow(image,vmin=0, vmax=1)
    plt.figure()
    plt.imshow(rgb,vmin=0, vmax=1)
    plt.show()


print 'Success!'

Yet after I run the script above, I got the following error:

I0114 15:04:08.348337 34065 layer_factory.hpp:74] Creating layer data
I0114 15:04:08.348378 34065 net.cpp:90] Creating Layer data
I0114 15:04:08.348390 34065 net.cpp:368] data -> data
I0114 15:04:08.348405 34065 net.cpp:120] Setting up data
F0114 15:04:08.348466 34065 db.cpp:18] Check failed: status.ok() Failed to open leveldb /home/ted/XMUdataset/test.txt
IO error: /home/ted/XMUdataset/test.txt/LOCK: Not a directory
*** Check failure stack trace: ***
Aborted (core dumped)

Could anyone give me a hint about where I am wrong? So many thanks in advance.

processing 4-channel images

Can SegNet process 4-channel ( or more channel ) images ?
What i need to change in source code, to process more then 3 channel images ?

pretrained model still need compute_bn_statistics

I'm wondering if I use pretrained model, do I need to correct for batchNorm? One thing I found is, if I use pretrained model, but don't do the compute_bn_statistics step, the model output different results when I run same input data twice. Is it related to batchnorm?

Thanks,

USE_CUDNN errors

Ubuntu 14.04
CUDA 7.5
cuDNN 4

I am getting multiple errors when I uncomment USE_CUDNN := 1 in the Makefile.config.

I can not reproduce the same errors when using vanilla caffe with similar Makefile.config.

Here is an example of the errors I am getting (1 of 20):

src/caffe/layers/cudnn_conv_layer.cu(81): error: argument of type "cudnnAddMode_t" is incompatible with parameter of type "const void *" detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Forward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=float]" (157): here

All errors relate to the file located here: src/caffe/layers/cudnn_conv_layer.cu
Is this an issue with cuDNN v4?

I can post all errors if required.

Segmentation fault at Softmax

Hi Alex,

Hope someone can help me with this issue.
I'm just reusing the network architecture with only 4 output classes. Image dimensions remain same. Made the necessary changes in the prototext files. I get the following error when I start the training.

I0531 09:45:15.546435 2031686400 net.cpp:248] Memory required for data: 1068595224
I0531 09:45:15.546778 2031686400 solver.cpp:42] Solver scaffolding done.
I0531 09:45:15.546954 2031686400 solver.cpp:250] Solving VGG_ILSVRC_16_layer
I0531 09:45:15.546959 2031686400 solver.cpp:251] Learning Rate Policy: step
*** Aborted at 1464668121 (unix time) try "date -d @1464668121" if you are using GNU date ***
PC: @ 0x10b25b442 caffe::SoftmaxWithLossLayer<>::Forward_cpu()
*** SIGSEGV (@0x216caa000) received by PID 55425 (TID 0x7fff79191300) stack trace: ***
@ 0x7fff918dff1a _sigtramp
@ 0x100000000000000 (unknown)
@ 0x10b2209d9 caffe::Layer<>::Forward()
@ 0x10b2760d9 caffe::Net<>::ForwardFromTo()
@ 0x10b276718 caffe::Net<>::Forward()
@ 0x10b285dcf caffe::Solver<>::Step()
@ 0x10b28577c caffe::Solver<>::Solve()
@ 0x10b1b1594 train()
@ 0x10b1b392f main
@ 0x7fff8c7e25c9 start
@ 0x4 (unknown)
Segmentation fault: 11

Any help in this regard. Appreciate your time being spent on this.

blob shape error

Hi, @kezbreen

I try your code, but got the following error.
would you pls help me to solve it?
thank you.

I0811 18:53:12.090750 13152 solver.cpp:250] Solving segnet
I0811 18:53:12.090750 13152 solver.cpp:251] Learning Rate Policy: fixed
I0811 18:53:12.095751 13152 solver.cpp:294] Iteration 0, Testing net (#0)
F0811 18:53:12.095751 13152 blob.cpp:24] Check failed: shape.size() <= kMaxBlobAxes (18446744073696301408 vs. 2147483647)
*** Check failure stack trace: ***

build error in caffe-segnet

When I build caffe-segnet, this error happens. I build with cudnn option, using NVIDIA 940M, in xubuntu 14.04, CUDA 7.5. I succeeded building with cudnn option in the BVLC caffe, but this one doesn't. What is wrong in this situation?

Here's my build error log. I did the command make all -j4.

src/caffe/layers/cudnn_conv_layer.cu(81): error: argument of type "cudnnAddMode_t" is incompatible with parameter of type "const void *"
          detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Forward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=float]" 
(157): here

src/caffe/layers/cudnn_conv_layer.cu(81): error: argument of type "const void *" is incompatible with parameter of type "cudnnTensorDescriptor_t"
          detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Forward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=float]" 
(157): here

src/caffe/layers/cudnn_conv_layer.cu(81): error: argument of type "const void *" is incompatible with parameter of type "cudnnTensorDescriptor_t"
          detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Forward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=float]" 
(157): here

src/caffe/layers/cudnn_conv_layer.cu(81): error: too many arguments in function call
          detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Forward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=float]" 
(157): here

src/caffe/layers/cudnn_conv_layer.cu(125): error: argument of type "const void *" is incompatible with parameter of type "cudnnConvolutionBwdFilterAlgo_t"
          detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Backward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<__nv_bool, std::allocator<__nv_bool>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=float]" 
(157): here

src/caffe/layers/cudnn_conv_layer.cu(125): error: argument of type "float *" is incompatible with parameter of type "size_t"
          detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Backward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<__nv_bool, std::allocator<__nv_bool>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=float]" 
(157): here

src/caffe/layers/cudnn_conv_layer.cu(125): error: too few arguments in function call
          detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Backward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<__nv_bool, std::allocator<__nv_bool>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=float]" 
(157): here

src/caffe/layers/cudnn_conv_layer.cu(140): error: argument of type "const void *" is incompatible with parameter of type "cudnnConvolutionBwdDataAlgo_t"
          detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Backward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<__nv_bool, std::allocator<__nv_bool>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=float]" 
(157): here

src/caffe/layers/cudnn_conv_layer.cu(140): error: argument of type "float *" is incompatible with parameter of type "size_t"
          detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Backward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<__nv_bool, std::allocator<__nv_bool>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=float]" 
(157): here

src/caffe/layers/cudnn_conv_layer.cu(140): error: too few arguments in function call
          detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Backward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<__nv_bool, std::allocator<__nv_bool>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=float]" 
(157): here

src/caffe/layers/cudnn_conv_layer.cu(81): error: argument of type "cudnnAddMode_t" is incompatible with parameter of type "const void *"
          detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Forward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=double]" 
(157): here

src/caffe/layers/cudnn_conv_layer.cu(81): error: argument of type "const void *" is incompatible with parameter of type "cudnnTensorDescriptor_t"
          detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Forward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=double]" 
(157): here

src/caffe/layers/cudnn_conv_layer.cu(81): error: argument of type "const void *" is incompatible with parameter of type "cudnnTensorDescriptor_t"
          detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Forward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=double]" 
(157): here

src/caffe/layers/cudnn_conv_layer.cu(81): error: too many arguments in function call
          detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Forward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=double]" 
(157): here

src/caffe/layers/cudnn_conv_layer.cu(125): error: argument of type "const void *" is incompatible with parameter of type "cudnnConvolutionBwdFilterAlgo_t"
          detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Backward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<__nv_bool, std::allocator<__nv_bool>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=double]" 
(157): here

src/caffe/layers/cudnn_conv_layer.cu(125): error: argument of type "double *" is incompatible with parameter of type "size_t"
          detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Backward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<__nv_bool, std::allocator<__nv_bool>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=double]" 
(157): here

src/caffe/layers/cudnn_conv_layer.cu(125): error: too few arguments in function call
          detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Backward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<__nv_bool, std::allocator<__nv_bool>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=double]" 
(157): here

src/caffe/layers/cudnn_conv_layer.cu(140): error: argument of type "const void *" is incompatible with parameter of type "cudnnConvolutionBwdDataAlgo_t"
          detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Backward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<__nv_bool, std::allocator<__nv_bool>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=double]" 
(157): here

src/caffe/layers/cudnn_conv_layer.cu(140): error: argument of type "double *" is incompatible with parameter of type "size_t"
          detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Backward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<__nv_bool, std::allocator<__nv_bool>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=double]" 
(157): here

src/caffe/layers/cudnn_conv_layer.cu(140): error: too few arguments in function call
          detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Backward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<__nv_bool, std::allocator<__nv_bool>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=double]" 
(157): here

20 errors detected in the compilation of "/tmp/tmpxft_00006569_00000000-16_cudnn_conv_layer.compute_50.cpp1.ii".
make: *** [.build_release/cuda/src/caffe/layers/cudnn_conv_layer.o] Error 1
make: *** Waiting for unfinished jobs....

Can you use multiple GPUs?

Looking through the code, I'm mainly seeing only a single GPU being used. Is it possible to use multiple GPUs?

ask HowTo do the training and label prediction for CamVid dataset

Hi, @alexgkendall ,

I had installed caffe-segnet successfully. Now I want to practice some examples such as training CamVid dataset which was discussed in paper (SegNet). Would you list the steps and corresponding command for achieving the training and label prediction for CamVid dataset?

Thx~
Milton

the size of output

Hi Alex,
I tried your web demo with KITTI images (their normal size is 375 by 1240), but the size of output changed into 360 by 480. I hope that the output size keeps the same as that of the KITTI image.
As I am still a newbie in deep networks, could you please explain to me in details how to get such an output? I will highly appreciate it.

output predictions are coarse

Hi,
I am using segnet basic to perform binary classification on images of size (hxw) 256 x 1240, the output predictions are coarse, I am trying to figure out the reason for this behavior as it looks like its detecting the objects in the correct region but there are gaps in between which I think are created by some down sampling or up sampling mismatch.

1. In your paper http://arxiv.org/pdf/1511.00561v2.pdf in section decoder variants its mentioned for the images(360×480) you have used kernel size of 7 × 7 and a pixel in the deepest layer feature map (layer 4) can be traced back to a context window in the input image of 106×106 pixels. Could you explain the math behind this? as I am not sure if using 7x7 kernel size for my images is the best approach

2)Also is there a way to visualize the activations and layer weights do better understand the reason for this coarse output

sample output

Data Question

Hi there,

When I run SegNet on my own data, I produce the following error:

---

I1016 18:40:51.863139 20505 net.cpp:120] Setting up upsample3
F1016 18:40:51.863159 20505 upsample_layer.cpp:63] Check failed: bottom[0]->height() == bottom[1]->height() (45 vs. 32)
*** Check failure stack trace: ***
@ 0x7faf9a4d8ddd (unknown)
@ 0x7faf9a4dacc0 (unknown)
@ 0x7faf9a4d89ac (unknown)
@ 0x7faf9a4db6be (unknown)
@ 0x7faf9fca971e caffe::UpsampleLayer<>::Reshape()
@ 0x7faf9fce8512 caffe::Net<>::Init()
@ 0x7faf9fceaa35 caffe::Net<>::Net()
@ 0x7faf9fcf60d6 caffe::Solver<>::InitTrainNet()
@ 0x7faf9fcf73c0 caffe::Solver<>::Init()
@ 0x7faf9fcf7586 caffe::Solver<>::Solver()
@ 0x40fd00 caffe::GetSolver<>()
@ 0x408050 train()
@ 0x405b79 main
@ 0x7faf92392700 __libc_start_main
@ 0x4061b9 _start
Aborted (core dumped)

---

The runtest is all good. And so is the example training. I'm trying to figure out the problem with my data. I'm very new to caffe so a little help would really be appreciated!

Minimum hardware specs

This is more of a question than an issue but not sure of the appropriate place to ask.

I was wondering if you had a minimum hardware requirement for SegNet. I was hoping to run (not train) on a Jetson TX1.

Also I was wondering if this would train on a laptop. It's reasonably high spec for a laptop but I realise that we are talking ANNs here.

I was going to build but then I saw issue #21 (Out of memory error!!)

My laptop has a GTX970M: http://www.geforce.com/hardware/notebook-gpus/geforce-gtx-970m/specifications
Here is more info on the TX1: http://www.nvidia.com/object/jetson-tx1-module.html

BTW I would like to congratulate you on the great work. The demo is impressive.

Cheers.

Upsample top index 1 out of range

First, I would like thankful to your good tutorial. Because I have not enough GPU memory (just 2GB, GTX 750 Ti), so I use CPU mode in the segnet_solver.prototxt as follows:

output solver snapshots
solver_mode: CPU

After that, I ran

./caffe-segnet/build/tools/caffe train -solver Models/segnet_solver.prototxt -weights Models/VGG_ILSVRC_16_layers.caffemodel

Then, I got the error as

F0504 21:52:28.590580  6107 upsample_layer.cpp:127] upsample top index 1 out of range - check scale settings match input pooling layer's downsample setup
*** Check failure stack trace: ***
    @     0x7f5aa3486daa  (unknown)
    @     0x7f5aa3486ce4  (unknown)
    @     0x7f5aa34866e6  (unknown)
    @     0x7f5aa3489687  (unknown)
    @     0x7f5aa382473f  caffe::UpsampleLayer<>::Backward_cpu()
    @     0x7f5aa3799d85  caffe::Net<>::BackwardFromTo()
    @     0x7f5aa3799f21  caffe::Net<>::Backward()
    @     0x7f5aa38b9f0d  caffe::Solver<>::Step()
    @     0x7f5aa38ba82f  caffe::Solver<>::Solve()
    @           0x4086c8  train()
    @           0x406c61  main
    @     0x7f5aa2998ec5  (unknown)
    @           0x40720d  (unknown)
    @              (nil)  (unknown)
Aborted (core dumped)

I hear that this issue only appears in CPU mode. Could you help me to fix it. Thank you in advanced

runtime error in debug mode

When I run the training demo (CamVid) in the release mode, it works. But when I run in the debug model, it generates the following checking failure error:
__
I0225 10:53:33.101868 3843 net.cpp:235] This network produces output accuracy
I0225 10:53:33.101879 3843 net.cpp:235] This network produces output loss
I0225 10:53:33.101889 3843 net.cpp:235] This network produces output per_class_accuracy
I0225 10:53:33.101984 3843 net.cpp:482] Collecting Learning Rate and Weight Decay.
I0225 10:53:33.102026 3843 net.cpp:247] Network initialization done.
I0225 10:53:33.102035 3843 net.cpp:248] Memory required for data: 4332441652
I0225 10:53:33.102377 3843 solver.cpp:42] Solver scaffolding done.
I0225 10:53:33.102727 3843 solver.cpp:250] Solving VGG_ILSVRC_16_layer
I0225 10:53:33.102740 3843 solver.cpp:251] Learning Rate Policy: step
I0225 10:53:33.151669 3849 dense_image_data_layer.cpp:181] Prefetch batch: 41 ms.
I0225 10:53:33.151747 3849 dense_image_data_layer.cpp:182] Read time: 26.489 ms.
I0225 10:53:33.151759 3849 dense_image_data_layer.cpp:183] Transform time: 14.965 ms.
**F0225 10:53:33.680835 3843 accuracy_layer.cpp:72] Check failed: label_value < num_labels (11 vs. 11)
***** Check failure stack trace: ***
@ 0x7f98fe11edaa (unknown)
@ 0x7f98fe11ece4 (unknown)
@ 0x7f98fe11e6e6 (unknown)
@ 0x7f98fe121687 (unknown)
@ 0x7f98fe56a015 caffe::AccuracyLayer<>::Forward_cpu()
@ 0x7f98fe569183 caffe::Layer<>::Forward_gpu()
@ 0x423024 caffe::Layer<>::Forward()
@ 0x7f98fe605cad caffe::Net<>::ForwardFromTo()
@ 0x7f98fe605a75 caffe::Net<>::ForwardPrefilled()
@ 0x7f98fe605e40 caffe::Net<>::Forward()
@ 0x7f98fe606755 caffe::Net<>::ForwardBackward()
@ 0x7f98fe6212bb caffe::Solver<>::Step()
@ 0x7f98fe620cc5 caffe::Solver<>::Solve()
@ 0x41e9a8 train()
@ 0x42095c main
@ 0x7f98fd934ec5 (unknown)
@ 0x41dda9 (unknown)
@ (nil) (unknown)
Aborted (core dumped)_
_

Anyone with the same problem and how can I resolve it?

upsample top index xxx out of range

Hi,@kezbreen
I try your code, but got the following error.
I followed the steps in README.md,step by step,here is the list:

download camvid dataset(701 images) and transform the label by using the script from issue #3
resized the original pics into [360 480].
generate data.txt
change the path in net.prototxt(first 2 layer)
change the 'dense_softmax_inner_prod' layer in "num_output: 32" (camvid 32 kinds of label)
change the solver.prototxt "solver_mode" to cpu(i used the Geforce 970,it showed error==cudaSuccess(2,0), i thought it was not enough memory ? So I change to cpu)
then I start training...

...
I1002 10:58:40.084039 3306 solver.cpp:42] Solver scaffolding done.
I1002 10:58:40.084084 3306 solver.cpp:250] Solving segnet
I1002 10:58:40.084090 3306 solver.cpp:251] Learning Rate Policy: fixed
I1002 10:58:40.085350 3306 solver.cpp:294] Iteration 0, Testing net (#0)
I1002 10:58:46.428558 3306 solver.cpp:343] Test net output #0: accuracy = 0.0286227
I1002 10:58:46.428601 3306 solver.cpp:343] Test net output #1: loss = 3.49524 (* 1 = 3.49524 loss)
F1002 10:58:58.492833 3306 upsample_layer.cpp:127] upsample top index 481 out of range - check scale settings match input pooling layer's downsample setup
*** Check failure stack trace: ***
@ 0x7fc354130ea4 (unknown)
@ 0x7fc354130deb (unknown)
@ 0x7fc3541307bf (unknown)
@ 0x7fc354133a35 (unknown)
@ 0x7fc359fa814d caffe::UpsampleLayer<>::Backward_cpu()
@ 0x7fc359f551e9 caffe::Net<>::BackwardFromTo()
@ 0x7fc359f55551 caffe::Net<>::Backward()
@ 0x7fc359f61135 caffe::Solver<>::Step()
@ 0x7fc359f61da5 caffe::Solver<>::Solve()
@ 0x407c7b train()
@ 0x404ed8 main
@ 0x7fc3509a9a40 (unknown)
@ 0x404a79 _start
@ (nil) (unknown)
Aborted (core dumped)
...

I change the batch size to 5 in net.prototxt, the result is
F1002 10:58:58.492833 3306 upsample_layer.cpp:127] upsample top index 0 out of range - check scale settings match input pooling layer's downsample setup

Please help, i'm new in caffe .Thanks

Out of memory error !!

I am trying to train camseq databse using segnet. I was able to prepare the dataset. But when I run training, I am getting Out of memory error. I am using nvidia gtx960 4 GB. I have even reduced batch size to 1 in both train and test, but I'm still getting the error

check failed: error == cudaSuccess (2 vs. 0) out of memory.

Missing folder build/tools/caffe

Hallo,

when I follow the tutorial it says:

./SegNet/caffe-segnet/build/tools/caffe train -gpu 0 -solver /SegNet/Models/segnet_solver.prototxt # This will begin training SegNet on GPU 0

However, this path does not exist for me. There is no build/tools/caffe.
In caffe-segnet are 3 caffe folders. In caffe-segnet/src/caffe, caffe-segnet/python/caffe and caffe-segnet/include/caffe. However, it does not work with any of these paths.

Can someone help?

Unknown blob input norm to layer 0 - can't find prototxt & dataset

Hello,

I can't find the proper net.prototxt, i'm trying to use the one in /examples but it seems to use an undefined layer called "norm".

Also in the tutorial the link to CamVid datase seems to be broken (I'm trying with http://mi.eng.cam.ac.uk/research/projects/VideoRec/ but there is much less frames (101 instead of 700)

Question about data

For example my images have only 2 class {0,1} labels, assume 0 class is background and 1 class in some object.
Is it will be helpfull if I use some images with only background or it will be harmfull, because of too unbalanced data?