alexgkendall/segnet-tutorial

Files for a tutorial to train SegNet for road scenes using the CamVid dataset

SegNet and Bayesian SegNet Tutorial

This repository contains all the files for you to complete the 'Getting Started with SegNet' and the 'Bayesian SegNet' tutorials here: http://mi.eng.cam.ac.uk/projects/segnet/tutorial.html

Please see this link for detailed instructions.

Caffe-SegNet

SegNet requires a modified version of Caffe to run. Please download and compile caffe-segnet to use these models: https://github.com/alexgkendall/caffe-segnet

This version supports cudnn v2 acceleration. @TimoSaemann has a branch supporting a more recent version of Caffe (Dec 2016) with cudnn v5.1: https://github.com/TimoSaemann/caffe-segnet-cudnn5

Getting Started with Live Demo

If you would just like to try out an example model, then you can find the model used in the SegNet webdemo in the folder Example_Models/. You will need to download the weights separately using the link in the SegNet Model Zoo.

First open Scripts/webcam_demo.py and edit line 14 to match the path to your installation of SegNet. You will also need a webcam, or alternatively edit line 39 to input a video file instead. To run the demo use the command:

python Scripts/webcam_demo.py --model Example_Models/segnet_model_driving_webdemo.prototxt --weights /Example_Models/segnet_weights_driving_webdemo.caffemodel --colours /Scripts/camvid12.png

Getting Started with Docker

Use docker to compile caffe and run the examples. In order to run caffe on the gpu using docker, please install nvidia-docker (see https://github.com/NVIDIA/nvidia-docker or using ansbile: https://galaxy.ansible.com/ryanolson/nvidia-docker/)

to run caffe on the CPU:

docker build -t bvlc/caffe:cpu ./cpu 
# check if working
docker run -ti bvlc/caffe:cpu caffe --version
# get a bash in container to run examples
docker run -ti --volume=$(pwd):/SegNet -u $(id -u):$(id -g) bvlc/caffe:cpu bash

to run caffe on the GPU:

docker build -t bvlc/caffe:gpu ./gpu
# check if working
docker run -ti bvlc/caffe:gpu caffe device_query -gpu 0
# get a bash in container to run examples
docker run -ti --volume=$(pwd):/SegNet -u $(id -u):$(id -g) bvlc/caffe:gpu bash

Example Models

A number of example models for indoor and outdoor road scene understanding can be found in the SegNet Model Zoo.

Publications

For more information about the SegNet architecture:

http://arxiv.org/abs/1511.02680 Alex Kendall, Vijay Badrinarayanan and Roberto Cipolla "Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding." arXiv preprint arXiv:1511.02680, 2015.

http://arxiv.org/abs/1511.00561 Vijay Badrinarayanan, Alex Kendall and Roberto Cipolla "SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation." PAMI, 2017.

License

This software is released under a creative commons license which allows for personal and research use only. For a commercial license please contact the authors. You can view a license summary here: http://creativecommons.org/licenses/by-nc/4.0/

Contact

Alex Kendall

[email protected]

Cambridge University

segnet-tutorial's Issues

Batch Normalization

Hi All,
I am running into issues when trying to run batch normalization. Specifically, I train a network to differentiate between two different pixel labels. The results converge and approach about 90% accuracy. When I test images using this network, the results are pretty good (using models_iter_20000.caffemodel). However, when I try to run normalization, the results get much much worse. Does anyone have a reason why this might happen? Or am I maybe messing something up during the normalization step?
Thanks!

Segnet+Cityscapes : Accuracy issue

Hi @alexgkendall
,
I trained segnet(non basic non Bayesian version ) on the Cityscapes dataset(44gb of 19k+ images and annotations that need to be converted to segnet annotations). However training it on cityscapes and testing it on a new test image does not produce the same "smooth" results as shown by webcam demo.
I tried to show the difference here(excuse the color difference:red is road and purple is pavement and so on:

My soln(the road and pavements are not smooth,many miscalssifications):

Your webdemo(testing on similar image of mine):

However on camvid testimage:

So here goes my question:
1)webdemo was trained on ~500 images it gives great results.I trained segnet with 19k+ images and 90000+ iterations and dint change anything else.Why am I not getting a smooth performance. Should it not do better.
2)Also the cityscapes dataset has a resolution of 1024x2048 which i rescaled to 360x480 for segnet. My test image which segnet has never trained on is 1080x1920 which i rescaled to 360x480.Does resolution matter?
3)Are you testing it differently to get a smoother performance.?All I am doing is similar to testcamvid.py in ur code:

image= cv2.resize(image, (input_shape[3], input_shape[2]))
input_image = image.transpose((2, 0, 1))
input_image = input_image[(2, 1, 0), :, :]
input_image = np.asarray([input_image])

out = net.forward_all(data=input_image)
segmentation_ind = np.squeeze(net.blobs['argmax'].data)
segmentation_ind_3ch = np.resize(segmentation_ind, (3, input_shape[2], input_shape[3]))
segmentation_ind_3ch = segmentation_ind_3ch.transpose(1, 2, 0).astype(np.uint8)
segmentation_rgb = np.zeros(segmentation_ind_3ch.shape, dtype=np.uint8)
cv2.LUT(segmentation_ind_3ch, label_colours, segmentation_rgb)
segmentation_rgb = segmentation_rgb.astype(float) / 255.0

Could you please help me understand this better?

also note: -the training began with vgg16 ILSVRC pretrained encoder

also pls find the training loss:

Problem in CPU mode

I am using caffe in CPU mode. I have made the necessary changes in the solver file. However what are the changes that need to be made on this terminal command?
/SegNet/caffe-segnet/build/tools/caffe train -gpu 0 -solver /SegNet/Models/segnet_solver.prototxt

Running error (Live Demo)

Dear @alexgkendall

I'm following the "Getting Started with Live Demo" section, but it fails.
Please see the running error as follows:

root@milton-Desktop:/SegNet# python Scripts/webcam_demo.py --model Example_Models/segnet_model_driving_webdemo.prototxt --weights Example_Models/segnet_weights_driving_webdemo.caffemodel --colours /Scripts/camvid12.png

libprotobuf ERROR google/protobuf/text_format.cc:172] Error parsing text-format caffe.NetParameter: 1605:9: Message type "caffe.ArgMaxParameter" has no field named "axis".
WARNING: Logging before InitGoogleLogging() is written to STDERR
F0114 09:55:46.792676 7218 upgrade_proto.cpp:928] Check failed: ReadProtoFromTextFile(param_file, param) Failed to parse NetParameter file: Example_Models/segnet_model_driving_webdemo.prototxt
*** Check failure stack trace: ***
Aborted (core dumped)

Any suggestion to fix this error?

Thanks~
Milton

"caffe.LayerParameter" has no field named "dense_image_data_param".

I got the following error when trying to run SegNet basic using only CPU. Basically the error says there is no field name called "dense_image_data_param". which it reads from segnet_basic_train.prototxt?

Error Output:

I1201 14:55:25.982578 4028 caffe.cpp:177] Use CPU.
I1201 14:55:25.983726 4028 solver.cpp:48] Initializing solver from parameters:
test_iter: 1
test_interval: 10000000
base_lr: 0.1
display: 20
max_iter: 10000
lr_policy: "step"
gamma: 1
momentum: 0.9
weight_decay: 0.0005
stepsize: 10000000
snapshot: 1000
snapshot_prefix: "/home/crefeda/Desktop/SegNet/Models/Training/segnet_basic"
solver_mode: CPU
net: "/home/crefeda/Desktop/SegNet/Models/segnet_basic_train.prototxt"
test_initialization: false
I1201 14:55:25.984231 4028 solver.cpp:91] Creating training net from net file: /home/crefeda/Desktop/SegNet/Models/segnet_basic_train.prototxt
[libprotobuf ERROR google/protobuf/text_format.cc:245] Error parsing text-format caffe.NetParameter: 7:26: Message type "caffe.LayerParameter" has no field named "dense_image_data_param".
F1201 14:55:25.984504 4028 upgrade_proto.cpp:68] Check failed: ReadProtoFromTextFile(param_file, param) Failed to parse NetParameter file: /home/crefeda/Desktop/SegNet/Models/segnet_basic_train.prototxt
*** Check failure stack trace: ***
@ 0x7f9e2a9e4daa (unknown)
@ 0x7f9e2a9e4ce4 (unknown)
@ 0x7f9e2a9e46e6 (unknown)
@ 0x7f9e2a9e7687 (unknown)
@ 0x7f9e2ae712ae caffe::ReadNetParamsFromTextFileOrDie()
@ 0x7f9e2ae5531b caffe::Solver<>::InitTrainNet()
@ 0x7f9e2ae563fc caffe::Solver<>::Init()
@ 0x7f9e2ae56709 caffe::Solver<>::Solver()
@ 0x7f9e2ad5a883 caffe::Creator_SGDSolver<>()
@ 0x40e35e caffe::SolverRegistry<>::CreateSolver()
@ 0x4076cb train()
@ 0x405651 main
@ 0x7f9e29cf2ec5 (unknown)
@ 0x405d61 (unknown)
@ (nil) (unknown)
Aborted (core dumped)

Add "AdaGrad" to solver file fails

Hi

if I add "AdaGrad" to segnet_solver.prototxt i get following error:
Message type "caffe.SolverParameter" has no field named "type".

Is this due to segnet-caffe? Is there a workaround?

Best
Timo

Classification in C++

Hi,
i am looking for a C++ example to classifiy an OpenCV mat (like the webcam demo)
Could you describe or document the corresponding C++ caffe calls?
Thank you and best Regards,
Udo

webcam demo results in black segmentation images

I have followed the instructions given by the SegNet Tutorial along with the README.md to run Scripts/webcam_demo.py on a KITTI image sequence.

I made the necessary changes to Scripts/webcam_demo.py to:

• point to my caffe-segnet
• read in the image sequence files rather than webcam frames
• save the segmentation results to disk rather than displayed in a cv2.namedWindow

I also downloaded the weights and executed the script exactly as given in the readme file.

The script iterates over all images in the sequence and saves them to disk, however all segmentation results are just black images.

As a simple diagnostic, I saved the resized input image to disk, which appear normal.

Opening the black segmentation image within Matlab and inspecting the pixel channel values reveals that the values consist of zeros and ones without any discernible pattern.

How to use compute_test_results.m script

Hi @alexgkendall ,

Thank for uploading the evaluation script. However, could you please write a few lines about how to use this script with your CamVid dataset? I don't familiar with Matlab thought.

Thanks

how to add more ignore_label?

Webcam demo results in random colored pixels

I use this commnad for webcam_demo.py (model and weights from SegNet Model Zoo):

> python Scripts/webcam_demo.py --model Example_Models/segnet_model_driving_webdemo.prototxt --weights /Example_Models/segnet_weights_driving_webdemo.caffemodel --colours /Scripts/camvid12.png

but I see something like this all the time

How to Label my own images?

I have my own images that I wish to annotate. Which tool did you use to label your dataset?

1. During training, in 'segnet_train.prototxt', the final output is of 11 classes, but in the 'segnet_model_driving_webdemo.prototxt', the final output is 12 classes. I don't understand why is that.

2. Secondly, the labels of the images in the CamVid contain a different shade of grey for every class. Can you list the classes and the corresponding grey labels for them?

pre-trained model on the KITTI dataset

Hi, @alexgkendall, could you please release your pre-trained model on the KITTI dataset?

SegNet Tutorial Test Output

Hi all,

I trained the segnet Model according to the segnet Tutorial with my own Data,everything works fine so far.
I have 174 training pictures, but when i run compute_bn_statistics.py the progress bar only goes from 1/174...to 173/174 but it finishes without an error. what happens here, shouldnt it run up to 174/174?i didnt change anything within the py-file except the caffe segnet path.
When i start the test segmentation file then i recognize that the input test Image n is shown in window Figure1 and in window Figure2 the Test Image n+1 is shown with some strange colored pixels. The prediction in window Figure2 seems to be correct again.

What should be correctly shown in Figure2?

Thank you!

No labeled Images?

Hi Alex,

when I follow your tutorial, I cannot train a model because the filenames in train.txt have changed.
In fact there don't seem to be any labeled images at all ... everything should be included in the repository, right?

How much GPU memory required for SegNet basic version

Hi I wanted to know how much GPU memory is required for SegNet basic version as I want to know how to fix the batch size.

Scripts/test_bayesian_segnet.py

when run test_bayesian_segnet.py follow the tutour,make following error
Traceback (most recent call last):
File "Scripts/test_bayesian_segnet.py", line 57, in
out = net.forward_all(data=input_image)
File "/home/caffe/dl/segment/caffe-segnet/python/caffe/pycaffe.py", line 168, in _Net_forward_all
outs = self.forward(blobs=blobs, **batch)
File "/home/caffe/dl/segment/caffe-segnet/python/caffe/pycaffe.py", line 87, in _Net_forward
raise Exception('Input blob arguments do not match net inputs.')
Exception: Input blob arguments do not match net inputs.

someone else run the same problem?

Filter Visualization

Is there a way to visualize the layer filters? similar to http://nbviewer.jupyter.org/github/BVLC/caffe/blob/master/examples/00-classification.ipynb

Which Iteration to use?

Hello, @alexgkendall.
Thanks to your guidance, I successfully ran SegNet with out custom dataset.
Looking into outputs, our network seemed to converge after 110 iterations.

If I understood correctly, I understood that your SegNet also converged around 100 iterations.
However, the tutorial uses iteration 10000 for testing SegNet.

I would like to ask your advice about which iterations I should be using for testing performance.
Thank you!

CPU Training

I don't have a powerful GPU for now.. so, Is it possible to train this model on a CPU instead of GPU ?..

I have already tried and it doesn't work.. Am I missing something ?.. or this model doesn't support CPU training ?

test_segmentation_camvid.py

Hi,

it comes to that line:
scipy.misc.toimage(rgb, cmin=0.0, cmax=255).save(IMAGE_FILE+'_segnet.png')

I get the error: IMAGE_FILE is not defined.

How can I get the names of the image files?
Thank you in advance!

Issue while running test_segmentation_camvid.py

I get the following error while running test_segmentation_camvid.py on segnet trained model.
rgb_gt[:,:,0] = r_gt/255.0
ValueError: could not broadcast input array from shape (4,1,360,480) into shape (360,480)
I dont get this error when i run on segnet basic trained model.

question about convert_mat_to_lmdb.py

Could you please describe how use the "convert_mat_to_lmdb.py" code?

How to input a new sample

Hello,
I have already ran your demo, but I don't know how to get the input image with corresponding ground truth labels. Do you have any suggestion or algorithm available?

bn_param not found

When running :

python Scripts/webcam_demo.py --model Example_Models/driving_web_demo/driving.prototxt --weights Models/Training/segnet_basic_iter_3000.caffemodel --colours Scripts/camvid12.png

Where The weights are the ones trained using the 'segnet_basic_solver', I get the following error. Anything I can change to get it working?

[libprotobuf ERROR google/protobuf/text_format.cc:245] Error parsing text-format caffe.NetParameter: 47:12: Message type "caffe.LayerParameter" has no field named "bn_param".
WARNING: Logging before InitGoogleLogging() is written to STDERR
F0111 13:43:12.378470 11185 upgrade_proto.cpp:928] Check failed: ReadProtoFromTextFile(param_file, param) Failed to parse NetParameter file: Example_Models/driving_web_demo/driving.prototxt
*** Check failure stack trace: ***
Aborted (core dumped)

CUDA Error: GPU out of memory with batch_size = 1.

Hi @alexgkendall , thank you so much for your guide.
I followed the instructions on http://mi.eng.cam.ac.uk/projects/segnet/tutorial.html to train the SegNet. But when I opened up a terminal and ran the commands:
./SegNet/caffe-segnet/build/tools/caffe train -gpu 0 -solver /SegNet/Models/segnet_basic_solver.prototxt
An error occurred:

It seems that the GPU is out of memory(batch_size = 1, so Memory required for data: 410926132).
So I checked the GPU with the command:
nvidia-smi
Result:

My GPU is GT 720 with 1G memory. Though the memory is small, it is much bigger than 245MB + 410MB(data required memory above) = 655MB.

So I would like to ask your advice for this issue :-)
Thank you!

Data Format for convert_mat_to_lmdb.py

Hello @alexgkendall. Thank you for sharing your work to the community!
I have a dataset and the input data is composed of 11 channels.
I thus would like to use your convert_mat_to_lmdb.py (https://github.com/alexgkendall/SegNet-Tutorial/blob/master/Scripts/convert_mat_to_lmdb.py).
But I would like to make sure with data format, e.g., for the label data, should I store them as number of data x row x column?
It would be awesome if you could share us the data/NYU/NYU_train_test_rgbdn_MoosmanNormals.mat (but I assume there might be a dataset copyright problem?)!
Thank you much for your time!

WebDemo Model

Hi @alexgkendall,

I have trained a model using SegNet and Bayesian SegNet as well and I would like to try the WebDemo model as well. I have the segnet_model_driving_webdemo.prototxt that is provided with the SegNet-Tutorial and have downloaded segnet_weights_driving_webdemo.caffemodel from the Model Zoo.

However, I am unable to get it to work correctly. Can I use these two files directly with the test_segmentation_camvid.py script or should I used the compute_bn_statistics.py first to form a set of weights for the model. It is my understanding that the segnet_weights_driving_webdemo.caffemodel is the set of weights that I need to use, however, I am unable to produce results.

Any help would be greatly appreciated.

Thanks.

Out of memory

Hi,

I am trying to work my way through the SegNet tutorial. During the training step, I right away get the error 'error == cudaSuccess (2 vs. 0) out of memory'. I am trying to train on my 4GB GTX980. I am have tried both SegNet and SegNet basic, with batch size as small as just 1. Is there anything else that may be causing this error? Or is my graphics card just insufficient.

Thanks,
Rick

How to handle unlabelled pixels?

I have my own dataset with 2 classes and a lot of pixels don't belong to either of the classes. I have assigned 0 to such pixels in the label image and 1,2 to the classes. I want to train only for labels 1,2 and not 0. Also how do I assign class weight in the softmax layer of the train.prototext file?

question about pixel-wise output

Hi, @alexgkendall ,

The comparison between the output from SegNet webdemo and trained SegNet (follow the tutorial steps) are detailed here:
https://goo.gl/LtNQgA

According to different predicted output from SegNet webdemo and trained SegNet (follow the tutorial steps), what other training procedures for the model in SegNet webdemo are applied ?
Use other training dataset such as ImageNet??
Use pre-trained model such as VGG-16/19??
Other training techniques not mentioned in tutorial??

Best~
Milton

Demo - Change input size

Hello,
I'm using the DEMO, and I'm trying to change my input size to 480x640. I have tryed it in two different ways, and I obtaind some errors that I don't know how to solve. Could somebody help me?

I have changed the input in Scripts/webcam_demo.py: input_shape = (1, 1, 480, 640)
And I obtain this error:
File "Scripts/webcam_demo.py", line 65, in
out = net.forward_all(data=input_image)
File "/imatge/nzoroa/Downloads/caffe-segnet/python/caffe/pycaffe.py", line 168, in _Net_forward_all
outs = self.forward(blobs=blobs, **batch)
File "/imatge/nzoroa/Downloads/caffe-segnet/python/caffe/pycaffe.py", line 93, in Net_forward
self.blobs[in].data[...] = blob
ValueError: could not broadcast input array from shape (1,3,480,640) into shape (1,3,360,480)

Another attemp that I have done is change the input_dim in Example_Models/segnet_model_driving_webdemo.prototxt:
input_dim: 1
input_dim: 3
input_dim: 480
input_dim: 640

But I obtain: F0411 12:07:27.651252 29563 upsample_layer.cpp:63] Check failed: bottom[0]->height() == bottom[1]->height() (23 vs. 30)
*** Check failure stack trace: ***

Thanks in advance!

Feedback to the developers

Hi all,
thank you very much for this great tutorial, i worked it through successfully with one exception at the end:
File "./SegNet/Scripts/test_segmentation_camvid.py", line 77, in
rgb_gt[:,:,1] = g_gt/255.0
ValueError: could not broadcast input array from shape (3,1,360,480) into shape (360,480)

when i comment these 3 lines out in the corresponding py-file the code runs through, but then i get a black picture :-)

I have to say i didn't train the net for the whole 10000 iterations, i stopped after 1000, just to take a first look if it works anyway. I don't think this error has to do with this, or? This seems more like an array conversion problem. Have you heard about this issue before?

One more note from my side:
in your tutorial you have a little clerical error in these two lines:

python /Segnet/Scripts/compute_bn_statistics.py /SegNet/Models/segnet_train.prototxt /SegNet/Models/Training/segnet_iter_10000.caffemodel /Segnet/Models/Inference/ # compute BN statistics for SegNet
python /Segnet/Scripts/compute_bn_statistics.py /SegNet/Models/segnet_basic_train.prototxt /SegNet/Models/Training/segnet_basic_iter_10000.caffemodel /Segnet/Models/Inference/ # compute BN statistics for SegNet-Basic

the marked "n"s should be written in upper case, just for information ;-)

Thank you again, i made a big progress with your tutorial for my own application.

Regards
Stefan

How to Label my own images?

I downloaded the interact labeler from the u-cambridge website, but I can't get it to run. I know it works on Windows, but it shows MSVCR71.dll is missing.

Can you help me on the solution?

about tutorial on Bayesian SegNet

Hi, @alexgkendall
It is so great to see the tutorial on Bayesian SegNet will be coming soon.

According to Bayesian SegNet paper, monte-carlo sampling and variational inference are required. Did you re-implement the those parts such as variational inference or apply other third-party tools?

Best~
Milton

Question | how to Unsupervised Train SegNet?

Hi @alexgkendall

I want to Unsupervised Train SegNet with unlabeled data then fine-tune it with labeled data.

I want to remove the soft_max layer at the end of the network, and make the back-propagate via get the difference in distance between the input data and itself.

How simply make that using the current model of SegNet?

Validation Dataset Unused in Tutorial?

Hello, @alexgkendall! Thank you for sharing the code.
I have looked at the SegNet tutorial and its corresponding model prototxt.
However, I have noticed that the validation dataset (val.txt) is not used.
Could you explain why? Also, is then the training accuracy based on the training dataset?
Thank you for your time! :-)

Calculating Accuracy for SegNet

Hi all,

i trained a couple of SegNet Nets so far, everything works fine testing my models with the test_segmentation.py file that shows me the results visually.
I am wondering how i can calculate the Global, Class an Mean I/U accuracy values like shown in the table of the results in the tutorial? I know the formula for this with the TP, TN, FP and FN meanings, but do i have to write a script by myself or is there anything coming with SegNet i didnt recognize so far?

Thank you!
Stefan

Error in Training

Hi,

I am trying to train a segmentation network with 5 categories. I have prepared my images and labels and modified the path.

The only change I made to the segnet_train.prototext file is: num_output and loss_param.

However, when I start training I got either 'caffe free(): invalide next size' or 'CUBLAS_INTERNAL_ERROR'.

I can run the training command listed on the tutorial without error.

What would be the problem here? Thanks in advance!

Display of output images

Hi

Is it possible to get a specific output images of the 233 test images? For example, if I want to display the image 100th.
And is it possible to click on the output images by a button click? The current situation is that you have to close all 3 output images, so that the loop cycle continues.
Thank you in advance!

Failed to parse NetParameter file: ./SegNet/Models/VGG_ILSVRC_16_layers.caffemodel

Hi,
I download weights of VGG and use the 3rd command ./SegNet/caffe-segnet/build/tools/caffe train -gpu 0 -solver /SegNet/Models/segnet_solver.prototxt -weights /SegNet/Models/VGG_ILSVRC_16_layers.caffemodel # This will begin training SegNet on GPU 0 with a pretrained encoder, but I got this error:

I0531 10:46:46.326437 12574 caffe.cpp:86] Finetuning from ./SegNet/Models/VGG_ILSVRC_16_layers.caffemodel
F0531 10:46:46.480464 12574 upgrade_proto.cpp:935] Check failed: ReadProtoFromBinaryFile(param_file, param) Failed to parse NetParameter file: ./SegNet/Models/VGG_ILSVRC_16_layers.caffemodel
*** Check failure stack trace: ***
@ 0x7fa3a96d9d7d google::LogMessage::Fail()
@ 0x7fa3a96dbc1d google::LogMessage::SendToLog()
@ 0x7fa3a96d9998 google::LogMessage::Flush()
@ 0x7fa3a96dc47e google::LogMessageFatal::~LogMessageFatal()
@ 0x7fa3a9a76a37 caffe::ReadNetParamsFromBinaryFileOrDie()
@ 0x7fa3a9b5f51a caffe::Net<>::CopyTrainedLayersFrom()
@ 0x407e0e CopyLayers()
@ 0x4082d5 train()
@ 0x406178 main
@ 0x7fa3a8bdea40 (unknown)
@ 0x406559 _start
Aborted (core dumped)

about driving.caffemodel fetch

Hi, @alexgkendall
Thanks for your excellent work!
Now I want to download the driving.caffemodel in Example_Models/driving_web_demo/, but find the lfs data quota is exceeded, Is there any alternative way to access the model file?

Here is what I've got after running git lfs fetch/pull:

Git LFS: (0 of 1 files) 0 B / 112.35 MB
This repository is over its data quota. Purchase more data packs to restore access.
Docs: https://help.github.com/articles/purchasing-additional-storage-and-bandwidth-for-a-personal-account/

Thanks.

Error on shared libraries

I am trying to use segnet in CPU only mode:

I tried ./caffe-segnet/build/tools/caffe train --solver=Models/segnet_basic_solver.prototxt

And I got the following error:
./caffe-segnet/build/tools/caffe: error while loading shared libraries: liblmdb.so: cannot open shared object file: No such file or directory

class_weighting

class_weighting: 0.2595
class_weighting: 0.1826
class_weighting: 4.5640
class_weighting: 0.1417
class_weighting: 0.9051
class_weighting: 0.3826
class_weighting: 9.6446
class_weighting: 1.8418
class_weighting: 0.6823
class_weighting: 6.2478
class_weighting: 7.3614

How do I calculate them？Please help me.

why no batch normalization at the end?

Hello there!

I'm trying your (needless to say) awesome, codebase, and i wonder just 2 things :

• Why omitting the batch norm layer at the end in your models? it seems you put batch normalization before each non-linearity but not before the softmax (which is probably the most severe non linearity w.r.t vanishing gradients) ? I'm trying right now this additional batch norm layer right before the SoftMaxWithLoss on your original camvid dataset and it does not seem to worsen the training... even more, i think adding this batch normalization would help working with smaller batch sizes right?
• Is the convolution after (max)-unpooling better than using (unstrided) deconvolution? Is it because convolution is much faster ( equal num of reads but much less write cycles) & similar in the end for all the valid positions?

Thanks a lot again for your time and answers,

Etienne

SegNet and NVIDIA DIGITS

Hi all,

did anybody succesfully trained SegNet with NVIDIA DIGITS?

As far as i understand only the "official" BVLC/caffe is supported from DIGITS but isn't there any way to use SegNet within DIGITS?

Actually the error i get is that when i copy the train net-definition into the custom Network area and try to start training it says "no Name bn_param in caffe.Layerparam". I recognized that there is no bn_param.cpp in the Layer Folder of the official caffe build, what explains this error. In other Forum i read that caffe supports "Batch_norm_param"-layer, but if i replace bn_param with Batch_norm_param another error occurs "no Name shift_filler found".

Is there any Chance to copy the missing (bn_param) layer into the Folders of the official caffe build? i guess ist not that easy, i tried a lot of things but nothing worked so far.

Or is there a possibility to tell DIGITS not to use the "Standard" caffe-build but the "caffe-segnet"-build? I added the caffe-segnet build path to my bashrc, didnt work too...

Any help would be appreciated!

thank you.

Regards
Stefan

Overfitting

Hey all,
I followed the tutorial trying to use Segnet with my own dataset. I only have about 90 labeled images at the moment, although could capture more if necessary. In my dataset, there are only 2 labeled classes. I am running into problems overfitting; accuracy approaches 97% on my training data but is significantly worse during testing. I am currently working to artificially generate more training data by applying transforms to my dataset. Does anyone have other advice or experience reducing overfitting using the segnet architecture?
Thanks,
Rick

Different image sizes

Hello, i am trying to run the provided tutorial. My graphics card doesn't have so much RAM (2GB), so i cant run the tutorial on images of size 360x480. Because of that i reshaped them to size of 120x240, changed the upsampling params in segnet_basic prototxt files, and trained the net using caffe.
I picked a model from the iteration achieving around 90% accuracy, ad the same per class accuracy (most classes dont have that high accuracy but most of them are >= 70%).

After that i tried to run the two provided scripts (changed paths accordingly and in the compute_bn script i also changed in_h and in_w to 120 and 240). But when i run the second script, i get totaly random output (although i ran everything according to the tutorial).

Does this mean that the provided code only works for images of size 360x480 or am i doing something wrong?

When running the test_segmentation_camvid.py script i ran it on the training set also (so changed the data source to the train.txt file), and it gave me the same noisy output

Visualize result as confidence map

Hi @alexgkendall ,

I followed this tutorial (http://nbviewer.ipython.org/github/BVLC/caffe/blob/master/examples/net_surgery.ipynb) and it has a nice visualization of confidence map for classification results.

Can we do the same thing with segnet? If you have time, it would be great if you can show this kind of demo.

Thank so much!

Not getting proper output

Hey,
I am training segnet using 74 images that i have taken randomley from camvid dataset.and i have trainied for 3000 iterations with learning rate 0.01. But when i tried to test it is giving me output as i attached the file. Can someone tell me the reason y i am getting this result?