kanezaki/pytorch-unsupervised-segmentation

Unsupervised Segmentation

This pytorch code generates segmentation labels of an input image.

Asako Kanezaki. Unsupervised Image Segmentation by Backpropagation. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2018. (pdf)

Note: The extended work has been accepted for publication in IEEE TIP! Please see the code.

Requirements

python3, pytorch, opencv2, scikit-image

Getting started

$ python demo.py --input images/101027.jpg

pytorch-unsupervised-segmentation's Issues

How can we specify the segmentation class number?

Dear @kanezaki ,

Thanks for sharing the great work.

It seems the number of classes is determined by SLIC.

pytorch-unsupervised-segmentation/demo.py

Lines 74 to 77 in 8dbde70

	u_labels = np.unique(labels)
	l_inds = []
	for i in range(len(u_labels)):
	l_inds.append( np.where( labels == u_labels[ i ] )[ 0 ] )

How can we manually specify the segmentation class number?
e.g., I only want the method segments the input image into 3 classes.

Best regards,
Jun

a problem about about FCN

Hello,
Thanks for the paper & open sourcing the code very much.
I noticed that in the paper you mentioned "...Note that these components for feature extraction are able to be replaced by alternatives such as fully convolutional networks (FCN) .. ", so I imported the fcn-8s-atonce model structure code, and tried to run it with pretrained parameters and with the default pytorch parameters. But both of the results are bad:

For the former, fcn-8s-atonce model with pretrained parameters, the output segmented graphs are like these:
the original image:

iteration 1:

iteration2:

iteration3:

And the following iterations result are just like iteration3, by the way, I deleted the limitation of the number of the output classes. It seems that overfit happens.

And For the latter one, fcn-8s-atonce model with default pytorch parameters, the output segmented graphs are like these:
the original image:

iteration 1:

iteration2:

iteration3:

iteration 4...14:

I think something wrong happened, and it seems that the backward propagation do not update the parameters effectively.
So I wanna ask that if you have tried with fcn before, and can you give me some advice about it?
Thank you very much !!!

Precision-recall curve

Hi, can you share the code how to draw precision-recall curve?
Thank you so much

Error when executing the demo command

Hi,
I tried to execute the demo command on README and get the below errors:

python demo.py --input images/101027.jpg
Traceback (most recent call last):
  File "demo.py", line 118, in <module>
    print (batch_idx, '/', args.maxIter, ':', nLabels, loss.data[0])
IndexError: invalid index of a 0-dim tensor. Use tensor.item() to convert a 0-dim tensor to a Python number

I'm not familiar with tensor though. Could you please suggest how to modify the code to solve this problem?
My system is Windows 10. Python version 3.7.3. PyTorch==1.2+cpu, opencv-python==4.1.1.26, scikit-image==0.15.0.

Can this segmentation be extended for 3D images?

Hi,

I was wondering if this method can be extended for 3D medical images. If it can be, what do you think could be the major bottlenecks?

Thank you.

Configuration for best average precision score

You mentioned in the paper that the number of convolutional components were varied from 1 to 5. The best "average precision score" was achieved with how many convolutional layers?

deleted

deleted

a bug?

You seems to be using batch normalization (self.bn0) twice.
https://github.com/kanezaki/pytorch-unsupervised-segmentation/blob/master/demo.py#L61
This is not the same code with that written in your paper.
Did you use this code for your experiment?

Need scikit-image as dependency

Hi,
The README file only states opencv2 and PyTorch as dependencies. However, this program also needs scikit-image to work. Perhaps consider to add it to the README file?

Why is the same color getting assigned to different regions of the an image?

[

In this case, the model assigns the same color to shoes' sole and those lights in the background. Does the algorithm not take into account the fact of "objectness"?

The model was run with --compactness 50 --minLabels 4

Inappropriate results

I tried running the code as it is but the result generated was quite bad. Any help would be appreciated

This result was generated with demo image 101027.jpg provided in repo

Training & Saving Model

Hi,
Thanks for the paper & open sourcing the code.
I was planning to use this approach to label a dataset and save the model. In future use the same model with same classes for new images.

How would you suggest to proceed with it , saving the model & classes ?