wasidennis/adaptsegnet

Learning to Adapt Structured Output Space for Semantic Segmentation, CVPR 2018 (spotlight)

Learning to Adapt Structured Output Space for Semantic Segmentation

Pytorch implementation of our method for adapting semantic segmentation from the synthetic dataset (source domain) to the real dataset (target domain). Based on this implementation, our result is ranked 3rd in the VisDA Challenge.

Contact: Yi-Hsuan Tsai (wasidennis at gmail dot com) and Wei-Chih Hung (whung8 at ucmerced dot edu)

Paper

Learning to Adapt Structured Output Space for Semantic Segmentation
Yi-Hsuan Tsai*, Wei-Chih Hung*, Samuel Schulter, Kihyuk Sohn, Ming-Hsuan Yang and Manmohan Chandraker
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018 (spotlight) (* indicates equal contribution).

Please cite our paper if you find it useful for your research.

@inproceedings{Tsai_adaptseg_2018,
  author = {Y.-H. Tsai and W.-C. Hung and S. Schulter and K. Sohn and M.-H. Yang and M. Chandraker},
  booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  title = {Learning to Adapt Structured Output Space for Semantic Segmentation},
  year = {2018}
}

Example Results

Quantitative Reuslts

Installation

• Install PyTorch from http://pytorch.org with Python 2 and CUDA 8.0

• NEW Add the LS-GAN objective to improve the performance

  • Usage: add --gan LS option during training (see below for more details)

• PyTorch 0.4 with Python 3 and CUDA 8.0

  • Usage: replace the training and evaluation codes with the ones in the pytorch_0.4 folder
  • Update: tensorboard is provided by adding --tensorboard in the command
  • Note: the single-level model works as expected, while the multi-level model requires smaller weights, e.g., --lambda-adv-target1 0.00005 --lambda-adv-target2 0.0005. We will investigate this issue soon.

• Clone this repo

git clone https://github.com/wasidennis/AdaptSegNet
cd AdaptSegNet

Dataset

• Download the GTA5 Dataset as the source domain, and put it in the data/GTA5 folder

• Download the Cityscapes Dataset as the target domain, and put it in the data/Cityscapes folder

Pre-trained Models

• Please find our-pretrained models using ResNet-101 on three benchmark settings here

• They include baselines (without adaptation and with feature adaptation) and our models (single-level and multi-level)

Testing

• NEW Update results using LS-GAN and using Synscapes as the source domain

  • Performance: check the appendix of the updated arXiv paper (updated on 10/17/2019)
  • Pre-trained models

• Download the pre-trained multi-level GTA5-to-Cityscapes model and put it in the model folder

• Test the model and results will be saved in the result folder

python evaluate_cityscapes.py --restore-from ./model/GTA2Cityscapes_multi-ed35151c.pth

• Or, test the VGG-16 based model Model Link

python evaluate_cityscapes.py --model DeeplabVGG --restore-from ./model/GTA2Cityscapes_vgg-ac4ac9f6.pth

• Compute the IoU on Cityscapes (thanks to the code from VisDA Challenge)

python compute_iou.py ./data/Cityscapes/data/gtFine/val result/cityscapes

Training Examples

• NEW Train the GTA5-to-Cityscapes model (single-level with LS-GAN)

python train_gta2cityscapes_multi.py --snapshot-dir ./snapshots/GTA2Cityscapes_single_lsgan \
                                     --lambda-seg 0.0 \
                                     --lambda-adv-target1 0.0 --lambda-adv-target2 0.01 \
                                     --gan LS

• Train the GTA5-to-Cityscapes model (multi-level)

python train_gta2cityscapes_multi.py --snapshot-dir ./snapshots/GTA2Cityscapes_multi \
                                     --lambda-seg 0.1 \
                                     --lambda-adv-target1 0.0002 --lambda-adv-target2 0.001

• Train the GTA5-to-Cityscapes model (single-level)

python train_gta2cityscapes_multi.py --snapshot-dir ./snapshots/GTA2Cityscapes_single \
                                     --lambda-seg 0.0 \
                                     --lambda-adv-target1 0.0 --lambda-adv-target2 0.001

Related Implementation and Dataset

• Y.-H. Tsai, K. Sohn, S. Schulter, and M. Chandraker. Domain Adaptation for Structured Output via Discriminative Patch Representations. In ICCV, 2019. (Oral) [paper] [project] [Implementation Guidance]
• W.-C. Hung, Y.-H Tsai, Y.-T. Liou, Y.-Y. Lin, and M.-H. Yang. Adversarial Learning for Semi-supervised Semantic Segmentation. In BMVC, 2018. [paper] [code]
• Y.-H. Chen, W.-Y. Chen, Y.-T. Chen, B.-C. Tsai, Y.-C. Frank Wang, and M. Sun. No More Discrimination: Cross City Adaptation of Road Scene Segmenters. In ICCV 2017. [paper] [project]

Acknowledgment

This code is heavily borrowed from Pytorch-Deeplab.

Note

The model and code are available for non-commercial research purposes only.

• 10/2019: update performance and training/evaluation codes for using LS-GAN and Synscapes (especially thanks to Yan-Ting Liu for helping experiments)
• 01/2019: upate the training code for PyTorch 0.4
• 07/23/2018: update evaluation code for PyTorch 0.4
• 06/04/2018: update pretrained VGG-16 model
• 02/2018: code released

adaptsegnet's Issues

some questions about the classifier model of deeplab

the return statement in forward function of Classifier_Module looks like misaligned.

AdaptSegNet/model/deeplab_multi.py

Lines 114 to 118 in e47871b

	def forward(self, x):
	out = self.conv2d_list[0](x)
	for i in range(len(self.conv2d_list) - 1):
	out += self.conv2d_list[i + 1](x)
	return out

AdaptSegNet/model/deeplab_vgg.py

Lines 17 to 21 in e47871b

	def forward(self, x):
	out = self.conv2d_list[0](x)
	for i in range(len(self.conv2d_list)-1):
	out += self.conv2d_list[i+1](x)
	return out

Not enough memory. Buy a new RAM!!!!!

My RAM is 8G. When I tried to train on my cpu, it says that my computer doesn't have enough memory........
How to solve this problem? Can anyone help me??Thanks!

Res_Deeplab

hello,
I wonder in the file train_gta2cityscapes_multi.py line 18:

from model.deeplab_multi import Res_Deeplab 

is the same as

from model.deeplab_multi import DeeplabMulti 

since I can't find the Res_Deeplab in that file.
Thanks.

Question on model.deeplab_multi.py

Hi,

am fascinated by your idea training the segmentation network and trying to reproduce the results. I am studying the deeplab_multi model structure and am a bit confused on the Classifer_Module, specifically on function Classifier.forward() on line 114.

As I understand from the paper that this is the ASPP layer that tries to summarize information at different resolution by convoluting at 4 different rates ([6, 12, 18, 24], 4 feature maps with shape (b, c, w, h)) and then sum the result together (become 1 feature map with shape (b, c, w, h)). In the actual implementation, I notice that Classifier.forward() is returned inside the for loop (line 118), which I believe will cause the module to return only the sum of first 2 rates ([6, 12]) instead of 4.

Was that intentional? or did I misunderstand any of the concepts?

Appreciate your answer. Would help me a great deal understanding your work.

Thanks.

Couldn't reproduce the result reported in the paper

Hi, thanks for the code! I am trying to reproduce your result using your code, but I only get a mIoU of 40 after 45000 iterations. The result get even worse after 65000 iterations. I am wondering if it is very sensitive to how many iterations to train? Also is there other things to mind using your code?

Where is the pretrained model of vgg-16 from?

I want to know how to get the pretrained model you use for training. Is it trained on ImageNet by pytorch official? Or is downloaded from here: http://www.robots.ox.ac.uk/~vgg/software/very_deep/caffe/VGG_ILSVRC_16_layers.caffemodel?

Output size of the discriminator

Hi Authors,
I was just curious to know what is the size of the output from the discriminator that you used in your experiments. Is it the same as the size of the input (as you mention in the paper). Or is it downsampled? - As the upsampling code was commented out in the file "AdaptSegNet/model/discriminator.py".
In either case, if possible please share some insights into how this could affect the adaptation.

Thanks!

Error when training using VGG-16 model

@wasidennis @m3phisto @hfslyc , i am trying to run the code (VGG source only). But when i run as you mentioned in previous thread, with --lambda-adv-target1 0 --lambda-adv-target2 i got this error: "train_gta2cityscapes_multi.py", line 311, in main
pred1, pred2 = model(images)
ValueError: not enough values to unpack (expected 2, got 1)`

Are there any modifications that have to be done for training source only (without) adaptation experiment?
If yes, i would be pleased if you could share the code.

Is the network really well trained?

I try to run the code with pytorch0.4.0, and the result of resnet101-multi-level is reproduced, but it is very unstable. Only at iteration 105000, the result in paper (42.4%) is reproduced.

iteration 5000      0.3098
iteration 10000     0.3488
iteration 15000     0.3575
iteration 20000     0.3531
iteration 25000     0.3715
iteration 30000     0.3851
iteration 35000     0.4022
iteration 40000     0.3675
iteration 45000     0.3738
iteration 50000     0.3930
iteration 55000     0.3799
iteration 60000     0.3602
iteration 65000     0.3745
iteration 70000     0.3816
iteration 75000     0.3752
iteration 80000     0.3582
iteration 85000     0.4025
iteration 90000     0.3873
iteration 95000     0.3761
iteration 100000    0.3729
iteration 105000    0.4246
iteration 110000    0.3850
iteration 115000    0.4084
iteration 120000    0.3983

I plot the loss by tensorboard (smoothed) and it looks strange.

@wasidennis
Is this the case when you trained the model for the CVPR paper?

What is the setting of the environment? cuda runtime error (2) : out of memory

My environment settings are with pytorch0.2.0, and CUDA Version 9.0.176. TITAN12g, python2.7, but I encounter the Cuda runtime error. @wgchang, @wasidennis can I ask how do you resolve the problem of runtime error?

Bug on deeplab classifier model

class Classifier_Module(nn.Module):
    def __init__(self, inplanes, dilation_series, padding_series, num_classes):
        super(Classifier_Module, self).__init__()
        self.conv2d_list = nn.ModuleList()
        for dilation, padding in zip(dilation_series, padding_series):
            self.conv2d_list.append(
                nn.Conv2d(inplanes, num_classes, kernel_size=3, stride=1, padding=padding, dilation=dilation, bias=True))

        for m in self.conv2d_list:
            m.weight.data.normal_(0, 0.01)

    def forward(self, x):
        out = self.conv2d_list[0](x)
        for i in range(len(self.conv2d_list) - 1):
            out += self.conv2d_list[i + 1](x)
            return out

Why is the “return out " statement on the for loop?
In my understanding, out is the sum of the pyramid layer , so it should be outside of the for loop.

License

Hi Dear,
Could you tell me what is the license of this software, please?
According to GitHub's Policy, all repositories without an explicit license are considered Copyrighted materials. Do the authors intend to make this software free?
Thank you!

Can not reproduce the result of VGG net

I directly dowloaded this repo and trained the model. The results of Res101-single-level and Res101-multi-level are reproduced, even that the training is not stable. However I can not reproduce the result of VGG16-single-level.

I trained it with the same hyper-parameters of Res101-single-level, and the best result is 32.56%. But the result in the paper is 35.00%

Do I need to use another group of hyper-parameters?

Change Class Number

I tried to learn with my own data set and tried to change the number of classes, but the following error occurred.
The number of classes of train_gta2cityscapes_multi.py and deeplab_multi.py has already been changed.
class number : From 19 to 1

RuntimeError: Error(s) in loading state_dict for ResNetMulti:
While copying the parameter named "layer5.conv2d_list.0.weight", whose dimensions in the model are torch.Size([1, 1024, 3, 3]) and whose dimensions in the checkpoint are torch.Size([21, 2048, 3, 3]).
While copying the parameter named "layer5.conv2d_list.0.bias", whose dimensions in the model are torch.Size([1]) and whose dimensions in the checkpoint are torch.Size([21]).
While copying the parameter named "layer5.conv2d_list.1.weight", whose dimensions in the model are torch.Size([1, 1024, 3, 3]) and whose dimensions in the checkpoint are torch.Size([21, 2048, 3, 3]).
While copying the parameter named "layer5.conv2d_list.1.bias", whose dimensions in the model are torch.Size([1]) and whose dimensions in the checkpoint are torch.Size([21]).
While copying the parameter named "layer5.conv2d_list.2.weight", whose dimensions in the model are torch.Size([1, 1024, 3, 3]) and whose dimensions in the checkpoint are torch.Size([21, 2048, 3, 3]).
While copying the parameter named "layer5.conv2d_list.2.bias", whose dimensions in the model are torch.Size([1]) and whose dimensions in the checkpoint are torch.Size([21]).
While copying the parameter named "layer5.conv2d_list.3.weight", whose dimensions in the model are torch.Size([1, 1024, 3, 3]) and whose dimensions in the checkpoint are torch.Size([21, 2048, 3, 3]).
While copying the parameter named "layer5.conv2d_list.3.bias", whose dimensions in the model are torch.Size([1]) and whose dimensions in the checkpoint are torch.Size([21]).

About miou on source domain

Hi, your work is amazing. I have a question that will miou on the source domain images be reduced after adaption using this method? Thank you very much!

Training settings when training source only models

I have trained DeepLab ResNet101 on GTA5，tested on Cityscapes, and got mIOU of 0.3 which is lower than 36.3. I use SGD optimizer with lr 1.0e-5, and poly lr scheduler with max_iter=100K. Can some share the correct training settings?

RuntimeError: cuda runtime error (2) : out of memory

Hello,Thanks for your shared code.when I train the GTA5-to-Cityscapes model (multi-level) on a GTX 1070 GPU with 8GB memory, there is a problem as follow:

UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
"See the documentation of nn.Upsample for details.".format(mode))
THCudaCheck FAIL file=/opt/conda/conda-bld/pytorch_1525909934016/work/aten/src/THC/generic/THCStorage.cu line=58 error=2 : out of memory
Traceback (most recent call last):
File "train_gta2cityscapes_multi.py", line 413, in
main()
File "train_gta2cityscapes_multi.py", line 293, in main
loss_seg1 = loss_calc(pred1, labels, args.gpu)
File "train_gta2cityscapes_multi.py", line 145, in loss_calc
return criterion(pred, label)
File "/home/student/anaconda2/lib/python2.7/site-packages/torch/nn/modules/module.py", line 491, in call
result = self.forward(*input, **kwargs)
File "/home/student/AdaptSegNet-master/utils/loss.py", line 34, in forward
predict = predict[target_mask.view(n, h, w, 1).repeat(1, 1, 1, c)].view(-1, c)
RuntimeError: cuda runtime error (2) : out of memory at /opt/conda/conda-bld/pytorch_1525909934016/work/aten/src/THC/generic/THCStorage.cu:58

I tried to use about 2,000 images in total and resized them to 512*216,but the problem still exists.How to solve it?Change the code?
Looking forward to your answer.

Why are the dimensions of predict and target different in cross entropy?

Hello! Your work is amazing! I have a question with the loss.py. I noticed that the cross entropy loss for semantic segmentation is implemented in the following code:
loss = F.cross_entropy(predict, target, weight=weight, size_average=self.size_average)
In this code, the dimension of predict is [h×w, c] and the dimension of target is [h×w]. I remember that when calculating the cross entropy loss, target and prediction should have the same dimensions.
Maybe my problem is very stupid. This problem really bothers me.
Thank you very much!!

Train the model on other dataset.

Hello！ I have trained your model successfully recently on GTA5 and Cityscapes. And I want to try it on other dataset. I am a newbie in this domain and I met a problem. The new dataset labels are black-and-white images. I found that there maybe some problems in labels. So I want to know that what does "label2train" mean in json file? I really can not understand these numbers([0, 255],
[1, 255],
[2, 255],
[3, 255],
[4, 255],
[5, 255],
[6, 255],
[7, 0],
[8, 1],
[9, 255],
[10, 255],
[11, 2],
[12, 3],
[13, 4],
[14, 255],
[15, 255],
[16, 255],
[17, 5],
[18, 255],
[19, 6],
[20, 7],
[21, 8],
[22, 9],
[23, 10],
[24, 11],
[25, 12],
[26, 13],
[27, 14],
[28, 15],
[29, 255],
[30, 255],
[31, 16],
[32, 17],
[33, 18],
[-1, 255]).

And some of them is applied in the gta5_dataset.py in init. I really can not understand these numbers.
Maybe my problem is very stupid. I am really a newbie just want to learn more.
Thank you very much!!

Discriminator of the feature-level adaptation

Hello, thanks for your work! In your paper 'Learning to Adapt Structured Output Space for Semantic Segmentation', you list the comparison between the adaptation in the feature space and output space. But I only found 'discriminator.py' in the directory 'model', which is obviously for the output space adaptation. So, could you please tell me the concrete architecture of discriminator of adaptation in the feature space? Thanks!

PyTorch 0.4

Just wondering if the PyTorch 0.4 version is available yet.

Deeplab-vgg16 pretrained pytorch model

Hi authors, could you please offer a pretrained Deeplab-vgg16 model in pytorch?
Thank you very much!

How to reproduce SYNTHIA-to-CityScapes

Hi, thank you code and paper :)
I am curious about SYNTHIA-to-CityScapes,
I have used GTA5-to-CityScapes get a better result,
However, I want to reproduce SYNTHIA-to-CityScapes.
I noticed that you follow the [3] setting which use 13 classes to evaluate.
Unfortunately [3] didn't release code,
So could you provide your SYNTHIA-to-CityScapes experiments materials(dataloader, evaluated)?

[3] Y.-H. Chen, W.-Y. Chen, Y.-T. Chen, B.-C. Tsai, Y.-C. Frank Wang, and M. Sun. No More Discrimination: Cross City Adaptation of Road Scene Segmenters. In ICCV 2017.

How is the trend of loss changing?

I train my data using this method. The loss_adv1 is increased. loss_seg1 and loss_D1 is decreased. In this situation, should I make LAMBDA_ADV_TARGET1 larger ?

VGG-16 is FCN8s or deeplab-largeFOV?

Symbols of arguments confused.Source_label=0? and target_label=1?

1. I found "train_gta2cityscapes_multi.py" has some mistakes in symbols of arguments,such as the confusion between iter_size and iter-size

2.Besides,in line256,257 of "train_gta2cityscapes_multi.py" : source_label = 0 target_label = 1
Isn't it on the contrary,that's said,source_label = 1 target_label = 0

I have a question about the loss_adv in your code ,can you spend a little time to help me ?

In your paper Learning to Adapt Structured Output Space for Semantic Segmentation loss_adv is designed to train the segmentation network and fool the discriminator by maximizing the probability of the target prediction being considered as the source prediction, you mean we should maximize the loss_adv for training segmentation network,right?if then, how about you maximizing the loss_adv in your code?
`pred_target1, pred_target2 = model(images)
pred_target1 = interp_target(pred_target1)
pred_target2 = interp_target(pred_target2)

D_out1 = model_D1(F.softmax(pred_target1))
D_out2 = model_D2(F.softmax(pred_target2))

loss_adv_target1 = bce_loss(D_out1,Variable(torch.FloatTensor(D_out1.data.size()).fill_(source_label)).cuda(args.gpu))

loss_adv_target2 = bce_loss(D_out2,Variable(torch.FloatTensor(D_out2.data.size()).fill_(source_label)).cuda(args.gpu))
loss = args.lambda_adv_target1 * loss_adv_target1 + args.lambda_adv_target2 * loss_adv_target2
loss = loss / args.iter_size
loss.backward()`
these codes is about loss_adv in you project.
thank you very much!

why stopping trainning at num-steps-stop(120000) ?

I see you set two iteration controller(num-step&num-steps-stop) in your code. Could you explain why you finish trainning at iteration num-steps-stop rather than iteration num-steps.

Baseline (sourceonly ResNet) miou is 36.6

The sourceonly resnet-101 performance is quite higher than mine, which is only ~27.

I have noticed that the parameters in batch_norm layer are not trained.
Do we need to update the batch statistics during training or just use the statistics load from the pre-trained resnet?

Could you provide more details based on resnet-101 model? @wasidennis

About fully-supervised (oracle) model results

Why the oracle resutls of ResNet101 in Table 2 and Table 5 in the paper are different? One is 65.1, and the other is 71.7. In my understanding, both of these results are trained on Cityscapes dataset using the full-supervised model, which should be the same. Thank you very much!

Any update info on the migration to pytorch 0.4 version

Hi, @wasidennis ,

Any update info on the migration to pytorch 0.4 version?

According to the code, batch-size is 1 for trainning?

About baseline result with res101-deeplabv2

Hi @wasidennis, I use your provided res101 backbone and train deeplab v2 on GTA as your code, Then I evaluate the model on cityscapes, but I only got 31.1 which is far from your baseline result of 36.6.
Did I miss anything? Looking forward to your reply.

RuntimeError: cuda runtime error (2) : out of memory

Hello. I tried to execute your code, but it gives out of memory error.
I am working with NVIDIA Titan Xp, which has 12GB memory capacity.
It seems that the input sizes of the source and target images are quite big. ((1280,720) and (1024,512))

Can I ask which GPU devices you use for training?

RuntimeError: there are no graph nodes that require computing gradients

Hi, Thank you for sharing your code. I met a problem which is as follows:

**Traceback (most recent call last):

File "train_gta2cityscapes_multi.py", line 412, in
main()
File "train_gta2cityscapes_multi.py", line 304, in main
loss.backward()
File "/home/yaxing/anaconda2/envs/pytorch/lib/python2.7/site-packages/torch/autograd/variable.py", line 156, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph, retain_variables)
File "/home/yaxing/anaconda2/envs/pytorch/lib/python2.7/site-packages/torch/autograd/init.py", line 98, in backward
variables, grad_variables, retain_graph)
RuntimeError: there are no graph nodes that require computing gradients**

Note I never change the code.
During training, the error happens instead of the beginning. There are three times(iter:25281/25000, 8500/25000, 600/25000 ).
The pytorch is 0.2.0_4 with python2.7.

please forgive me to bother you

Mutli gpu training

Hi currently training on GTA2Cityscapes takes 2 days for 100k epochs which is very slow.
How can I make this run in multi gpu?

Will ICLR'19-Disentangled Patch code be released?

Hi,
I noticed a paper "Domain Adaptation for Structured Output via Disentangled Patch Representations" which submit to ICLR'19,
After reading this paper, I found out that you are the first author.

I think that using patch-level alignment is interesting,
However, I am curious about the module H implement detail,
Will you release your code recently?

why stopping trainning at num-steps-stop ?

how is this deeplabv2 weight obtained?

it seems that you use this weight:

http://vllab.ucmerced.edu/ytsai/CVPR18/DeepLab_resnet_pretrained_init-f81d91e8.pth

Is this weight the fine-tuned result on PascalVOC Dataset?

Training of custom dataset

Are there guidelines on running the network on custom dataset ?

Error on pytorch 0.4

I am trying to run the code with pytorch 0.4 but got the following error. How can I solve the problem? Thanks in advance.

Downloading: "http://vllab.ucmerced.edu/ytsai/CVPR18/DeepLab_resnet_pretrained_init-f81d91e8.pth" to /home/redhat/.torch/models/DeepLab_resnet_pretrained_init-f81d91e8.pth
Traceback (most recent call last):
File "train_gta2cityscapes_multi.py", line 422, in
main()
File "train_gta2cityscapes_multi.py", line 178, in main
saved_state_dict = model_zoo.load_url(args.restore_from)
File "/home/redhat/anaconda3/lib/python3.6/site-packages/torch/utils/model_zoo.py", line 65, in load_url
_download_url_to_file(url, cached_file, hash_prefix, progress=progress)
File "/home/redhat/anaconda3/lib/python3.6/site-packages/torch/utils/model_zoo.py", line 72, in _download_url_to_file
file_size = int(u.headers["Content-Length"])
File "/home/redhat/anaconda3/lib/python3.6/site-packages/requests/structures.py", line 54, in getitem
return self._store[key.lower()][1]
KeyError: 'content-length'

Cityscapes Test result

Does your Cityscapes test dataset have labels?I have downloaded a label without a test data set from the official website.So I can't get the test mean IOU. Thank you!

the size

Sorry to trouble,
I sufferred from such an issue,
when I try to run the code (multi or single), I modify the input_size = '1280,720' to input_size = '1280,512', while the in fact images size of GTA is 1914,1052, otherwise, this error may happen:

THCudaCheck FAIL file=/pytorch/aten/src/THC/THCTensorRandom.cu line=25 error=2 : out of memory
Traceback (most recent call last):
  File "train_gta2cityscapes_multi.py", line 411, in <module>
    main()
  File "train_gta2cityscapes_multi.py", line 201, in main
    model.cuda(args.gpu)
  File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py", line 249, in cuda
    return self._apply(lambda t: t.cuda(device))
  File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py", line 176, in _apply
    module._apply(fn)
  File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py", line 182, in _apply
    param.data = fn(param.data)
  File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py", line 249, in <lambda>
    return self._apply(lambda t: t.cuda(device))
RuntimeError: cuda runtime error (2) : out of memory at /pytorch/aten/src/THC/THCTensorRandom.cu:25

Also, I modified the loss_adv_target1.data.cpu().numpy()[0] to loss_adv_target1.data.cpu().numpy(), since it showed too many indices

And after I modified the input_size as '1280,512' and finished training, and performed the evaluation:
python evaluate_cityscapes.py --restore-from ./snapshots/GTA2Cityscapes_single/GTA5_250000.pth
this issue happened:

THCudaCheck FAIL file=/pytorch/aten/src/THC/THCTensorRandom.cu line=25 error=2 : out of memoryTraceback (most recent call last):
  File "evaluate_cityscapes.py", line 134, in <module>
    main()
  File "evaluate_cityscapes.py", line 100, in main
    saved_state_dict = torch.load(args.restore_from)
  File "/usr/local/lib/python2.7/dist-packages/torch/serialization.py", line 303, in load
    return _load(f, map_location, pickle_module)
  File "/usr/local/lib/python2.7/dist-packages/torch/serialization.py", line 469, in _load
    result = unpickler.load()
  File "/usr/local/lib/python2.7/dist-packages/torch/serialization.py", line 437, in persistent_load
    data_type(size), location)
  File "/usr/local/lib/python2.7/dist-packages/torch/serialization.py", line 88, in default_restore_location
    result = fn(storage, location)
  File "/usr/local/lib/python2.7/dist-packages/torch/serialization.py", line 70, in _cuda_deserialize
    return obj.cuda(device)
  File "/usr/local/lib/python2.7/dist-packages/torch/_utils.py", line 68, in _cuda
    with torch.cuda.device(device):
  File "/usr/local/lib/python2.7/dist-packages/torch/cuda/__init__.py", line 228, in __enter__
    _lazy_init()
  File "/usr/local/lib/python2.7/dist-packages/torch/cuda/__init__.py", line 161, in _lazy_init
    torch._C._cuda_init()
RuntimeError: cuda runtime error (2) : out of memory at /pytorch/aten/src/THC/THCTensorRandom.cu:25

So, I wonder whether my input_size setting is wrong.
Thanks

Learning rate adjustment

AdaptSegNet/train_gta2cityscapes_multi.py

Line 273 in edda594

adjust_learning_rate_D(optimizer_D1, i_iter)

Do we need learning rate adjustment for Adam optimizer?

License

Thank you, @wasidennis! Could you add a license in your repo that better reflects your desire to make this software free only for academic purpose, please?
Best,

VGG16-based model

Any training details for the baseline result?

Hi, I am wondering how you trained the baseline models. Could you provide more details regarding this like the batch-size, lr etc.? Thank you very much.

wrong order

AdaptSegNet/train_gta2cityscapes_multi.py

Line 173 in edda594

h, w = map(int, args.input_size.split(','))

w, h instead of h, w

Difference in training scheme?

The optimization problem for G given in paper is max_D min_G (L) in eq.6. In actual code however, you only perform min_G(L) and not the max_D. Will it make a difference in the final output?

Evaluation of the model

Hello,

I have evaluate the model GTA2Cityscapes_multi-ed35151c.pth provided in your link. But, I find the result of it is a bit different from your paper. So I wonder it's just part of the checkpoints. Thanks.

items	GTA2Cityscapes_multi-ed35151c.pth	Paper
road:	85.11	86.5
sidewalk:	36.14	36.0
building:	79.0	79..9
wall:	28.6	23.4
fence:	22.03	23.3
pole:	18.89	23.9
light:	31.74	35.2
sign:	12.54	14.8
vegetation:	81.37	83.4
terrain:	36.4	33.3
sky:	72.56	75.6
person:	52.15	58.5
rider:	24.81	27.6
car:	69.34	73.7
truck:	33.84	32.5
bus:	34.99	35.4
train:	3.27	3.9
motocycle:	26.02	30.1
bicycle:	26.43	28.1
mIoU:	40.8	42.4