jxhuang0508/cvrn

Cross-View Regularization for Domain Adaptive Panoptic Segmentation

cvrn's Issues

CUDA out of memory

Thanks for your great job！I followed the instrution to run the instance segmentation branch with one GPU ：TITAN XP(12GB), but it soon got out of memory, can you guide me what's the matter with it？Thanks in advance.

2022-02-06 13:35:05,877 | optimizer.py | line 59 : {'lr': 1, 'momentum': 0.9, 'dampening': 0, 'weight_decay': 0.0005, 'nesterov': False}
loading annotations into memory...
Done (t=21.05s)
creating index...
index created!
loading annotations into memory...
Done (t=11.43s)
creating index...
index created!
Filtered roidb entries: 5950
len_source_loader: 16556
len_target_loader: 5950
2022-02-06 13:37:34,389 | callback.py | line 40 : Batch [20] Speed: 0.32 samples/sec Train-rpn_cls_loss=0.613139, rpn_bbox_loss=0.335659, rcnn_accuracy=0.761426, cls_loss=0.779723, bbox_loss=0.249559, mask_loss=0.750133,
2022-02-06 13:37:36,418 | callback.py | line 40 : Batch [20] Speed: 9.86 samples/sec Train-rcnn_sida_loss=0.009439, mask_sida_loss=0.097951,
Traceback (most recent call last):
File "upsnet/train_seed1234_co_training_cross_style.py", line 447, in
upsnet_train()
File "upsnet/train_seed1234_co_training_cross_style.py", line 360, in upsnet_train
output = train_model(*batch_target)
File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "upsnet/../lib/utils/data_parallel.py", line 110, in forward
return self.module(*inputs[0], **kwargs[0])
File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "upsnet/../upsnet/models/resnet_upsnet_multiscale_branch_lr10.py", line 300, in forward
mask_score_sc = self.mask_branch([fpn_p2_sc, fpn_p3_sc, fpn_p4_sc, fpn_p5_sc], rois_sc)
File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "upsnet/../upsnet/models/rcnn.py", line 85, in forward
mask_deconv1 = self.mask_deconv1(mask_conv4)
File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/container.py", line 91, in forward
input = module(input)
File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 691, in forward
output_padding, self.groups, self.dilation)
RuntimeError: CUDA error: out of memory

The implement in pfpn and upsnet

Hi, Could you share the code training in pfpn and upsnet too?

Use SYNTHIA dataset

Hello, this CVRN is a very interesing work. I am wondering how do you use SYHTHIA dataset in the experiments. Could you share like which verision of SYNTHIA dataset do you use and the format and some prepocessing steps you used in the experiments? Thanks in advance!

Why k+1 and not k?

CVRN/data loader and processing/init_vistas2cityscapes_format.py

Line 61 in 570743d

label_copy[label == k+1] = v

About Inter-style regularization and the instance pseudo label generation

Hi, I'm interested in the ISR module, but I didn't find the code of 'generating the pseudo labels by an inter-style pseudo-label
unification function' and either the 'Inter-style regularization loss'. Besides, I didn't find the code of instance pseudo label generation too, can you give me some guidance?

Mapillary labels into Cityscapes label conversion

Mapillary labels into Cityscapes label conversion

Dear author, thank you for providing the code!

The following script converts Mapillary labels into Cityscapes label space: CVRN/data loader and processing/init_vistas2cityscapes_format.py.

I have a question here. This code takes the semantic ground truth label files (in PNG format) as inputs. The original label PNG files of Maipillary are three-channel color PNG images. By looking at your code :

CVRN/data loader and processing/init_vistas2cityscapes_format.py

Line 20 in cea4fec

LABEL_dirs = os.path.join(dataset_path, 'annotations', 'panoptic_{}_semantic_trainid'.format(set))

and

CVRN/data loader and processing/init_vistas2cityscapes_format.py

Lines 58 to 61 in cea4fec

	label = cv2.imread(os.path.join(LABEL_dirs,image_id + '.png'),-1)
	label_copy = label.copy()
	for k, v in id_to_trainid.items():
	label_copy[label == k+1] = v

, it seems we need a one-channel PNG semantic label file as input instead of three-channel color semantic labels.
Could you please let me know how you converted the Mapillary color semantic label images to single-channel label PNG files?
Is there any RGB to id encoding scheme you are using for that?

training code

Hi, thanks for the great work!
Will the training code be released soon?

Mappilary data prepration

Hi, Thank ​you for releasing the code.
I am trying to reproduce your results on the Mapillary dataset.
You provide script to convert 19 class cityscape into 16 class version in cvrn/init_citiscapes_19cls_to_16cls.py.
Can this script also be used to preprocess mapillary dataset as well?
What do I need to be able to evaluate your model to get the number shown in your paper?
I see there is dataloader to mapillary dataset, but that doesn't seem to include conversion to 16 class dataset.

Can you explain the steps required?

Many thanks
Gurkirt

ModuleNotFoundError: No module named 'upsnet.operators.functions.entropy_loss'

Hi, when I run your code, I encounter the following error:

Traceback (most recent call last):
File "upsnet/upsnet_end2end_train.py", line 61, in
from upsnet.models import *
File "upsnet/../upsnet/models/init.py", line 2, in
from .resnet_upsnet_multiscale_branch_lr10 import resnet_101_upsnet_multiscale_branch_lr10, resnet_50_upsnet_multiscale_branch_lr10
File "upsnet/../upsnet/models/resnet_upsnet_multiscale_branch_lr10.py", line 20, in
from upsnet.operators.functions.entropy_loss import prob_2_entropy, sigmoid_2_entropy, entropy_loss
ModuleNotFoundError: No module named 'upsnet.operators.functions.entropy_loss'

It seems that there is no a module called entropy_loss under the upsnet.operators.functions. I cannot find this module under your repo or original UPSNet repo. Could you help me solve it? Thanks in advance!