cr-gjx/scc

Pytorch implementation of "Alleviating Semantics Distortion in Unsupervised Low-Level Image-to-Image Translation via Structure Consistency Constraint." (CVPR 2022).

Alleviating Semantics Distortion in Unsupervised Low-Level Image-to-Image Translation via Structure Consistency Constraint.

Pytorch implementation of "Alleviating Semantics Distortion in Unsupervised Low-Level Image-to-Image Translation via Structure Consistency Constraint." (CVPR 2022).

The SCC loss aims to reduce the geometrical distortion during the image translation.

The translated images are shown as follows:

You can directly insert the codes of SCC into GcGAN, and run the codes following GcGAN scripts.

Specifically, please set

batch_size = 1

scc's Issues

SCC becomes negative

Hello, I also have a similar problem, my SCC is negative during training.I set the batch_size to be 1 and lambda_scc to be 5 following the setting suggested in the paper.

SCC becomes NAN

Hello, I use SCC Loss in my image translation task, but after 1000 iterations, SCC loss becomes NAN. What is the problem

The SCC+CycleGAN model is generating random noise with negative with SCC loss values

Hello,
I have attempted to train the SCC + CycleGAN model on the GTA to Cityscapes dataset which contains +22K images, and after approximately 200 training steps, the model starts generating noise images that bear no resemblance to the input (in both directions of translation). It's worth noting that I'm observing negative SCC loss values, which sometimes explode and reach values as low as -3000. Additionally, I've noticed that the identity losses (both A and B) exhibit high values after a certain number of training steps from the beginning of the training process. Below is the complete configuration of the training process:

batch_size:6
beta1:0.5
bf16:false
continue_train:false
crop_size:256
dataset_mode:"unaligned"
direction:"AtoB"
display_env:"main"
display_freq:400
display_id:1
display_ncols:4
display_port:8,097
display_server:http://localhost/
display_winsize:256
epoch:"latest"
epoch_count:1
gan_mode:"lsgan"
init_gain:0.02
init_type:"normal"
input_nc:3
isTrain:true
lambda_A:10
lambda_B:10
lambda_GAN_A:1
lambda_GAN_B:1
lambda_identity:0.5
lambda_SCC:0.9
load_iter:0
load_size:256
lr:0.0002
lr_decay_iters:50
lr_policy:"linear"
max_dataset_size:∞
model:"scc_cycle_gan"
n_epochs:40
n_epochs_decay:10
n_layers_D:3
name:"gta2city_scc_cycle_train_fp32"
ndf:64
netD:"basic"
netG:"resnet_9blocks"
ngf:64
no_dropout:true
no_flip:false
no_html:false
norm:"instance"
num_threads:4
output_nc:3
phase:"train"
pool_size:50
preprocess:"scale_width"
print_freq:100
save_by_iter:false
save_epoch_freq:5
save_latest_freq:5,000
serial_batches:false
update_html_freq:1,000
use_wandb:true
verbose:false
wandb_project_name:"gta2city"

SCC loss is always negtive and generated picture become very ugly when I use scc loss in CUT.

I try to add scc loss to CUT model in sim-2-real application which aims to translate from GTA5 picture to Citiscapes picture.
I follow your advices in paper with lambda_SCC=0.5 and per_GPU_batchsize=1
but even in the beggining, the SCC loss is negtive and results quickly become ugly( blurring , distortion, only several color exist, e.g red, white and black, and no object can be recoganized)
I have try that train CUT in lambda_SCC=0 for 5 epoch and then add scc loss with lambda_SCC=0.5, the results in first 5 epoch is normal and then quickly become ugly when scc loss added.
Could you please give me some more advice ?
and describe the input format of class SCCLoss?
I only add it as
`
if self.opt.lambda_SCC == 0:

        self.loss_SCC = 0.0

    else:

        self.loss_SCC = self.criterionSCC(self.real_A, self.fake_B) * self.opt.lambda_SCC

`
@CR-Gjx

损失函数为负数

您好，感谢您的代码分享。我和其他几个问题一样，也遇到了损失函数为负值的情况，这种情况需要调大cyclegan的系数，减少scc的系数吗？还有一个问题，使用scc损失的时候，batchsize必须为1吗，期待您的回答。

SCC becomes negative

Hello, I incorporate this loss into the CycleGan modified in the file around those lines Link and I found the negative value appears. Is this normal using SCC loss?

about the scc loss

hi, I just finished reading your paper,

and now I am trying to use your cyclegan + scc loss code on github,

and after setting the batch size to 1, it seems like it is working fine. (well, at least it is generating images..)

but, just like other issues uploaded before, I am getting negative scc loss values on training.
like SCC : -2.5 when lambda_scc is 5 and etc.

I already tried other lambda_scc values (all positive values, 0.5 included)
and the results are same.

is it normal to get negative values for scc loss?
(it seems like it is decreasing over time in the negative series.)

@CR-Gjx