zhendong-wang/diffusion-gan

Official PyTorch implementation for paper: Diffusion-GAN: Training GANs with Diffusion

diffusion-gan's Issues

Bug encountered when running generate.py

Hi,

I encounter the following bug when running generate.py (command included in the snapshot). I guess the reason is that the current code is inconsistent with the persistent class stored in the pickle.

(base) user@xxx:/data/user/Diffusion-GAN/diffusion-stylegan2$ python generate.py --outdir=out --seeds=1-100 \
>     --network=https://tsciencescu.blob.core.windows.net/projectshzheng/DiffusionGAN/diffusion-stylegan2-ffhq.pkl
Loading networks from "https://tsciencescu.blob.core.windows.net/projectshzheng/DiffusionGAN/diffusion-stylegan2-ffhq.pkl"...
Downloading https://tsciencescu.blob.core.windows.net/projectshzheng/DiffusionGAN/diffusion-stylegan2-ffhq.pkl ... done
Traceback (most recent call last):
  File "/data/user/Diffusion-GAN/diffusion-stylegan2/generate.py", line 127, in <module>
    generate_images() # pylint: disable=no-value-for-parameter
  File "/home/user/anaconda3/lib/python3.9/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/home/user/anaconda3/lib/python3.9/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/home/user/anaconda3/lib/python3.9/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/user/anaconda3/lib/python3.9/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/home/user/anaconda3/lib/python3.9/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/data/user/Diffusion-GAN/diffusion-stylegan2/generate.py", line 84, in generate_images
    G = legacy.load_network_pkl(f)['G_ema'].to(device) # type: ignore
  File "/data/user/Diffusion-GAN/diffusion-stylegan2/legacy.py", line 21, in load_network_pkl
    data = _LegacyUnpickler(f).load()
  File "/data/user/Diffusion-GAN/diffusion-stylegan2/legacy.py", line 71, in find_class
    return super().find_class(module, name)
ModuleNotFoundError: No module named 'training.augment'

InsGen cannot run on a single GPU.

InsGen cannot run on a single GPU. Because init_process_group is not called. The original authors of InsGen haven't fixed this bug.

Application

Hi, can we use this code for paired/ unpaired translation?

How to apply DIffusion-GAN to StyleGAN3

Hi, I want to ask what change I need to make to apply Diffusion-GAN to the StyleGAN3 repo ?

License

I do see many scripts in the repo that have "Copyright (c) 2021, NVIDIA CORPORATION. All rights reserved..." on the very top. Does this conflict with the MIT License associated with the repo?

Thanks,
Husam

ninja: build stopped: subcommand failed.

got error ninja build stopped, when training diffsuion stylegan2. and because of this, i got warnings.warn('Failed to build CUDA kernels for upfirdn2d. Falling back to slow reference implementation. Details:\n\n' + traceback.format_exc()) and no module named upfirdn2d too.

my environment is same with environment.yml
my gcc version is 9.4.0

Traceback (most recent call last):
File "/opt/conda/envs/difgan/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 1808, in _run_ninja_build
subprocess.run(
File "/opt/conda/envs/difgan/lib/python3.9/subprocess.py", line 528, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/octadion/diffusion-gan/Diffusion-GAN/diffusion-stylegan2/torch_utils/ops/bias_act.py", line 41, in init
plugin = custom_ops.get_plugin('bias_act_plugin', sources=sources, extra_cuda_cflags=['--use_fast_math'])
File "/home/octadion/diffusion-gan/Diffusion-GAN/diffusion-stylegan2/torch_utils/custom_ops.py", line 103, in get_plugin
torch.utils.cpp_extension.load(name=module_name, verbose=verbose_build, sources=sources, **build_kwargs)
File "/opt/conda/envs/difgan/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 1202, in load
return jit_compile(
File "/opt/conda/envs/difgan/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 1425, in jit_compile
write_ninja_file_and_build_library(
File "/opt/conda/envs/difgan/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 1537, in write_ninja_file_and_build_library
run_ninja_build(
File "/opt/conda/envs/difgan/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 1824, in run_ninja_build
raise RuntimeError(message) from e
RuntimeError: Error building extension 'bias_act_plugin': [1/3] /usr/local/cuda/bin/nvcc -DTORCH_EXTENSION_NAME=bias_act_plugin -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="gcc" -DPYBIND11_STDLIB="libstdcpp" -DPYBIND11_BUILD_ABI="cxxabi1011" -isystem /opt/conda/envs/difgan/lib/python3.9/site-packages/torch/include -isystem /opt/conda/envs/difgan/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -isystem /opt/conda/envs/difgan/lib/python3.9/site-packages/torch/include/TH -isystem /opt/conda/envs/difgan/lib/python3.9/site-packages/torch/include/THC -isystem /usr/local/cuda/include -isystem /opt/conda/envs/difgan/include/python3.9 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS -D__CUDA_NO_BFLOAT16_CONVERSIONS -D__CUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr -gencode=arch=compute_80,code=compute_80 -gencode=arch=compute_80,code=sm_80 --compiler-options '-fPIC' --use_fast_math -std=c++14 -c /home/octadion/diffusion-gan/Diffusion-GAN/diffusion-stylegan2/torch_utils/ops/bias_act.cu -o bias_act.cuda.o
FAILED: bias_act.cuda.o
/usr/local/cuda/bin/nvcc -DTORCH_EXTENSION_NAME=bias_act_plugin -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="gcc" -DPYBIND11_STDLIB="libstdcpp" -DPYBIND11_BUILD_ABI="cxxabi1011" -isystem /opt/conda/envs/difgan/lib/python3.9/site-packages/torch/include -isystem /opt/conda/envs/difgan/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -isystem /opt/conda/envs/difgan/lib/python3.9/site-packages/torch/include/TH -isystem /opt/conda/envs/difgan/lib/python3.9/site-packages/torch/include/THC -isystem /usr/local/cuda/include -isystem /opt/conda/envs/difgan/include/python3.9 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS -D__CUDA_NO_BFLOAT16_CONVERSIONS -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_80,code=compute_80 -gencode=arch=compute_80,code=sm_80 --compiler-options '-fPIC' --use_fast_math -std=c++14 -c /home/octadion/diffusion-gan/Diffusion-GAN/diffusion-stylegan2/torch_utils/ops/bias_act.cu -o bias_act.cuda.o
/bin/sh: 1: /usr/local/cuda/bin/nvcc: not found
[2/3] c++ -MMD -MF bias_act.o.d -DTORCH_EXTENSION_NAME=bias_act_plugin -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="_gcc" -DPYBIND11_STDLIB="_libstdcpp" -DPYBIND11_BUILD_ABI="_cxxabi1011" -isystem /opt/conda/envs/difgan/lib/python3.9/site-packages/torch/include -isystem /opt/conda/envs/difgan/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -isystem /opt/conda/envs/difgan/lib/python3.9/site-packages/torch/include/TH -isystem /opt/conda/envs/difgan/lib/python3.9/site-packages/torch/include/THC -isystem /usr/local/cuda/include -isystem /opt/conda/envs/difgan/include/python3.9 -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++14 -c /home/octadion/diffusion-gan/Diffusion-GAN/diffusion-stylegan2/torch_utils/ops/bias_act.cpp -o bias_act.o
FAILED: bias_act.o
c++ -MMD -MF bias_act.o.d -DTORCH_EXTENSION_NAME=bias_act_plugin -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="_gcc" -DPYBIND11_STDLIB="_libstdcpp" -DPYBIND11_BUILD_ABI="_cxxabi1011" -isystem /opt/conda/envs/difgan/lib/python3.9/site-packages/torch/include -isystem /opt/conda/envs/difgan/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -isystem /opt/conda/envs/difgan/lib/python3.9/site-packages/torch/include/TH -isystem /opt/conda/envs/difgan/lib/python3.9/site-packages/torch/include/THC -isystem /usr/local/cuda/include -isystem /opt/conda/envs/difgan/include/python3.9 -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++14 -c /home/octadion/diffusion-gan/Diffusion-GAN/diffusion-stylegan2/torch_utils/ops/bias_act.cpp -o bias_act.o
In file included from /home/octadion/diffusion-gan/Diffusion-GAN/diffusion-stylegan2/torch_utils/ops/bias_act.cpp:10:
/opt/conda/envs/difgan/lib/python3.9/site-packages/torch/include/ATen/cuda/CUDAContext.h:5:10: fatal error: cuda_runtime_api.h: No such file or directory
5 | #include <cuda_runtime_api.h>
| ^~~~~~~~~~~~~~~~~~~~
compilation terminated.
ninja: build stopped: subcommand failed.

grid_sampler_2d_backward() is missing value for argument 'output_mask'.

你好，我尝试在云gpu上运行该代码，出现了如下错误，请问该如何解决呢，劳烦大神回复，谢谢
Traceback (most recent call last):
File "train.py", line 603, in
main() # pylint: disable=no-value-for-parameter
File "/root/miniconda3/lib/python3.8/site-packages/click/core.py", line 1157, in call
return self.main(*args, **kwargs)
File "/root/miniconda3/lib/python3.8/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/root/miniconda3/lib/python3.8/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/root/miniconda3/lib/python3.8/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/root/miniconda3/lib/python3.8/site-packages/click/decorators.py", line 33, in new_func
return f(get_current_context(), *args, **kwargs)
File "train.py", line 596, in main
subprocess_fn(rank=0, args=args, temp_dir=temp_dir)
File "train.py", line 422, in subprocess_fn
training_loop.training_loop(rank=rank, **args)
File "/root/GAN/Diffusion-GAN-main/diffusion-insgen/training/training_loop.py", line 351, in training_loop
loss.accumulate_gradients(phase=phase.name, real_img=real_img, real_c=real_c, gen_z=gen_z, gen_c=gen_c, sync=sync, gain=gain, cl_phases=cl_phases, D_ema=D_ema, g_fake_cl=not no_cl_on_g, **cl_loss_weight)
File "/root/GAN/Diffusion-GAN-main/diffusion-insgen/training/contrastive_loss.py", line 107, in accumulate_gradients
loss_Gmain.mean().mul(gain).backward()
File "/root/miniconda3/lib/python3.8/site-packages/torch/_tensor.py", line 363, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
File "/root/miniconda3/lib/python3.8/site-packages/torch/autograd/init.py", line 173, in backward
Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
File "/root/miniconda3/lib/python3.8/site-packages/torch/autograd/function.py", line 253, in apply
return user_fn(self, *args)
File "/root/GAN/Diffusion-GAN-main/diffusion-insgen/torch_utils/ops/grid_sample_gradfix.py", line 50, in backward
grad_input, grad_grid = _GridSample2dBackward.apply(grad_output, input, grid)
File "/root/GAN/Diffusion-GAN-main/diffusion-insgen/torch_utils/ops/grid_sample_gradfix.py", line 59, in forward
grad_input, grad_grid = op(grad_output, input, grid, 0, 0, False)
RuntimeError: aten::grid_sampler_2d_backward() is missing value for argument 'output_mask'. Declaration: aten::grid_sampler_2d_backward(Tensor grad_output, Tensor input, Tensor grid, int interpolation_mode, int padding_mode, bool align_corners, bool[2] output_mask) -> (Tensor, Tensor)

training time

Thank you for your excellent work. Since I have been studying StyleGAN2 before, I want to know how is the training time of diffusion-Gan compared with StyleGAN2?

GAN

Hi, I want to ask what change I need to make to apply Diffusion-GAN to the other GAN repo ?

Generation includes reverse process of Diffusion?

Hello, thanks for providing your amazing work.

I wonder if there is a reverse process in generate.py to generate images just like other diffusion models.
Seemingly you only use GAN models for generation, so I just want to make whether my understand is proper clear!

Thanks!

How to use in Our Model？

If I want to plug and play in my model how do I do that?

How to sample the images with trained model?

Hi, I'm Goo-Young Moon, a master's degree student at Korea University. I read your paper interestingly while investigating the research on diffusion models. But, I opened the issue as I had some problems while I was executing codes you uploaded.

I trained the diffusion-projected-gan and got the best_model.pkl. But there is a problem that the best_model.pkl is not loaded properly when sampling images through 'gen_images.py'. I attached options and error messages below.

I'm sorry to bother you, but I'd appreciate your help. Thank you for your research and hard work.

A tiny question in diffusion.py

Thanks for the nice work, and I am trying to utilize the diffusion in my GAN model.
And I have a minor question in your code:

(diffusion.py)

        if self.ts_dist == 'priority':
            prob_t = np.arange(t) / np.arange(t).sum()
            t_diffusion = np.random.choice(np.arange(1, t + 1), size=diffusion_ind, p=prob_t)

You set prob_t as np.arange(t) / np.arange(t).sum(), not np.arange(1, t + 1) / np.arange(1, t + 1).sum().
According to the former case (your code), the first diffused image is skipped.
If T is 10,
prob_t is [0, 0.02222222, 0.04444444, 0.06666667, 0.08888889, 0.11111111, 0.13333333, 0.15555556, 0.17777778, 0.2], and
if alphas_bar_sqrt is tensor([1.0000, 0.9999, 0.9988, 0.9965, 0.9932, 0.9887, 0.9832, 0.9766, 0.9690, 0.9603, 0.9507]) and
one_minus_alphas_bar_sqrt is tensor([0.0000, 0.0100, 0.0491, 0.0832, 0.1167, 0.1498, 0.1826, 0.2151, 0.2472, 0.2790, 0.3103]),
the second items in those tensors, that is, 0.9999 and 0.0100 are never drawn to the time step t.

Is this the intended behavior? or is there any misunderstanding of myself?

Thanks in advance, and again thank you for your nice work.

RuntimeError: Both events must be recorded before calculating elapsed time.

Describing bug:
crashes after evaluating metrics

Training:
diffusion-stylegan2

Environment:
same with environment.yml

Cfg:
cfg=paper256 --aug no --target 0.6 --noise_sd 0.05 --ts_dist priority

Loading training set...

Num images: 20000
Image shape: [3, 256, 256]
Label shape: [0]

Constructing networks...
Setting up augmentation...
Resuming from "/home/octa/diffusion-gan/Diffusion-GAN/pretrained/diffusion-stylegan2-lsun-bedroom.pkl"
Setting up PyTorch plugin "bias_act_plugin"... Done.
Setting up PyTorch plugin "upfirdn2d_plugin"... Done.

Generator Parameters Buffers Output shape Datatype

---

mapping.fc0 262656 - [16, 512] float32
mapping.fc1 262656 - [16, 512] float32
mapping.fc2 262656 - [16, 512] float32
mapping.fc3 262656 - [16, 512] float32
mapping.fc4 262656 - [16, 512] float32
mapping.fc5 262656 - [16, 512] float32
mapping.fc6 262656 - [16, 512] float32
mapping.fc7 262656 - [16, 512] float32
mapping - 512 [16, 14, 512] float32
synthesis.b4.conv1 2622465 32 [16, 512, 4, 4] float32
synthesis.b4.torgb 264195 - [16, 3, 4, 4] float32
synthesis.b4:0 8192 16 [16, 512, 4, 4] float32
synthesis.b4:1 - - [16, 512, 4, 4] float32
synthesis.b8.conv0 2622465 80 [16, 512, 8, 8] float32
synthesis.b8.conv1 2622465 80 [16, 512, 8, 8] float32
synthesis.b8.torgb 264195 - [16, 3, 8, 8] float32
synthesis.b8:0 - 16 [16, 512, 8, 8] float32
synthesis.b8:1 - - [16, 512, 8, 8] float32
synthesis.b16.conv0 2622465 272 [16, 512, 16, 16] float32
synthesis.b16.conv1 2622465 272 [16, 512, 16, 16] float32
synthesis.b16.torgb 264195 - [16, 3, 16, 16] float32
synthesis.b16:0 - 16 [16, 512, 16, 16] float32
synthesis.b16:1 - - [16, 512, 16, 16] float32
synthesis.b32.conv0 2622465 1040 [16, 512, 32, 32] float16
synthesis.b32.conv1 2622465 1040 [16, 512, 32, 32] float16
synthesis.b32.torgb 264195 - [16, 3, 32, 32] float16
synthesis.b32:0 - 16 [16, 512, 32, 32] float16
synthesis.b32:1 - - [16, 512, 32, 32] float32
synthesis.b64.conv0 1442561 4112 [16, 256, 64, 64] float16
synthesis.b64.conv1 721409 4112 [16, 256, 64, 64] float16
synthesis.b64.torgb 132099 - [16, 3, 64, 64] float16
synthesis.b64:0 - 16 [16, 256, 64, 64] float16
synthesis.b64:1 - - [16, 256, 64, 64] float32
synthesis.b128.conv0 426369 16400 [16, 128, 128, 128] float16
synthesis.b128.conv1 213249 16400 [16, 128, 128, 128] float16
synthesis.b128.torgb 66051 - [16, 3, 128, 128] float16
synthesis.b128:0 - 16 [16, 128, 128, 128] float16
synthesis.b128:1 - - [16, 128, 128, 128] float32
synthesis.b256.conv0 139457 65552 [16, 64, 256, 256] float16
synthesis.b256.conv1 69761 65552 [16, 64, 256, 256] float16
synthesis.b256.torgb 33027 - [16, 3, 256, 256] float16
synthesis.b256:0 - 16 [16, 64, 256, 256] float16
synthesis.b256:1 - - [16, 64, 256, 256] float32

---

Total 24767458 175568 - -

Discriminator Parameters Buffers Output shape Datatype

---

b256.fromrgb 256 16 [16, 64, 256, 256] float16
b256.skip 8192 16 [16, 128, 128, 128] float16
b256.conv0 36928 16 [16, 64, 256, 256] float16
b256.conv1 73856 16 [16, 128, 128, 128] float16
b256 - 16 [16, 128, 128, 128] float16
b128.skip 32768 16 [16, 256, 64, 64] float16
b128.conv0 147584 16 [16, 128, 128, 128] float16
b128.conv1 295168 16 [16, 256, 64, 64] float16
b128 - 16 [16, 256, 64, 64] float16
b64.skip 131072 16 [16, 512, 32, 32] float16
b64.conv0 590080 16 [16, 256, 64, 64] float16
b64.conv1 1180160 16 [16, 512, 32, 32] float16
b64 - 16 [16, 512, 32, 32] float16
b32.skip 262144 16 [16, 512, 16, 16] float16
b32.conv0 2359808 16 [16, 512, 32, 32] float16
b32.conv1 2359808 16 [16, 512, 16, 16] float16
b32 - 16 [16, 512, 16, 16] float16
b16.skip 262144 16 [16, 512, 8, 8] float32
b16.conv0 2359808 16 [16, 512, 16, 16] float32
b16.conv1 2359808 16 [16, 512, 8, 8] float32
b16 - 16 [16, 512, 8, 8] float32
b8.skip 262144 16 [16, 512, 4, 4] float32
b8.conv0 2359808 16 [16, 512, 8, 8] float32
b8.conv1 2359808 16 [16, 512, 4, 4] float32
b8 - 16 [16, 512, 4, 4] float32
mapping.embed 1024 - [16, 512] float32
mapping.fc0 262656 - [16, 512] float32
mapping.fc1 262656 - [16, 512] float32
mapping.fc2 262656 - [16, 512] float32
mapping.fc3 262656 - [16, 512] float32
mapping.fc4 262656 - [16, 512] float32
mapping.fc5 262656 - [16, 512] float32
mapping.fc6 262656 - [16, 512] float32
mapping.fc7 262656 - [16, 512] float32
b4.mbstd - - [16, 513, 4, 4] float32
b4.conv 2364416 16 [16, 512, 4, 4] float32
b4.fc 4194816 - [16, 512] float32
b4.out 262656 - [16, 512] float32
b4 - - [16, 1] float32

---

Total 26365504 416 - -

Distributing across 1 GPUs...
Setting up training phases...
Exporting sample images...
Initializing logs...
Skipping tfevents export: No module named 'tensorboard'
Training for 25000 kimg...

tick 12450 kimg 50212.2 time 1m 20s sec/tick 12.6 sec/kimg 197.43 maintenance 66.9 cpumem 3.70 gpumem 11.26 augment 0.997 T 10.0
Evaluating metrics...
{"results": {"fid50k_full": 271.45591156832063}, "metric": "fid50k_full", "total_time": 540.888575553894, "total_time_str": "9m 01s", "num_gpus": 1, "snapshot_pkl": "network-snapshot.pkl", "timestamp": 1680022555.4954345}
Traceback (most recent call last):
File "/home/octa/diffusion-gan/Diffusion-GAN/diffusion-stylegan2/train.py", line 531, in
main() # pylint: disable=no-value-for-parameter
File "/home/octa/anaconda3/envs/difgan/lib/python3.9/site-packages/click/core.py", line 1128, in call
return self.main(*args, **kwargs)
File "/home/octa/anaconda3/envs/difgan/lib/python3.9/site-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File "/home/octa/anaconda3/envs/difgan/lib/python3.9/site-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/octa/anaconda3/envs/difgan/lib/python3.9/site-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/home/octa/anaconda3/envs/difgan/lib/python3.9/site-packages/click/decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
File "/home/octa/diffusion-gan/Diffusion-GAN/diffusion-stylegan2/train.py", line 524, in main
subprocess_fn(rank=0, args=args, temp_dir=temp_dir)
File "/home/octa/diffusion-gan/Diffusion-GAN/diffusion-stylegan2/train.py", line 357, in subprocess_fn
training_loop.training_loop(rank=rank, **args)
File "/home/octa/diffusion-gan/Diffusion-GAN/diffusion-stylegan2/training/training_loop.py", line 437, in training_loop
value = phase.start_event.elapsed_time(phase.end_event)
File "/home/octa/anaconda3/envs/difgan/lib/python3.9/site-packages/torch/cuda/streams.py", line 204, in elapsed_time
return super(Event, self).elapsed_time(end_event)
RuntimeError: Both events must be recorded before calculating elapsed time.

'tuple' object is not callable

Hi
First I want to thank you for publishing this code.
I tried to train a model as you described in this repository without any changes to the code, but I have this error.

Traceback (most recent call last):
File "/content/Diffusion-GAN/diffusion-insgen/train.py", line 603, in
main() # pylint: disable=no-value-for-parameter
File "/usr/local/lib/python3.9/dist-packages/click/core.py", line 1130, in call
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.9/dist-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.9/dist-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.9/dist-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.9/dist-packages/click/decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
File "/content/Diffusion-GAN/diffusion-insgen/train.py", line 596, in main
subprocess_fn(rank=0, args=args, temp_dir=temp_dir)
File "/content/Diffusion-GAN/diffusion-insgen/train.py", line 422, in subprocess_fn
training_loop.training_loop(rank=rank, **args)
File "/content/Diffusion-GAN/diffusion-insgen/training/training_loop.py", line 351, in training_loop
loss.accumulate_gradients(phase=phase.name, real_img=real_img, real_c=real_c, gen_z=gen_z, gen_c=gen_c, sync=sync, gain=gain, cl_phases=cl_phases, D_ema=D_ema, g_fake_cl=not no_cl_on_g, **cl_loss_weight)
File "/content/Diffusion-GAN/diffusion-insgen/training/contrastive_loss.py", line 107, in accumulate_gradients
loss_Gmain.mean().mul(gain).backward()
File "/usr/local/lib/python3.9/dist-packages/torch/_tensor.py", line 487, in backward
torch.autograd.backward(
File "/usr/local/lib/python3.9/dist-packages/torch/autograd/init.py", line 200, in backward
Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
File "/usr/local/lib/python3.9/dist-packages/torch/autograd/function.py", line 274, in apply
return user_fn(self, *args)
File "/content/Diffusion-GAN/diffusion-insgen/torch_utils/ops/grid_sample_gradfix.py", line 50, in backward
grad_input, grad_grid = _GridSample2dBackward.apply(grad_output, input, grid)
File "/usr/local/lib/python3.9/dist-packages/torch/autograd/function.py", line 506, in apply
return super().apply(*args, **kwargs) # type: ignore[misc]
File "/content/Diffusion-GAN/diffusion-insgen/torch_utils/ops/grid_sample_gradfix.py", line 59, in forward
grad_input, grad_grid = op(grad_output, input, grid, 0, 0, False)
TypeError: 'tuple' object is not callable

I didn't made any changes to the code and I run the code on google colab with the exact arguments that is written in this repository.
Thanks for your help in advance

Download link for pretrained model.

Hi, I think you did a great research and I want to test with your model.

But when I tried to download the pretrained Diffusion-StyleGAN2 weights trained on CelebA 64x64 dataset,
I got an error that "requests.exceptions.HTTPError: 409 Client Error: Public access is not permitted on this storage account. for url: https://tsciencescu.blob.core.windows.net/projectshzheng/DiffusionGAN/diffusion-stylegan2-celeba64.pkl"

Would you please check this?

about the value of target

Hello, may I ask how the value of ada_target is set

diffusion.py

I am interested in your work, but I have difficulty designing a diffusion plugin for my own GAN, do you have a simple diffusion plugin for reference

Error while running train.py

Hi,
I wanted to train a model with cifar dataset, so I followed the steps described in the readme; but while running the train.py file exactly as you described in the readme, I got an error which you can see below:

Traceback (most recent call last):
File "train.py", line 603, in
main() # pylint: disable=no-value-for-parameter
File "C:\ProgramData\miniconda3\envs\difgan\lib\site-packages\click\core.py", line 1128, in call
return self.main(*args, **kwargs)
File "C:\ProgramData\miniconda3\envs\difgan\lib\site-packages\click\core.py", line 1053, in main
rv = self.invoke(ctx)
File "C:\ProgramData\miniconda3\envs\difgan\lib\site-packages\click\core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "C:\ProgramData\miniconda3\envs\difgan\lib\site-packages\click\core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "C:\ProgramData\miniconda3\envs\difgan\lib\site-packages\click\decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
File "train.py", line 596, in main
subprocess_fn(rank=0, args=args, temp_dir=temp_dir)
File "train.py", line 422, in subprocess_fn
training_loop.training_loop(rank=rank, **args)
File "C:\Users\user\Desktop\Project\Diffusion-GAN\diffusion-insgen\training\training_loop.py", line 351, in training_loop
loss.accumulate_gradients(phase=phase.name, real_img=real_img, real_c=real_c, gen_z=gen_z, gen_c=gen_c, sync=sync, gain=gain, cl_phases=cl_phases, D_ema=D_ema, g_fake_cl=not no_cl_on_g, **cl_loss_weight)
File "C:\Users\user\Desktop\Project\Diffusion-GAN\diffusion-insgen\training\contrastive_loss.py", line 92, in accumulate_gradients
gen_logits = self.run_D(gen_img, gen_c, sync=False)
File "C:\Users\user\Desktop\Project\Diffusion-GAN\diffusion-insgen\training\contrastive_loss.py", line 55, in run_D
img, t = self.augment_pipe(img)
File "C:\ProgramData\miniconda3\envs\difgan\lib\site-packages\torch\nn\modules\module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "C:\Users\user\Desktop\Project\Diffusion-GAN\diffusion-insgen\training\augment.py", line 203, in forward
x_t = q_sample(x_0, alphas_bar_sqrt, one_minus_alphas_bar_sqrt, t,
File "C:\Users\user\Desktop\Project\Diffusion-GAN\diffusion-insgen\training\augment.py", line 86, in q_sample
alphas_t_sqrt = alphas_bar_sqrt[t].view(-1, 1, 1, 1)
IndexError: tensors used as indices must be long, byte or bool tensors

I used environment.yml given in the directory as my environment.
My PC specs: Windows 10 64bit - Nvidia GTX 1660
Thanks in advance for your help

Questions about adaptive diffusion.

As the paper state, the metric rd that estimates how much the discriminator overfits to the data is defined as rd=E[sign(D(y, t) - 0.5)]. But when it comes to the implementation of the code, rd is defined as rd=E[sign(D(y, t))], without subtracting 0.5. My question is if i switch to a different GAN, such as LSGAN or Vanilla GAN, whose loss function is different from StyleGAN2, can I still use the rt defined above? Do I need to make appropriate modifications to rt?

Can you add an requirements file?

First of all , It's an outstanding work! Good Job!
Can you add an requirements file?
Thank you!

regarding the setting of the hyperparameter "ada_kimg"

Hello, thank you for your excellent work. Regarding your code, there is one point that I am not clear about, what does the parameter "ada_kimg" represent?

if update_diffusion: # batch_idx % ada_interval == 0 adjust = np.sign(sign(Discriminator(real_images)) - ada_target) * C # C = (batch_size * ada_interval) / (ada_kimg * 1000) diffusion.p = (diffusion.p + adjust).clip(min=0., max=1.) diffusion.update_T()

How to modify discriminator to incorporate diffusion timestep t?

Hi, thank you for your excellent work. I am trying to use this work to improve my GAN training, according to the readme
https://github.com/Zhendong-Wang/Diffusion-GAN#full-version
it's better to add diffusion timestep t as an input for discriminator, i just don't konw how to add timestep t as an input for discriminator since t is just a scalar while input image is 2D. What i can think of is extending t to the size of input image, i.e HxW, and then concatenate it with input image along the channel dimension. Will this simple method work or is there a better method to incorporate timestep t in discriminator?

Error resuming Diffusion-InsGen

  File "train.py", line 613, in <module>                                                            
    main() # pylint: disable=no-value-for-parameter                                                                                                                                                     
  File "diffusion-gan/diffusion-stylegan2/.env/lib/python3.8/site-packages/click/core.py", line 1130, in __call__                                                                  
    return self.main(*args, **kwargs)                                                               
  File "diffusion-gan/diffusion-stylegan2/.env/lib/python3.8/site-packages/click/core.py", line 1055, in main                                                                      
    rv = self.invoke(ctx)                                                                           
  File "diffusion-gan/diffusion-stylegan2/.env/lib/python3.8/site-packages/click/core.py", line 1404, in invoke                                                                    
    return ctx.invoke(self.callback, **ctx.params)                                                  
  File "diffusion-gan/diffusion-stylegan2/.env/lib/python3.8/site-packages/click/core.py", line 760, in invoke                                                                     
    return __callback(*args, **kwargs)                                                              
  File "diffusion-gan/diffusion-stylegan2/.env/lib/python3.8/site-packages/click/decorators.py", line 26, in new_func                                                              
    return f(get_current_context(), *args, **kwargs)                                                
  File "train.py", line 606, in main                                                                
    subprocess_fn(rank=0, args=args, temp_dir=temp_dir)                                             
  File "train.py", line 432, in subprocess_fn                                                       
    training_loop.training_loop(rank=rank, **args)                                                  
  File "diffusion-gan/diffusion-insgen/training/training_loop.py", line 193, in training_loop                                                                                      
    misc.copy_params_and_buffers(resume_data[name], module, require_all=False)                      
KeyError: 'D_ema'  

I think the fix I just to add:

--- a/diffusion-insgen/training/training_loop.py
+++ b/diffusion-insgen/training/training_loop.py
@@ -417,21 +417,21 @@ def training_loop(
         # Save network snapshot.
         snapshot_pkl = None
         snapshot_data = None
         if (network_snapshot_ticks is not None) and (done or cur_tick % network_snapshot_ticks == 0):
             snapshot_data = dict(training_set_kwargs=dict(training_set_kwargs))
-            for name, module in [('G', G), ('D', D), ('G_ema', G_ema), ('augment_pipe', augment_pipe)]:
+            for name, module in [('G', G), ('D', D), ('G_ema', G_ema), ('D_ema', D_ema), ('DHead', DHead), ('GHead', GHead), ('augment_pipe', augment_pipe)]:

But then the following error occurs:

RuntimeError: a leaf Variable that requires grad is being used in an in-place operation.

Furthermore this issue also appears to be present in the upstream version of insgen.

No module named 'upfirdn2d_plugin'

Can you please upload the requirements.txt file.

Some questions

adjust = np.sign(sign(Discriminator(real_images)) - ada_target) * C # C = (batch_size * ada_interval) / (ada_kimg * 1000)
or
adjust = np.sign(sign(Discriminator(real_images) - 0.5) - ada_target) * C # C = (batch_size * ada_interval) / (ada_kimg * 1000) according to the paper?

1. Which one in correct?
2. Any suggestions for the setting of 'ada_kimg'？
3. Are these lines of code suitable for multi-gpu training? 'Diffusion' will get different updates on different ranks? And how to set the 'batch_size'?

I think your work is wonderful! Thank you for your reply in advance!

DCGAN/SNGAN architecture with discriminator conditioning

Hi, what architecture did you use for the DCGAN/SNGAN experiments in the supplementary?

did you just concatenate learned t-embedding to the discriminator input like here ?

class Discriminator(nn.Module):
    def __init__(self):
        super(Discriminator, self).__init__()

        self.label_embedding = nn.Embedding(opt.n_classes, opt.n_classes)

        self.model = block(in_dim=int(np.prod(img_shape)) + opt.n_classes)

    def forward(self, img, labels):
        # Concatenate label embedding and image to produce input
        d_in = torch.cat((img.view(img.size(0), -1), self.label_embedding(labels)), -1)
        validity = self.model(d_in)
        return validity

Error when resuming training on Diffusion-StyleGAN2 for CelebA-64 dataset

Hello,
I am trying to resume training using the pretrained weights on the CelebA-64 dataset, but I am having problem loading the weights into the model. I am currently using these commands:

python train.py --outdir=[outdir] --data=[data folder] --gpus=1 --cfg auto --kimg 25000 --aug no --target 0.6 --noise_sd 0.05 --ts_dist priority --resume pretrained/diffusion-stylegan2-celeba64.pkl

I am getting the following error message:

Constructing networks...
Setting up augmentation...
Resuming from "pretrained/diffusion-stylegan2-celeba64.pkl"
Traceback (most recent call last):
  File "train.py", line 531, in <module>
    main() # pylint: disable=no-value-for-parameter
  File "anaconda3/envs/diffusiongan2/lib/python3.7/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "anaconda3/envs/diffusiongan2/lib/python3.7/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "anaconda3/envs/diffusiongan2/lib/python3.7/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "anaconda3/envs/diffusiongan2/lib/python3.7/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "anaconda3/envs/diffusiongan2/lib/python3.7/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "train.py", line 524, in main
    subprocess_fn(rank=0, args=args, temp_dir=temp_dir)
  File "train.py", line 357, in subprocess_fn
    training_loop.training_loop(rank=rank, **args)
  File "Diffusion-GAN-main/diffusion-stylegan2/training/training_loop.py", line 180, in training_loop
    misc.copy_params_and_buffers(resume_data[name], module, require_all=False)
  File "Diffusion-GAN-main/diffusion-stylegan2/torch_utils/misc.py", line 160, in copy_params_and_buffers
    tensor.copy_(src_tensors[name].detach()).requires_grad_(tensor.requires_grad)
RuntimeError: The size of tensor a (256) must match the size of tensor b (512) at non-singleton dimension 0

I believe this may have something to do with mismatching channel dimensions. How can I fix these?

Unexpected Keyword Argument 'Diffusion'

Hey there,
I stumbled upon this project when I was studying different types of GANS.
I ran train.py on my own dataset,
everything worked out, until right after the networks finished loading in, it gave me the error
"TypeError: init() got an unexpected keyword argument 'Diffusion'"
do you have any idea how I could fix this

The error happened during "setting up training phases..."
Thanks

Full Error:

Traceback (most recent call last): File "train.py", line 531, in <module> main() # pylint: disable=no-value-for-parameter File "/home/quantium/anaconda3/envs/stylegan2-ada/lib/python3.7/site-packages/click/core.py", line 1130, in __call__ return self.main(*args, **kwargs) File "/home/quantium/anaconda3/envs/stylegan2-ada/lib/python3.7/site-packages/click/core.py", line 1055, in main rv = self.invoke(ctx) File "/home/quantium/anaconda3/envs/stylegan2-ada/lib/python3.7/site-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, **ctx.params) File "/home/quantium/anaconda3/envs/stylegan2-ada/lib/python3.7/site-packages/click/core.py", line 760, in invoke return __callback(*args, **kwargs) File "/home/quantium/anaconda3/envs/stylegan2-ada/lib/python3.7/site-packages/click/decorators.py", line 26, in new_func return f(get_current_context(), *args, **kwargs) File "train.py", line 524, in main subprocess_fn(rank=0, args=args, temp_dir=temp_dir) File "train.py", line 357, in subprocess_fn training_loop.training_loop(rank=rank, **args) File "/mnt/c/Users/quant/DGAN/diffusion-stylegan2/training/training_loop.py", line 213, in training_loop loss = dnnlib.util.construct_class_by_name(device=device, **ddp_modules, **loss_kwargs) # subclass of training.loss.Loss File "/mnt/c/Users/quant/DGAN/diffusion-stylegan2/dnnlib/util.py", line 289, in construct_class_by_name return call_func_by_name(*args, func_name=class_name, **kwargs) File "/mnt/c/Users/quant/DGAN/diffusion-stylegan2/dnnlib/util.py", line 284, in call_func_by_name return func_obj(*args, **kwargs) TypeError: __init__() got an unexpected keyword argument 'Diffusion'

Possible typo in the adjusting formula in the paper?

Hi,
In the paper you used the following formula for the adjusting variable rd

But in the StyleGAN-ADA paper you're refereeing to the formula is

Did you indeed refer to rt in StyleGAN-ADA

Substituting 0.5 from the discriminator usually makes no sense as the values are no between [0,1] as shown in the StyleGAN-ADA paper they are simply symmetrical around 0.

Am I missing something?

FID discrepancy

The stylegan2 FID on FFHQ reported in the paper (4.41) is significantly worse than the official evaluation (2.70), is there an explanation for this?

Use Diffusion-GAN in Other GAN Architecture

Hello @Zhendong-Wang and Team,

I would like to firstly say that it's a great work! Thank you for sharing the code. I am trying to use Diffusion-GAN in a GAN architecture for image enhancement. Can you please help me by letting me know how do I use the three steps mentioned for Simple Plug-in by you in the readme in the below code -

for epoch in range(num_epochs):
    for n_batch, (blur_batch, clean_batch) in enumerate(data_loader):
        
        real_data = clean_batch.float().cuda()
        noised_data = blur_batch.float().cuda()
        
        # 1. Train Discriminator
        # Generate fake data
        fake_data = generator(noised_data)

        # Reset gradients
        d_optimizer.zero_grad()
        
        # 1.1 Train on Real Data
        prediction_real = discriminator(real_data, noised_data)

        # Calculate error and backpropagate
        real_data_target = torch.ones_like(prediction_real)
        loss_real = loss1(prediction_real, real_data_target)

        # 1.2 Train on Fake Data, you would need to add one more component
        prediction_fake = discriminator(fake_data, noised_data)

        # Calculate error and backpropagate
        fake_data_target = torch.zeros_like(prediction_real)
        loss_fake = loss1(prediction_fake, fake_data_target)

        loss_d = (loss_real + loss_fake)/2
        loss_d.backward(retain_graph=True)
        
        # 1.3 Update weights with gradients
        d_optimizer.step()
  
        # 2. Train Generator
        g_optimizer.zero_grad()

        # Sample noise and generate fake data
        prediction = discriminator(fake_data, real_data)
        
        # Calculate error and backpropagate
        real_data_target = torch.ones_like(prediction)
        #import pdb; pdb.set_trace();

        loss_g1 = loss1(prediction, real_data_target)
        loss_g2 = loss1(fake_data, real_data)*500
        loss_g = loss_g1 + loss_g2

        loss_g.backward()

        # Update weights with gradients
        g_optimizer.step()
                
        # Log error
        logger.log(loss_d, loss_g, epoch, n_batch, num_batches)

        # Display Progress
        if (n_batch) % 100 == 0:
            display.clear_output(True)
            # Display Images
            test_images = vectors_to_images(generator(test_noise())).data.cpu()
            logger.log_images(test_images, num_test_samples, epoch, n_batch, num_batches);
            # Display status Logs
            logger.display_status(
                epoch, num_epochs, n_batch, num_batches,
                loss_d, loss_g, prediction_real, prediction_fake
            )
        # Model Checkpoints
        logger.save_models(generator, discriminator, epoch)

Thank you so much :)

AssertionError: Default process group is not initialized

raceback (most recent call last):
File "train.py", line 603, in
main() # pylint: disable=no-value-for-parameter
File "E:\anconda3\envs\diffusionGAN\lib\site-packages\click\core.py", line 1130, in call
return self.main(*args, **kwargs)
File "E:\anconda3\envs\diffusionGAN\lib\site-packages\click\core.py", line 1055, in main
rv = self.invoke(ctx)
File "E:\anconda3\envs\diffusionGAN\lib\site-packages\click\core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "E:\anconda3\envs\diffusionGAN\lib\site-packages\click\core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "E:\anconda3\envs\diffusionGAN\lib\site-packages\click\decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
File "train.py", line 596, in main
subprocess_fn(rank=0, args=args, temp_dir=temp_dir)
File "train.py", line 422, in subprocess_fn
training_loop.training_loop(rank=rank, **args)
File "E:\Python\Diffusion-GAN-main\diffusion-insgen\training\training_loop.py", line 351, in training_loop
loss.accumulate_gradients(phase=phase.name, real_img=real_img, real_c=real_c, gen_z=gen_z, gen_c=gen_c, sync=sync, gain=gain, cl_phases=cl_phases, D_ema=D_ema, g_fake_cl=not no_cl_on_g, **cl_loss_weight)
File "E:\Python\Diffusion-GAN-main\diffusion-insgen\training\contrastive_loss.py", line 162, in accumulate_gradients
loss_Dreal = loss_Dreal + lw_real_cl * self.run_cl(real_img_tmp, real_c, sync, Dphase.module, D_ema, loss_name='D_cl')
File "E:\Python\Diffusion-GAN-main\diffusion-insgen\training\contrastive_loss.py", line 77, in run_cl
loss = contrastive_head(logits0, logits1, loss_only=loss_only, update_q=update_q)
File "E:\anconda3\envs\diffusionGAN\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "E:\Python\Diffusion-GAN-main\diffusion-insgen\training\contrastive_head.py", line 183, in forward
self._dequeue_and_enqueue(k)
File "E:\anconda3\envs\diffusionGAN\lib\site-packages\torch\autograd\grad_mode.py", line 26, in decorate_context
return func(*args, **kwargs)
File "E:\Python\Diffusion-GAN-main\diffusion-insgen\training\contrastive_head.py", line 51, in _dequeue_and_enqueue
keys = concat_all_gather(keys)
File "E:\anconda3\envs\diffusionGAN\lib\site-packages\torch\autograd\grad_mode.py", line 26, in decorate_context
return func(*args, **kwargs)
File "E:\Python\Diffusion-GAN-main\diffusion-insgen\training\contrastive_head.py", line 197, in concat_all_gather
for _ in range(torch.distributed.get_world_size())]
File "E:\anconda3\envs\diffusionGAN\lib\site-packages\torch\distributed\distributed_c10d.py", line 638, in get_world_size
return _get_group_size(group)
File "E:\anconda3\envs\diffusionGAN\lib\site-packages\torch\distributed\distributed_c10d.py", line 220, in _get_group_size
_check_default_pg()
File "E:\anconda3\envs\diffusionGAN\lib\site-packages\torch\distributed\distributed_c10d.py", line 211, in _check_default_pg
"Default process group is not initialized"
AssertionError: Default process group is not initialized

How to use Diffusion-GAN in GP-UNIT

Really amazing work, I want to use your method to train GP-UNIT, but don't know how to modify my code, can you help me it is very appreciated, here is the code link of GP-UNIT
https://github.com/williamyang1991/GP-UNIT

About the dataset preprocessing

Hi, I want to train StyleGAN on Celeba64, could you please tell me how to prepare the celeba64.zip? Thanks!

how to lower fid?

now I'm trying to generate images in the form of textile motifs with diff-stylegan, I'm using transfer learning from lsun bedroom, cfg paper256 and the same diffusion configuration as the example, the fid I get ranges from 111 when it converges, is there any tips for can lower fid? now I'm trying to find a suitable learning rate, besides the learning rate are there other configurations that play an important role? thank you.

Question about the plug-&-play method

Hello and thanks for a great project and a well written paper.

I fell like there is a contradiction between what is suggested in the paper and the README of this paper.
In the paper it's said that simple noise injection was not found to be helpful in stabilizing GANs (citing results from "Stabilizing training of generative adversarial networks through regularization")

However in the README a "simple-plug-in" method is suggested as equivalent to the full diffusionGAN method with discriminator conditioning:
"Currently, we didn't find significant empirical differences of the two approaches"

Can you explain this?
is varying noise magnitudes the reason for success? how come Roth et. al failed to make use of noise injection.
maybe it is the adaptive noise magnitude?

Thanks again.

Inconsistent instructions and object creation

Great work! Thank you for sharing the code.

I would like to reproduce some results but the arguments are flawed. For example this line is suggested in order to train Diffusion-ProjectedGAN:
python train.py --outdir=training-runs --data="~/cifar10.zip" --gpus=4 --cfg cifar --kimg 50000 --target 0.45 --d_pos first --noise_sd 0.5

Argument target does not exist (I suppose you mean --d_target?).
cifar is not a valid choice for cfg.

Then when trying e.g this:
python train.py --outdir=training-runs --data="~/cifar10.zip" --gpus=1 --cfg stylegan2 --kimg 50000 --d_target 0.45 --d_pos first --noise_sd 0.5 --batch 16 I run into a bug:

In diffusion-projected-gan.pg_modules in line 119 a Diffusion-object is constructed:
self.diffusion = Diffusion(t_init=5, t_max=500, beta_start=1e-4, beta_end=1e-2)
but t_init is no valid argument of the Diffusion-class. Thus I changed it to:
self.diffusion = Diffusion(t_min=5, t_max=500, beta_start=1e-4, beta_end=1e-2)

Training now works.
Are these changes correct?

Conditioning signal in Diffusion ProjectedGAN

Great work! Thank you for sharing your code.

I have a question about Diffusion ProjectedGAN.

In Diffusion StyleGAN2 and Diffusion InsGAN, time-step information $t$ is provided to discriminators.

Diffusion-GAN/diffusion-stylegan2/training/networks.py

Lines 729 to 732 in 05c6b0f

	c = torch.cat((c, t), dim=1) if c is not None else t
	cmap = self.mapping(None, c)
	x = self.b4(x, img, cmap)

Diffusion-GAN/diffusion-insgen/training/networks.py

Lines 732 to 735 in 05c6b0f

	c = torch.cat((c, t), dim=1) if c is not None else t
	cmap = self.mapping(None, c)
	out = self.b4(x, img, cmap)

On the other hand, $t$ is not given to a discriminator in Diffusion ProjectedGAN, as the paper mentions in Appendix. $t$ is just used in a feature network where feature vectors are diffused.

Diffusion-GAN/diffusion-projected-gan/pg_modules/discriminator.py

Line 184 in 05c6b0f

logits = self.discriminator(features, c)

I'm wondering why a discriminator in Diffusion ProjectedGAN works well without time-step information $t$. I'd appreciated it if you would give some insights.

Best regards.