cyclomon/diffuseit

Official repository of "Diffusion-based Image Translation using Disentangled Style and Content Representation" ( ICLR 2023 )

diffuseit's Issues

train on my own dataset

Hi, this is a great work! could you please provide the training codes for own dataset when you are available? Thanks!

How to train the code in this repository?

Hello,

Thanks for sharing the code.
Could you please provide some guidance on how to set up the training process?
Looking forward to your response.

Question on FFHQ setup.

Hello, thank you for sharing your awesome project :)
I have a question on the hyperparameters that you used for Figure 8 in your paper.

I guess the command would look like this:
python main.py -I "source_image" --output_path "output_path"
-tg "target_image" --use_range_restart --diff_iter 100 --timestep_respacing 200 --skip_timesteps 80
--use_colormatch --use_noise_aug_all --use_ffhq

Please correct me if I am wrong.
Thank you for reading my question!

Minor thing to note:
You may want to add information about the pretrained arc face model to the README file.

No results generated

Hello,when I use the second training method（Image-guided Image translation to train）. No pictures are generated in the folder generated after training.
The nomenclature I use is as follows: python main.py -i "input_example/source.jpg" --output_path "./outputs/output_reptile3" -tg "input_example/leopard.jpg" --use_range_restart --diff_iter 100 --timestep_respacing 200 --skip_timesteps 80
Why? What should I do? Thank you very much!!

How to train this model

Hi author, thanks for your awesome project. Also, I wonder if you could provide scripts for training this model? Thanks in advance.

Runtime Error while Using model

/content/DiffuseIT2
Using device: cpu
Using cache found in /root/.cache/torch/hub/facebookresearch_dino_main
Start iterations 0
0% 0/120 [00:00<?, ?it/s]
Traceback (most recent call last):
File "/content/DiffuseIT2/main.py", line 8, in
image_editor.edit_image_by_prompt()
File "/content/DiffuseIT2/optimization/image_editor.py", line 287, in edit_image_by_prompt
for j, sample in enumerate(samples):
File "/content/DiffuseIT2/guided_diffusion/guided_diffusion/gaussian_diffusion.py", line 542, in p_sample_loop_progressive
out = self.p_sample(
File "/content/DiffuseIT2/guided_diffusion/guided_diffusion/gaussian_diffusion.py", line 386, in p_sample
out = self.p_mean_variance(
File "/content/DiffuseIT2/guided_diffusion/guided_diffusion/respace.py", line 91, in p_mean_variance
return super().p_mean_variance(self._wrap_model(model), *args, **kwargs)
File "/content/DiffuseIT2/guided_diffusion/guided_diffusion/gaussian_diffusion.py", line 240, in p_mean_variance
model_output = model(x, self._scale_timesteps(t), **model_kwargs)
File "/content/DiffuseIT2/guided_diffusion/guided_diffusion/respace.py", line 128, in call
return self.model(x, new_ts, **kwargs)
File "/usr/local/lib/python3.9/dist-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/content/DiffuseIT2/guided_diffusion/guided_diffusion/unet.py", line 656, in forward
h = module(h, emb)
File "/usr/local/lib/python3.9/dist-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/content/DiffuseIT2/guided_diffusion/guided_diffusion/unet.py", line 77, in forward
x = layer(x)
File "/usr/local/lib/python3.9/dist-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/usr/local/lib/python3.9/dist-packages/torch/nn/modules/conv.py", line 463, in forward
return self._conv_forward(input, self.weight, self.bias)
File "/usr/local/lib/python3.9/dist-packages/torch/nn/modules/conv.py", line 459, in _conv_forward
return F.conv2d(input, weight, bias, self.stride,
RuntimeError: "slow_conv2d_cpu" not implemented for 'Half'

urllib.error.URLError: <urlopen error [Errno 101] Network is unreachable>

Dear @cyclomon

I am running your Text-guided Image translation script python main.py -p "Black Leopard" -s "Lion" -i "input_example/lion1.jpg" --output_path "./outputs/output_leopard" --use_range_restart --use_noise_aug_all --regularize_content unfortunately it gives an error.

Traceback

    result = func(*args)
  File "/home2/coremax/anaconda3/envs/DiffuseIT/lib/python3.9/urllib/request.py", line 1389, in https_open
    return self.do_open(http.client.HTTPSConnection, req,
  File "/home2/coremax/anaconda3/envs/DiffuseIT/lib/python3.9/urllib/request.py", line 1349, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 101] Network is unreachable>

what do you mean about face identity loss in github?

hi, thank you for your work. I want to know about identity loss you wrote in github.
"For face identity loss when using FFHQ pre-trained model, download pre-trained ArcFace model LINK"

Question about the equation on the paper

Hello, thanks for your great work! Here I have a question regarding equation 9 on the paper. Is it x_(t-1) instead of x_(t-1) in the second term of right hand side, i.e.,

Thank you!

configurations for reproducing Figure 1

First of all, thank you for your great work!

I'd like to know the exact configurations for reproducing the figure 1. It would be great if you can provide the details.

Thanks!

Info about pre-trained diffusion models.

Hi, I really enjoyed going through your work!!!
Could you please address the official link(github/paper) of pre-trained diffusion model used for FFHQ dataset? Is it a guided diffusion trained on FFHQ dataset?

How to train a custom model

Hey, I tried your pre-trained models. They are really good. Do you have code to train a custom model on a different dataset from the ones mentioned in the Readme file.