peterwang512/gansketching

Sketch Your Own GAN: Customizing a GAN model with hand-drawn sketches.

gansketching's Issues

Evaluation script error

Thank you friends for suggesting interesting things!
python run_metrics.py --models_list weights / eval_list --outputmetric_results.csv

When I do this I get this error:
usage: run_metrics.py [-h] [--models_list MODELS_LIST] [--output OUTPUT] [--model_root MODEL_ROOT] [--eval_root EVAL_ROOT] [--sample_root SAMPLE_ROOT] [--batch_size BATCH_SIZE] [--eval_samples EVAL_SAMPLES] [--device DEVICE] run_metrics.py: error: unrecognized arguments: / eval_list --outputmetric_results.csv

It looks like the arguments aren't working. I want you to find a concrete solution.
thank you!

Segmentation fault (Core dumped) when loading the compiled module fused

Hello, thank you for your valuable advice. Now, I have solved the compilation problem about the fused module under the environment of pytorch **1.7.1**. Now I've got the required .so file:

(test4torch) yuanWang@server-TiTan:~/.cache/torch_extensions/fused$ ls
build.ninja  fused_bias_act_kernel.cuda.o  fused_bias_act.o  fused.so

I encountered a new problem when I tried to use _import_module_from_library to load the compiled module fused, I changed the beginning code in fused_act.py to:

try:
    user_home_path = os.path.expanduser('~')
    fused = _import_module_from_library('fused', user_home_path+'/.cache/torch_extensions/fused', True)
    print(f'Load fused from {user_home_path}/.cache/torch_extensions/fused')
    print("Load success!")
except:
    module_path = os.path.dirname(__file__)
    fused = load(
        name='fused',
        sources=[
            os.path.join(module_path, 'fused_bias_act.cpp'),
            os.path.join(module_path, 'fused_bias_act_kernel.cu'),
        ],
        verbose=True
    )
    print(f'Load function used. Build fused from cpp & cu files')

Executing the code step by step will not report an error, but when I type quit (), the command line will report an error Segmentation fault (Core dumped).
My gdb traceback content shows:

(gdb) run fused_act.py
Starting program: /data4/yuanWang/anaconda3/envs/test4torch/bin/python fused_act.py
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7fff7d446700 (LWP 4153)]
Load fused from /data4/yuanWang/.cache/torch_extensions/fused
Load success!

Thread 1 "python" received signal SIGSEGV, Segmentation fault.
0x00007fffdeafa9b8 in ?? ()
   from /data4/yuanWang/anaconda3/envs/test4torch/lib/python3.7/site-packages/torch/lib/../../../../libcudart.so.10.1
(gdb) backtrace
#0  0x00007fffdeafa9b8 in ?? ()
   from /data4/yuanWang/anaconda3/envs/test4torch/lib/python3.7/site-packages/torch/lib/../../../../libcudart.so.10.1
#1  0x00007fffdeafb1a3 in ?? ()
   from /data4/yuanWang/anaconda3/envs/test4torch/lib/python3.7/site-packages/torch/lib/../../../../libcudart.so.10.1
#2  0x00007fffdeafb8a5 in ?? ()
   from /data4/yuanWang/anaconda3/envs/test4torch/lib/python3.7/site-packages/torch/lib/../../../../libcudart.so.10.1
#3  0x00007ffff7806031 in __run_exit_handlers (status=0, 
    listp=0x7ffff7bae718 <__exit_funcs>, 
    run_list_atexit=run_list_atexit@entry=true, run_dtors=run_dtors@entry=true)
    at exit.c:108
#4  0x00007ffff780612a in __GI_exit (status=<optimized out>) at exit.c:139
#5  0x00007ffff77e4c8e in __libc_start_main (main=0x555555645ab0 <main>, argc=2, 
    argv=0x7fffffffe158, init=<optimized out>, fini=<optimized out>, 
    rtld_fini=<optimized out>, stack_end=0x7fffffffe148)
    at ../csu/libc-start.c:344
#6  0x000055555572b73d in _start () at ../sysdeps/x86_64/elf/start.S:103

Do you know how to solve this problem? By the way, do you choose to use Windows as the operating system to run code? I strongly suspect that this problem is related to the Ubuntu operating system. Thank you for your time.
Best wishes.

ninja: build stopped

I ran this code

bash scripts/train_photosketch_horse_riders.sh

There is an error.
Is there a solution?

/home/a/GANSketching/training/networks/op/fused_bias_act.cpp:13:5: note: in expansion of macro ‘CHECK_CUDA’ CHECK_CUDA(input); ^ ninja: build stopped: subcommand failed.
python 3.6
ninja 1.10.2.3
CUDA 8.0

CUSTOM C++ AND CUDA EXTENSIONS

Thank you for providing the code.

Please let me know if possible.

When I run generate sample script, but processing stop at line 11 of fused_act.py

torch.utils.cpp_extension.load

What are the possible causes?

Library version

• torch 1.9.0
• cuda 10.2

How to transform my own sketch to latent z ?

Hi! Glad to see your work!
But I have a question, as follows.

Consider practical usage:
step 1: I make a cat sketch image by hand.
step 2: transform the sketch image to latent_z.
step 3: feed latent_z to netG network to get a cat image.

I am surprised how to realize step2 ?
Do you mean that I need netG, photo2Sketch network, and use pix2latent method? Or only need netG and use pix2latent method?
If I just use netG to get z, it will still generate the cat sketch, but not cat image. Is not it ?

Thanks!

Training process

I was trying to train for the first time using the example:

"python train.py --name cat_train_5 --batch 16 --dataroot_sketch ./data/sketch/by_author/cat --dataroot_image ./data/image/cat --l_image 0.7 - -g_pretrained ./pretrained/stylegan2-cat/netG.pth --d_pretrained ./pretrained/stylegan2-cat/netD.pth --max_iter 150000 --disable_eval --diffaug_policy translation --no_wandb"

And everything seems to be fine:

Using pretrained weight for D1...
Using pretrained weight for D_image...
----------------- Options ---------------
batch: 16 [default: 4]
beta1: 0.0
beta2: 0.99
channel_multiplier: 2
checkpoints_dir: checkpoint
d_pretrained: ./pretrained/stylegan2-cat/netD.pth [default: ]
d_reg_every: 16
dataroot_image: ./data/image/cat [default: None]
dataroot_sketch: ./data/sketch/by_author/cat [default: None]
diffaug_policy: translation [default: ]
disable_eval: True [default: False]
display_freq: 2500
display_winsize: 400
dsketch_no_pretrain: False
eval_batch: 50
eval_dir: None
eval_freq: 5000
g_pretrained: ./pretrained/stylegan2-cat/netG.pth [default: ]
gan_mode: softplus
isTrain: True [default: None]
l_image: 0.7 [default: 0]
l_weight: 0
latent_avg_samples: 8192
lr: 0.002
lr_mlp: 0.01
max_epoch: 1000000
max_iter: 150000 [default: 75001]
mixing: 0.9
n_mlp: 8
name: cat_train_5 [default: None]
no_d_regularize: False
no_html: False
no_wandb: True [default: False]
optim_param_g: style
photosketch_path: ./pretrained/photosketch.pth
print_freq: 100
r1: 10
reduce_visuals: False
resume_iter: None
save_freq: 2500
size: 256
sketch_channel: 1
transform_fake: toSketch,to3ch
transform_real: to3ch
use_cpu: False
z_dim: 512
----------------- End -------------------

-------------- Trainables ---------------
(G trainable parameters)
style.1.weight
style.1.bias
style.2.weight
style.2.bias
style.3.weight
style.3.bias
style.4.weight
style.4.bias
style.5.weight
style.5.bias
style.6.weight
style.6.bias
style.7.weight
style.7.bias
style.8.weight
style.8.bias
----------------- End -------------------
create web directory checkpoint\cat_train_5\web...
Training was successfully finished.

I expected that the weights would be generated in the checkpoint folder and then used to generate the images, but I only found two files "ops" and "log_loss" which is empty.

I am running this on Windows 10.

The process is correct? have I forgotten something? Could it be that the problem is OS?

I'm new working with gans, so any help will be appreciated.

The importError when runing the generate sample script

When I run generate sample script,there is a importError "ImportError: /home/dzl/.cache/torch_extensions/py38_cu113/fused/fused.so: cannot open shared object file: No such file or directory"

Library version:

python 3.8.12
torch 1.10.1
cuda 11.3
ninja 1.10.2

could you tell me what your library version?
thank you

Problems about fused_act.py. TypeError: bump_version_if_changed() got an unexpected keyword argument 'is_python_module' and subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

When I was running fused_act.py, I got a bug about TypeError. I am using a Linux server without root and sudo permission, and my virtual environment in conda is: Python 3.7, pytorch 1.7.1, torchvision 0.8.2, cudatoolkit V10.1.243, gcc version 7.5.0 (Ubuntu 7.5.0-3ubuntu1~18.04) and ninja 1.10.2. I used conda install -c conda-forge ninja to install ninja but not sure about ninja's working status.
I already changed line 1631 in myName/anaconda3/envs/test1/lib/python3.7/site-packages/torch/utils/cpp_extension.py, from ['ninja', '-v'] to ['ninja', '-version'] to avoid subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status. After that, I got the TypeError problem. Could anybody help me please?
Logs in detail:
Traceback (most recent call last): (in virtual environment pytorch 1.7.1)
File "fused_act.py", line 15, in
os.path.join(module_path, "fused_bias_act_kernel.cu"),
File "/data4/yuanWang/anaconda3/envs/test3/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1091, in load
keep_intermediates=keep_intermediates)
File "/data4/yuanWang/anaconda3/envs/test3/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1269, in _jit_compile
is_standalone=is_standalone,
TypeError: bump_version_if_changed() got an unexpected keyword argument 'is_python_module'

datasets

hello，I‘m very glad to see the project，Could you share your dataset？

loss value

Follow the script you provided, my training losses fluctuate wildly, unable to convergence. Will your training losses be significantly reduced?

A TypeError occurred while running training scripts.

When I was running bash scripts/train_teaser_cat.sh in my Linux server, I got a TypeError. My virtual environment in conda already installed: Python 3.7.12, pytorch 1.7.1, cudatoolkit 10.1.243, torchvision 0.8.2, and ninja 1.10.2. I already tried higher version of pytorch 1.8.1, but it has another problem of subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status. So I returned to the pytorch 1.7.1 environment. I have been troubled by the TypeError problem for several days. Can someone help me solve it? Thanks.

Logs in detail:

(test3) yuanWang@server-TiTan:~/GANtest1$ bash scripts/train_teaser_cat.sh
Traceback (most recent call last):
  File "train.py", line 6, in <module>
    from eval import Evaluator
  File "/data4/yuanWang/GANtest1/eval/__init__.py", line 1, in <module>
    from .evaluation import Evaluator
  File "/data4/yuanWang/GANtest1/eval/evaluation.py", line 7, in <module>
    from run_metrics import get_vgg_features, make_eval_images
  File "/data4/yuanWang/GANtest1/run_metrics.py", line 13, in <module>
    from training.networks.stylegan2 import Generator
  File "/data4/yuanWang/GANtest1/training/networks/__init__.py", line 2, in <module>
    from .misc import *
  File "/data4/yuanWang/GANtest1/training/networks/misc.py", line 1, in <module>
    from . import stylegan2
  File "/data4/yuanWang/GANtest1/training/networks/stylegan2.py", line 8, in <module>
    from .op import FusedLeakyReLU, fused_leaky_relu, upfirdn2d
  File "/data4/yuanWang/GANtest1/training/networks/op/__init__.py", line 1, in <module>
    from .fused_act import FusedLeakyReLU, fused_leaky_relu
  File "/data4/yuanWang/GANtest1/training/networks/op/fused_act.py", line 15, in <module>
    os.path.join(module_path, "fused_bias_act_kernel.cu"),
  File "/data4/yuanWang/anaconda3/envs/test3/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1091, in load
    keep_intermediates=keep_intermediates)
  File "/data4/yuanWang/anaconda3/envs/test3/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1269, in _jit_compile
    is_standalone=is_standalone,
TypeError: bump_version_if_changed() got an unexpected keyword argument 'is_python_module'

system

Is it possible to run the sample code under windows10 system?

The GPU Requirement

My GPU is GTX 2060 super，Can it run this program？If can‘t，could you tell me what is the minimum requirement for gpu？

Is it possible to generate higher resolution images e.g. 1024X1024

Hello, Excuse me，Is it possible to generate higher resolution images e.g. 1024X1024

When sampling, the generated images are all pure gray

use
python generate.py --ckpt weights/photosketch_standing_cat_noaug.pth --save_dir output/samples_standing_cat
get pure gray images

Pretrained stylegan2 models

Thanks for sharing this repo. I would like to train my own sketchgan for generating pictures of vehicles. Could you provide the pytorch pretrained stylegan2-car (and train, bus etc.) netD and netG model? Thanks.

Error(s) in loading state_dict for Generator

I can run the first two samples to generate the cat image,
but when I run the third sample command, I get an error with the message:

 File "generate.py", line 93, in <module>
    netG.load_state_dict(checkpoint)
  File "D:\Code\Anaconda\envs\test\lib\site-packages\torch\nn\modules\module.py", line 1407, in load_state_dict
    self.__class__.__name__, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for Generator:
        Unexpected key(s) in state_dict: "convs.12.conv.weight", "convs.12.conv.blur.kernel", "convs.12.conv.modulation.weight", "convs.12.conv.modulation.bias", "convs.12.noise.weight      netG.load_state_dict(checkpoint)

```

Google Colab for Examples

Not exactly an issue.

I created a Google Colab Notebook that sets up the repo and calls the example commands in the readme.
I did this because of issues I was having on my local setup. Hopefully this could be added to give an easier quick start for those interested in the project.

Colab Link

ImportError: No module named 'fused'

Hi Wang,

Thanks for sharing your code.

I have installed all the prerequisite packages. When I try to compile the module 'fused', errors below occur. I removed all cache file in torch_extensions folder by using rm -rf before every building tests but still got the same error. By the way I already changed '['ninja', '-v']' to '['ninja', '--version']' in cpp_extension.py under anaconda3/envs/CTtest1/lib/python3.7/site-packages/torch/utils/ to avoid another error. Could you please tell me how to fix it?

Detected CUDA files, patching ldflags
Emitting ninja build file /home/slhua/.cache/torch_extensions/fused/build.ninja...
Building extension module fused...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
1.7.2
Loading extension module fused...
Traceback (most recent call last):
  File "fused_act.py", line 25, in <module>
    is_python_module = True)
  File "/home/slhua/anaconda3/envs/CTtest1/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 997, in load
    keep_intermediates=keep_intermediates)
  File "/home/slhua/anaconda3/envs/CTtest1/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1213, in _jit_compile
    return _import_module_from_library(name, build_directory, is_python_module)
  File "/home/slhua/anaconda3/envs/CTtest1/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1560, in _import_module_from_library
    file, path, description = imp.find_module(module_name, [path])
  File "/home/slhua/anaconda3/envs/CTtest1/lib/python3.7/imp.py", line 296, in find_module
    raise ImportError(_ERR_MSG.format(name), name=name)
ImportError: No module named 'fused'

My environment:
cudatoolkit 10.2.89, cudnn 7.6.5, python 3.7.6, pytorch 1.7.1, torchvision 0.8.2, ninja 1.7.2

Is it possible to use models of StyleGan2-ada?

Thanks for sharing! And do you know how can I use Stylegan2-ada to replace Stylgan2's model in your code? Does the code support it?

running Evaluations without GPU and CUDA

Hi Thank you for providing the code,

I am trying to run the evaluation's on Windows without GPU and CUDA support. when i run the the python file

python run_metrics.py --models_list weights/eval_list --output metric_results.csv --device=cpu

Traceback (most recent call last):
File "run_metrics.py", line 12, in
from eval.ppl import compute_ppl
File "C:\Users\Downloads\Test\eval_init.py", line 1, in
from .evaluation import Evaluator
File "C:\Users\Downloads\Test\eval\evaluation.py", line 7, in
from run_metrics import get_vgg_features, make_eval_images
File "C:\Users\Downloads\Test\run_metrics.py", line 13, in
from training.networks.stylegan2 import Generator
File "C:\Users\Downloads\Test\training\networks_init_.py", line 2, in
from .misc import *
File "C:\Users\Downloads\Test\training\networks\misc.py", line 1, in
from . import stylegan2
File "C:\Users\Downloads\Test\training\networks\stylegan2.py", line 8, in
from .op import FusedLeakyReLU, fused_leaky_relu, upfirdn2d
File "C:\Users\Downloads\Test\training\networks\op_init_.py", line 1, in
from .fused_act import FusedLeakyReLU, fused_leaky_relu
File "C:\Users\Downloads\Test\training\networks\op\fused_act.py", line 15, in
os.path.join(module_path, "fused_bias_act_kernel.cu"),
File "C:\Users\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\utils\cpp_extension.py", line 1091, in load
keep_intermediates=\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\utils\cpp_extension.py", line 1302, in _jit_compile
is_standalone=is_standalone)
File "C:\Users\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\utils\cpp_extension.py", line 1385, in _write_ninja_file_and_build_library
is_standalone)
File "C:\Users\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\utils\cpp_extension.py", line 1484, in _prepare_ldflags
extra_ldflags.append(f'/LIBPATH:{_join_cuda_home("lib/x64")}')
File "C:\Users\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\utils\cpp_extension.py", line 1982, in join_cuda_home
raise EnvironmentError('CUDA_HOME environment variable is not set. '
**OSError: CUDA_HOME environment variable is not set. Please set it to your CUDA install root.**

Is CUDA and GPU is mandatory to run the evaluation ?

Evaluation FID in trained model doesn't match the one from the pre-trained model

Hello, thank you for the code available,

I executed the training script:

bash scripts/train_photosketch_horse_riders.sh

and then I compared the metrics of the trained model with those of the file:

weights/photosketch_horse_riders_aug.pth

using the evaluation script:

run_metrics.py

with the command:

python run_metrics.py --models_list weights/eval_list --output metric_results.csv

I added the trained weights into the weights folder and into the eval_list file so that I could check its FID too.

I very different FIDs for the photosketch_horse_riders_aug.pth (FID ~20.13) and the trained one (FID ~37.07). I assumed that the training script would generate a model similar to the one stored at photosketch_horse_riders_aug.pth.

There is some other different procedure to get a trained model with the same FID as photosketch_horse_riders_aug.pth, which is very close but not quite the same as in the paper (FID ~19.94)?

How to generate using your own sketches?

Is it possible to generate cats by providing my own sketches? Looking at the code it seems that all sketches are hardcoded, but maybe I'm missing something.

How to further modify the generate model through GANspace

How can I use GANspace to adjust the image generated by the net_G? For example, the original generator can generate cats pictures indoor but I want the background to be outdoor grassland. Is such an operation possible? If possible, how can I operate through GANspace? Thank you very much for your help!

How to load the parameters of the latest model and continue training.

I found that there are codes in train.py that automatically saves the latest model after a certain number of iterations. When I stop training manually, how can I continue the unfinished training with the saved model? Thanks for your help！

how to convert sketch into fixed_z (.pth file) when running generate.py

I tried pix2latent to get fixed_z (.pth file), but met several problems:

1. after running invert_stylegan2_*.py, a .npy file is saved for variables, but how can I get a .pth file?
2. the repo for stylegan2 is for size 384x512, while my sketch is 256x256, thus it reports many errors for wrong dims. I don't know how to correct it.
   Thanks a lot and looking forward to responding.