ptran1203/pytorch-animegan

Pytorch implementation of AnimeGAN for fast photo animation

pytorch-animegan's Issues

Suggestions to convert animeGAN v1 to bryandlee / animegan2-pytorch

Since the v2 training code is unavailable from https://github.com/bryandlee/animegan2-pytorch, do you have any suggestions for improving on your wonderful pytorch v1 version here, for those that want the faster, smaller v2 model? V1 model outputs ~15mb, while the V2 pretrained models are ~8mb. Are there some tweaks I can make to your model to approach the V2 model?

What is the exact input shape of model?

I integrated/adapted your code to my pipeline. However, when I train the model, I get some strange results. I think that it comes from the input matrices of cartoons (edged/gray/normal)...

What are the expected cartoon and picture input shapes/order of channels ?

How to make smooth image， how to sematic source Image？

Please tell me how to make smooth image from source image， how to sematic source image to size？
Is there any python file to use?

Have you ever trained the data of face cartoonization or face animation?

how do you set args.bn and args.d_noise

Very good job

The code can indeed run and the effect is very good. I don’t understand why there are so few people star this project. I hope more and more people can pay attention to this.

Unzip anime-gan.zip failed

Would you mind sharing your training dataset? The dataset I downloaded with wget -O anime-gan.zip https://github.com/ptran1203/pytorch-animeGAN/releases/download/v1.0/dataset_v1.zip cannot be decompressed.

unzip: cannot find zipfile directory in one of anime-gan.zip or anime-gan.zip.zip, and cannot find anime-gan.zip.ZIP, period.

Please create the full model (weights plus architecture) to torch.hub.load

Thank you for your amazing work! If your team is able to upload the full model (weights plus architecture) to torch.hub.load then the model could be converted to a mobile version. Or do you already have the pretrained model weights plus architecture file? I can see that your .pth is only the OrderedStrict weights.

How much epoch do it need to run？

I have run 100 epoch for shikai dataset ，and adv loss is more and more larger。and the result is not real like shikai style and do it wrong to get result？

About reintroducing AnimeGAN

After reading the primitive paper and analyzing your code, I find that the generator network in your code is not the same as said in the paper, and your learning rates of G and D are not equal to that in their paper. Also weights of loss. Why? Is this based on your practical experience ? Thank you！

Where is the weight of the discriminator, please?

==== Train Config ====

dataset Hayao
data_dir /content/dataset
epochs 100
init_epochs 4
batch_size 6
checkpoint_dir {ckp_dir}
save_image_dir {save_img_dir}
gan_loss lsgan
resume GD
use_sn True
save_interval 1
debug_samples 0
lr_g 2e-05
lr_d 4e-05
init_lr 0.0001
wadvg 10.0
wadvd 10.0
wcon 1.5
wgra 3.0
wcol 30.0
d_layers 3
d_noise False

Init models...
Compute mean (R, G, B) from 1792 images
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1792/1792 [00:04<00:00, 373.49it/s]
Mean(B, G, R) of Hayao are [-4.4346958 -8.66591597 13.10061177]
Dataset: real 6656 style 1792, smooth 1792
G weight loaded
Could not load checkpoint, train from scratch [Errno 2] No such file or directory: '{ckp_dir}/discriminator_Hayao.pth'
Epoch 13/100
0%| | 0/1110 [00:06<?, ?it/s]
Traceback (most recent call last):
File "train.py", line 247, in
main(args)
File "train.py", line 191, in main
fake_img = G(img).detach()
File "/root/miniconda3/envs/AGAN/lib/python3.6/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/root/autodl-tmp/pytorch-animeGAN-master/modeling/anime_gan.py", line 55, in forward
out = self.encode_blocks(x)
File "/root/miniconda3/envs/AGAN/lib/python3.6/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/root/miniconda3/envs/AGAN/lib/python3.6/site-packages/torch/nn/modules/container.py", line 119, in forward
input = module(input)
File "/root/miniconda3/envs/AGAN/lib/python3.6/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/root/autodl-tmp/pytorch-animeGAN-master/modeling/conv_blocks.py", line 73, in forward
out = self.ins_norm(out)
File "/root/miniconda3/envs/AGAN/lib/python3.6/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/root/miniconda3/envs/AGAN/lib/python3.6/site-packages/torch/nn/modules/instancenorm.py", line 59, in forward
self.training or not self.track_running_stats, self.momentum, self.eps)
File "/root/miniconda3/envs/AGAN/lib/python3.6/site-packages/torch/nn/functional.py", line 2183, in instance_norm
input, weight, bias, running_mean, running_var, use_input_stats, momentum, eps, torch.backends.cudnn.enabled
RuntimeError: CUDA error: no kernel image is available for execution on the device

Problems with resume trainning

Hi!

I'm facing a issue when i have to resume the trainning, even when I reach 80+ epochcs, the transformer has no effect on the images with the trained weights. I tried Hayao and other custom datasets, here are images with 15 epochs (without resume), 30, 40, 60 and 80 epochs with Hayao Dataset, and the changes are barely visible, if existent, I'm using a copy of the Google Collab Notebook.

Is there anything I'm doing wrong with the process? Here are the params used for trainning:

!python3 train.py --dataset 'Hayao'
--batch 6
--debug-samples 0
--init-epochs 10
--epochs 100 (Changed at each trainning resume)
--checkpoint-dir {ckp_dir}
--save-image-dir {save_img_dir}
--save-interval 1
--gan-loss lsgan
--init-lr 0.0001
--lr-g 0.00002
--lr-d 0.00004
--wadvd 10.0
--wadvg 10.0
--wcon 1.5
--wgra 3.0
--wcol 70.0
--resume GD
--use_sn\

Is this a problem with the Hayao dataset?

Init models...
Compute mean (R, G, B) from 1793 images
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▊| 1791/1793 [00:06<00:00, 281.65it/s]
Traceback (most recent call last):
File "train.py", line 247, in
main(args)
File "train.py", line 124, in main
AnimeDataSet(args),
File "/root/autodl-tmp/pytorch-animeGAN-master/dataset.py", line 32, in init
self.mean = compute_data_mean(os.path.join(anime_dir, 'style'))
File "/root/autodl-tmp/pytorch-animeGAN-master/utils/image_processing.py", line 114, in compute_data_mean
total += image.mean(axis=(0, 1))
AttributeError: 'NoneType' object has no attribute 'mean'

about license

Hello. Thank you for good documentation.
can i ask if what is this repository's license is? is it follows animeGAN?

G_loss Label is 1

pytorch-animeGAN/modeling/losses.py

Line 111 in a1642db

return self.bce_loss(pred, torch.zeros_like(pred))

G_loss label : torch.ones_like(pred) or torch.zeros_like(pred) ?

Is this a problem with the image itself or with the code?

Init models.. .
Compute mean (R,G，B)from 1792 images100%|[
Mean(B，G， R)of Hayao are [-4.4346958-8.6659159713.10061177]Dataset: real 9 style 1792,smooth 1793
Epoch 0/100
O%|
Traceback (most recent call last):
File "train. py"， line 247,in
main(args)
File "train. py"，line 162,in mainfor img,*_ in bar:
File "/root/miniconda3/envs/AGAN/lib/python3.8/site-packages/tqdm/
/ tadm. py"，line 919,in _iter
for obj in iterable:
File"/root/miniconda3/envs/AGAV/1ib/python3.8/site-packages/torch/utils/data/dataloader.py"，line 517，in_nextdata = self._next_data()
File "/root/minicandla3/envs/AGNV1ib/python3.8/site packages/torch/utils/data/dataloader. py"，line 1199，in _next_datareturn self. _process_data(data)
File "/rot/minicomda83/envs /ACGNV/lib/python3.8/site-packages/torch/utils/data/dataloader.py"，line 1225,in _process_datadata.reraise()
File "/root/miniconda3/envs/AGAV/1ib/python3.8/site packages/torch/_utils.py"，line 429,in reraiseraise self.exc_type(msg)
TypeError: Caught TypeError in DataLoader worker process 0.Original Traceback (most recent call last):
File "/root/miniconda3/envs/AGNV1ib/python3.8/site-packages/torch/utils/data/l_utils/worker.py"，line 202,in _worker_loopdata = fetcher.fetch(index)
File"/root/miniconda3/envs/AGAV1ib/python3.8/site packages/torch/utils/data/_utils/fetch. py"，line 44，in fetchdata = [self.dataset[idx] for idx in possibly_batched_index]
File "/root/miniconda3/envs/AGAV/1ib/python3.8/site packages/torch/uti1s/data/_utils/fetch.py"， line 44，in data = [self.dataset[idx] for idx in possibly_batched_index]
File "/root/autodl-tmp/pytorch-animeGAN-master/dataset.py"，line 67,in _getitemimage = self.load photo(index)
File "/root/autodl-tmp/pytorch-animeGAN-master/dataset.py"，line 79, in load photoimage = cv2.imread(fpath)[:,:,::-1]
TypeError: 'NoneType’object is not subscriptable

Training triggers cuda out of memory

Training triggers cuda out of memory, any workaround available?

murugan86@murugan86-IdeaPad-Gaming3-15ARH05D:~/anime/AnimeGenDir/animagen-pytorch-mur/pytorch-animeGAN$ python3 train.py --dataset Kimetsu --batch 6 --init-epochs 4 --checkpoint-dir {ckp_dir} --save-image-dir {save_img_dir} --save-interval 1 --gan-loss lsgan --init-lr 0.0001 --lr-g 0.00002 --lr-d 0.00004 --wadvd 10.0 --wadvg 10.0 --wcon 1.5 --wgra 3.0 --wcol 30.0

==== Train Config ====

dataset Kimetsu
data_dir /content/dataset
epochs 100
init_epochs 4
batch_size 6
checkpoint_dir {ckp_dir}
save_image_dir {save_img_dir}
gan_loss lsgan
resume False
use_sn False
save_interval 1
debug_samples 0
lr_g 2e-05
lr_d 4e-05
init_lr 0.0001
wadvg 10.0
wadvd 10.0
wcon 1.5
wgra 3.0
wcol 30.0
d_layers 3
d_noise False

• {save_img_dir} does not exist, creating...
• {ckp_dir} does not exist, creating...
  Init models...
  Downloading: "https://download.pytorch.org/models/vgg19-dcbb9e9d.pth" to /home/murugan86/.cache/torch/hub/checkpoints/vgg19-dcbb9e9d.pth
  100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 548M/548M [01:13<00:00, 7.80MB/s]
  Compute mean (R, G, B) from 1800 images
  100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1800/1800 [00:03<00:00, 496.33it/s]
  Mean(B, G, R) of Kimetsu are [-2.76436356 0.35512874 2.40923482]
  Dataset: real 6656 style 1800, smooth 1800
  Epoch 0/100
  0%| | 0/1110 [00:00<?, ?it/s]/home/murugan86/.local/lib/python3.8/site-packages/torch/nn/functional.py:3060: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
  warnings.warn("Default upsampling behavior when mode={} is changed "
  /home/murugan86/.local/lib/python3.8/site-packages/torch/nn/functional.py:3103: UserWarning: The default behavior for interpolate/upsample with float scale_factor changed in 1.6.0 to align with other frameworks/libraries, and now uses scale_factor directly, instead of relying on the computed output size. If you wish to restore the old behavior, please set recompute_scale_factor=True. See the documentation of nn.Upsample for details.
  warnings.warn("The default behavior for interpolate/upsample with float scale_factor changed "
  0%| | 0/1110 [00:01<?, ?it/s]
  Traceback (most recent call last):
  File "train.py", line 247, in
  main(args)
  File "train.py", line 167, in main
  fake_img = G(img)
  File "/home/murugan86/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
  result = self.forward(*input, **kwargs)
  File "/home/murugan86/anime/AnimeGenDir/animagen-pytorch-mur/pytorch-animeGAN/modeling/anime_gan.py", line 56, in forward
  out = self.res_blocks(out)
  File "/home/murugan86/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
  result = self.forward(*input, **kwargs)
  File "/home/murugan86/.local/lib/python3.8/site-packages/torch/nn/modules/container.py", line 117, in forward
  input = module(input)
  File "/home/murugan86/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
  result = self.forward(*input, **kwargs)
  File "/home/murugan86/anime/AnimeGenDir/animagen-pytorch-mur/pytorch-animeGAN/modeling/conv_blocks.py", line 98, in forward
  out = self.ins_norm1(out)
  File "/home/murugan86/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
  result = self.forward(*input, **kwargs)
  File "/home/murugan86/.local/lib/python3.8/site-packages/torch/nn/modules/instancenorm.py", line 55, in forward
  return F.instance_norm(
  File "/home/murugan86/.local/lib/python3.8/site-packages/torch/nn/functional.py", line 2078, in instance_norm
  return torch.instance_norm(
  RuntimeError: CUDA out of memory. Tried to allocate 48.00 MiB (GPU 0; 3.82 GiB total capacity; 2.57 GiB already allocated; 30.75 MiB free; 2.57 GiB reserved in total by PyTorch)

Error in Video transform self.proc.stdin.write(img_array.tobytes()) ValueError: write to closed file

def video():
os.system("python inference_video.py "
"--checkpoint generator_hayao.pth "
"--src C:/Zero/Python/AnimeGAN/test.mp4 "
"--dest res.mp4 "
"--batch 2")

When I run this, got the error above, I have print the val of img it is correct I think sth wrong with the writer

Why do not use gradient_panalty?

Why do not use gradient_panalty, can it be train faster?

About the pretrained model

I would like to know whether you have released the pretrained model for inference?