xuezhemax/wolf

Invertible Generative Flows

wolf's Issues

CelebHQ checkpoint

Hi @XuezheMax ,

Thanks for your nice works,
for the "MaCow" paper, would you please share checkpoints for "CelebHQ dataset"?
to have "Celeb-HQ" images, I have used "CelebAMask-HQ" dataset which has 2993 validation and 2824 test samples. is that correct?

Unable to train getting this error: "TypeError: ActNorm1dFlow.forward: kwargs is not present."

Hello!

I am getting an error TypeError: ActNorm1dFlow.forward: kwargs is not present.

This is still occurs regardless of config file or the dataset being used.
Still happens after clean installs of the packages and also when trying the previous MaCow repository.
How can I resolve this?

Thank you for the help!!!

Terminal:
 (wolf) (base) james@LXMint:/media/james/LinuxStorage/Bayes/wolf/experiments$ python -u train.py \
 >     --config  configs/cifar10/macow/macow-base-var.json \
 >     --epochs 15000 --valid_epochs 10 \
 >     --batch_size 512 --batch_steps 2 --eval_batch_size 1000 --init_batch_size 2048 \
 >     --lr 0.001 --beta1 0.9 --beta2 0.999 --eps 1e-8 --warmup_steps 50 --weight_decay 1e-6 --grad_clip 0 \
 >     --image_size 32 --n_bits 8 \
 >     --data_path data/cifar10 --model_path models

Traceback (most recent call last):
  File "train.py", line 23, in <module>
    from wolf.data import load_datasets, get_batch, preprocess, postprocess
  File "/media/james/LinuxStorage/Bayes/wolf/wolf/__init__.py", line 3, in <module>
    from wolf.wolf import WolfModel
  File "/media/james/LinuxStorage/Bayes/wolf/wolf/wolf.py", line 12, in <module>
    from wolf.modules import DeQuantizer
  File "/media/james/LinuxStorage/Bayes/wolf/wolf/modules/__init__.py", line 3, in <module>
    from wolf.modules.dequantization import *
  File "/media/james/LinuxStorage/Bayes/wolf/wolf/modules/dequantization/__init__.py", line 3, in <module>
    from wolf.modules.dequantization.dequantizer import DeQuantizer, UniformDeQuantizer, FlowDeQuantizer
  File "/media/james/LinuxStorage/Bayes/wolf/wolf/modules/dequantization/dequantizer.py", line 9, in <module>
    from wolf.flows.flow import Flow
  File "/media/james/LinuxStorage/Bayes/wolf/wolf/flows/__init__.py", line 4, in <module>
    from wolf.flows.normalization import ActNorm1dFlow, ActNorm2dFlow
  File "/media/james/LinuxStorage/Bayes/wolf/wolf/flows/normalization.py", line 13, in <module>
    class ActNorm1dFlow(Flow):
  File "/media/james/LinuxStorage/Bayes/wolf/wolf/flows/normalization.py", line 26, in ActNorm1dFlow
    def forward(self, input: torch.Tensor, mask: Union[torch.Tensor, None] = None) -> Tuple[torch.Tensor, torch.Tensor]:
  File "/media/james/LinuxStorage/Bayes/wolf/lib/python3.7/site-packages/overrides/overrides.py", line 88, in overrides
    return _overrides(method, check_signature, check_at_runtime)
  File "/media/james/LinuxStorage/Bayes/wolf/lib/python3.7/site-packages/overrides/overrides.py", line 114, in _overrides
    _validate_method(method, super_class, check_signature)
  File "/media/james/LinuxStorage/Bayes/wolf/lib/python3.7/site-packages/overrides/overrides.py", line 135, in _validate_method
    ensure_signature_is_compatible(super_method, method, is_static)
  File "/media/james/LinuxStorage/Bayes/wolf/lib/python3.7/site-packages/overrides/signature.py", line 95, in ensure_signature_is_compatible
    super_sig, sub_sig, super_type_hints, sub_type_hints, is_static, method_name
  File "/media/james/LinuxStorage/Bayes/wolf/lib/python3.7/site-packages/overrides/signature.py", line 136, in ensure_all_kwargs_defined_in_sub
    raise TypeError(f"{method_name}: `{name}` is not present.")
TypeError: ActNorm1dFlow.forward: `kwargs` is not present.

Number of Epochs

Hi,

I am trying to run the cifar10 example in the README file. The command line arguments there specify 15000 as the number of epochs. How important is it to train the model for that many epochs? In other words, what is the minimum number of epochs to train for and still get reasonable results? Based on the speed I am seeing so far, it would take my system (with a single GPU) at least 7 weeks to finish 15000 epochs.

Thanks!

RuntimeError: Output 1 of SplitBackward is a view and is being modified inplace.

Thanks for great repository. I got an error while testing your code. Why it happens?
The versions of pytorch and cuda are 1.9.1 and 11.1, respectively.

Rank -1: random seed=65537
Rank -1: Namespace(amsgrad=False, batch_size=512, batch_steps=2, beta1=0.9, beta2=0.999, category=None, checkpoint_name='test_cifar10/checkpoint', config='configs/cifar10/macow/macow-base-var.json', cuda=True, data_path='../../../data/cifar-10', dataset='cifar10', epochs=15000, eps=1e-08, eval_batch_size=1000, grad_clip=0.0, image_size=32, init_batch_size=2048, local_rank=0, log=<io.TextIOWrapper name='test_cifar10/log.txt' mode='w' encoding='UTF-8'>, log_interval=10, lr=0.001, lr_decay=0.999997, model_path='test_cifar10', n_bins=256.0, n_bits=8, nx=3072, rank=-1, recover=-1, result_path='test_cifar10/images', seed=65537, test_k=5, train_k=1, valid_epochs=10, warmup_steps=50, weight_decay=1e-06, workers=4, world_size=1)
Files already downloaded and verified
Data size: training: 50000, val: 10000
of Parameters: 51046555
Rank -1, init model: 2048 instances
Epoch: 1 (lr=0.000000, betas=(0.9, 0.999), eps=1.0e-08, amsgrad=False, lr decay=0.999997, clip=0.0, l2=1.0e-06, train_k=1)
Traceback (most recent call last):
File "train.py", line 506, in
main(args)
File "train.py", line 500, in main
train(args, train_loader, train_index, train_sampler, val_loader, val_data, val_index, wolf)
File "train.py", line 334, in train
loss_gen, loss_kl, loss_dequant = wolf.loss(data, y=y, n_bits=n_bits, nsamples=train_k)
File "/root/share/NF/wolf/wolf/wolf.py", line 242, in loss
return core(data, y=y, n_bits=n_bits, nsamples=nsamples)
File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1051, in call_impl
return forward_call(*input, **kwargs)
File "/root/share/NF/wolf/wolf/wolf.py", line 78, in forward
u, log_probs_dequant = self.dequantizer.dequantize(x, nsamples=nsamples)
File "/root/share/NF/wolf/wolf/modules/dequantization/dequantizer.py", line 93, in dequantize
u, logdet = self.flow.fwdpass(epsilon, h)
File "/root/share/NF/wolf/wolf/flows/flow.py", line 73, in fwdpass
return self.forward(x, *h, **kwargs)
File "/root/share/NF/wolf/wolf/modules/dequantization/dequantizer.py", line 134, in forward
out, logdet_accum = self.core.forward(input, h=h)
File "/root/share/NF/wolf/wolf/flows/multiscale_architecture.py", line 301, in forward
out, logdet = block.forward(out, h=h)
File "/root/share/NF/wolf/wolf/flows/multiscale_architecture.py", line 102, in forward
out, logdet = step.forward(out, h=h)
File "/root/share/NF/wolf/wolf/flows/macow.py", line 152, in forward
out, logdet = unit.forward(out, h=h)
File "/root/share/NF/wolf/wolf/flows/macow.py", line 42, in forward
out, logdet_accum = self.conv1.forward(input, h=h)
File "/root/share/NF/wolf/wolf/flows/couplings/coupling.py", line 485, in forward
params = self.transform.calc_params(self.calc_params(input, h=h))
File "/root/share/NF/wolf/wolf/flows/couplings/transform.py", line 58, in calc_params
scale = log_scale.mul(0.5).tanh().mul(self.alpha).add(1.0)
RuntimeError: Output 1 of SplitBackward is a view and is being modified inplace. This view is an output of a function that returns multiple views. Inplace operators on such views is forbidden. You should replace the inplace operation by an out-of-place one.