rnwzd/fspbt-image-translation

"Interactive Video Stylization Using Few-Shot Patch-Based Training" by O. Texler et al. in PyTorch Lightning

fspbt-image-translation's Issues

Some questions

Hello, after several days of exploration, I still made some mistakes. I'd like to ask why you downloaded the CPU version of pytorch (conda install pytorch torchvision torchaudio cpuonly -c pytorch). After entering "Python train.py" , there will still be a "raise MisconfigurationException("GPUs requested but none are available.")
pytorch_lightning.utilities.exceptions.MisconfigurationException: GPUs requested but none are available."
I use a notebook with dual graphics cards. which is tested with Ubuntu 20.04. Is there an error in the pytorch version I downloaded?
In addition, after entering "Python eval.py", “FileNotFoundError: [Errno 2] No such file or directory: 'data/webcam/models/generator.pt'”appears . generator.pt appears. Where can I find the file "generator.pt"? Or which step might have gone wrong? I hope to ask for advice
The code is as follows:

(base) aaa@aaa-virtual-machine:/桌面$ cd /home/aaa/下载/FSPBT-Image-Translation-master
(base) aaa@aaa-virtual-machine:/下载/FSPBT-Image-Translation-master$ conda activate FSPBT
(FSPBT) aaa@aaa-virtual-machine:/下载/FSPBT-Image-Translation-master$ python train.py
Traceback (most recent call last):
File "/home/aaa/下载/FSPBT-Image-Translation-master/train.py", line 41, in
trainer = pl.Trainer(
File "/home/aaa/anaconda3/envs/FSPBT/lib/python3.9/site-packages/pytorch_lightning/trainer/connectors/env_vars_connector.py", line 40, in insert_env_defaults
return fn(self, **kwargs)
File "/home/aaa/anaconda3/envs/FSPBT/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 346, in init
gpu_ids, tpu_cores = self._parse_devices(gpus, auto_select_gpus, tpu_cores)
File "/home/aaa/anaconda3/envs/FSPBT/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 1251, in _parse_devices
gpu_ids = device_parser.parse_gpu_ids(gpus)
File "/home/aaa/anaconda3/envs/FSPBT/lib/python3.9/site-packages/pytorch_lightning/utilities/device_parser.py", line 87, in parse_gpu_ids
raise MisconfigurationException("GPUs requested but none are available.")
pytorch_lightning.utilities.exceptions.MisconfigurationException: GPUs requested but none are available.
(FSPBT) aaa@aaa-virtual-machine:/下载/FSPBT-Image-Translation-master$ conda activate FSPBT
(FSPBT) aaa@aaa-virtual-machine:/下载/FSPBT-Image-Translation-master$ python eval.py
Traceback (most recent call last):
File "/home/aaa/下载/FSPBT-Image-Translation-master/eval.py", line 18, in
generator = torch.load(model_save_path/"generator.pt")
File "/home/aaa/anaconda3/envs/FSPBT/lib/python3.9/site-packages/torch/serialization.py", line 594, in load
with _open_file_like(f, 'rb') as opened_file:
File "/home/aaa/anaconda3/envs/FSPBT/lib/python3.9/site-packages/torch/serialization.py", line 230, in _open_file_like
return _open_file(name_or_buffer, mode)
File "/home/aaa/anaconda3/envs/FSPBT/lib/python3.9/site-packages/torch/serialization.py", line 211, in init
super(_open_file, self).init(open(name, mode))
FileNotFoundError: [Errno 2] No such file or directory: 'data/webcam/models/generator.pt'
(FSPBT) aaa@aaa-virtual-machine:/下载/FSPBT-Image-Translation-master$

Data and pre-trainied weights cannot be downloaded

It seems that this link https://drive.google.com/file/d/1WI71nYP-z0mfDpuUW36s3sswpwRwwfrN/view don't have files and download_data.sh script is something wrong.
would you please update these two links ? really thanks !

Additional tests

Hello, I have successfully tested the model and output the files in the "output" folder. If I want to test with some of my pictures, can I put some of my pictures in the "target" folder and run "eval. Py"? Or what kind of operation is needed?

Issue with tensor missizing

Hi there, my style images and video frames are the same size and in the data structure mentioned however I get the error:

Validation sanity check: 0% 0/1 [00:00<?, ?it/s]Traceback (most recent call last):
File "train.py", line 56, in
trainer.fit(model, dm)
File "/usr/local/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 553, in fit
self._run(model)
File "/usr/local/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 918, in _run
self._dispatch()
File "/usr/local/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 986, in _dispatch
self.accelerator.start_training(self)
File "/usr/local/lib/python3.7/site-packages/pytorch_lightning/accelerators/accelerator.py", line 92, in start_training
self.training_type_plugin.start_training(trainer)
File "/usr/local/lib/python3.7/site-packages/pytorch_lightning/plugins/training_type/training_type_plugin.py", line 161, in start_training
self._results = trainer.run_stage()
File "/usr/local/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 996, in run_stage
return self._run_train()
File "/usr/local/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1031, in _run_train
self._run_sanity_check(self.lightning_module)
File "/usr/local/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1115, in _run_sanity_check
self._evaluation_loop.run()
File "/usr/local/lib/python3.7/site-packages/pytorch_lightning/loops/base.py", line 111, in run
self.advance(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/pytorch_lightning/loops/dataloader/evaluation_loop.py", line 111, in advance
dataloader_iter, self.current_dataloader_idx, dl_max_batches, self.num_dataloaders
File "/usr/local/lib/python3.7/site-packages/pytorch_lightning/loops/base.py", line 111, in run
self.advance(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/pytorch_lightning/loops/epoch/evaluation_epoch_loop.py", line 110, in advance
output = self.evaluation_step(batch, batch_idx, dataloader_idx)
File "/usr/local/lib/python3.7/site-packages/pytorch_lightning/loops/epoch/evaluation_epoch_loop.py", line 154, in evaluation_step
output = self.trainer.accelerator.validation_step(step_kwargs)
File "/usr/local/lib/python3.7/site-packages/pytorch_lightning/accelerators/accelerator.py", line 211, in validation_step
return self.training_type_plugin.validation_step(*step_kwargs.values())
File "/usr/local/lib/python3.7/site-packages/pytorch_lightning/plugins/training_type/training_type_plugin.py", line 178, in validation_step
return self.model.validation_step(*args, **kwargs)
File "/content/FSPBT-Image-Translation/litmodels.py", line 105, in validation_step
output = self.forward(input)
File "/content/FSPBT-Image-Translation/litmodels.py", line 36, in forward
output = self.generator(input)
File "/usr/local/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/content/FSPBT-Image-Translation/original_models.py", line 142, in forward
output_u1 = self.upconv1(torch.cat((output_u2, output_d1), dim=1))
RuntimeError: Sizes of tensors must match except in dimension 2. Got 101 and 102 (The offending index is 0)

Im hoping you can help, thanks

crashes when I change precision

I want to change the value of precision to make the script run a little better but it crashes when i enter anything other then 16
normally it should have 32 , 64 , bf16 , mixed available too.
any way to have more options to change up the model besides epochs and val?
thanks :)

No such file or directory

Hi,Is the file missing？I got the following error：

Exception has occurred: FileNotFoundError
[Errno 2] No such file or directory: 'data/webcam/target'
File "/home/lina/style/FSPBT-Image/data.py", line 87, in
name_ls = [file.name for file in (
File "/home/lina/style/FSPBT-Image/data.py", line 87, in prepare_data
name_ls = [file.name for file in (
File "/home/lina/style/FSPBT-Image/train.py", line 29, in
train_image_dd = prepare_data(data_path)

License

Dear Midas,

thanks for implementing the paper in PyTorch Lightning!
We would like to reuse your code base for an art project in our local city, but could not find a license for it and we can't afford to risk legal disputes.
Is it possible for you to add one?

Kind regards,
Marenz

Unable to download test data

I cannot download the test data on Google Drive.
It would be great if you could make this one available for download!
https://drive.google.com/file/d/1WI71nYP-z0mfDpuUW36s3sswpwRwwfrN/view

Can you tell me more about the data for training?
The original paper has a folder structure like Zuzka1_Train and Zuzka1_gen, is this the same?