nvlabs/few-shot-vid2vid

Pytorch implementation for few-shot photorealistic video-to-video translation.

Few-shot vid2vid

Project | YouTube | arXiv

[Note] This repo is now deprecated. Please refer to the new Imaginaire repo: https://github.com/NVlabs/imaginaire.

Pytorch implementation for few-shot photorealistic video-to-video translation. It can be used for generating human motions from poses, synthesizing people talking from edge maps, or turning semantic label maps into photo-realistic videos. The core of video-to-video translation is image-to-image translation. Some of our work in that space can be found in pix2pixHD and SPADE.

Few-shot Video-to-Video Synthesis
Ting-Chun Wang, Ming-Yu Liu, Andrew Tao, Guilin Liu, Jan Kautz, Bryan Catanzaro
NVIDIA Corporation
In Neural Information Processing Systems (NeurIPS) 2019

License

Copyright (C) 2019 NVIDIA Corporation. All rights reserved.

This work is made available under the Nvidia Source Code License (1-Way Commercial). To view a copy of this license, visit https://nvlabs.github.io/few-shot-vid2vid/License.txt

The code is released for academic research use only. For business inquiries, please visit our website and submit the form: NVIDIA Research Licensing

Example Results

• Dance Videos

• Talking Head Videos

• Street View Videos

Prerequisites

• Linux or macOS
• Python 3
• NVIDIA GPU + CUDA cuDNN
• PyTorch 1.2

Getting Started

Installation

[Note] This repo is now deprecated. Please refer to the new Imaginaire repo: https://github.com/NVlabs/imaginaire.

• Install python libraries dominate and requests.

pip install dominate requests

• If you plan to train with face datasets, please install dlib.

pip install dlib

• If you plan to train with pose datasets, please install DensePose and/or OpenPose.
• Clone this repo:

git clone https://github.com/NVlabs/few-shot-vid2vid
cd few-shot-vid2vid

Dataset

• Pose
  • We use random dancing videos found on YouTube to crease a YouTube Dancing Dataset. We then apply DensePose / OpenPose to estimate the poses for each frame. The video URLs and code to preprocess them can be found in data/preprocess. Please note that the copyright of the dataset belongs to the original owners.
• Face
  • We use the FaceForensics dataset. We then use landmark detection to estimate the face keypoints, and interpolate them to get face edges.
• Street
  • We use a mix of sequences from different cities, which include Cityscapes official website and other cities found on YouTube.
  • We apply a pre-trained segmentation algorithm to get the corresponding semantic maps.
• Please add the obtained images to the datasets folder in the same way the example images are provided.

Training

[Note] This repo is now deprecated. Please refer to the new Imaginaire repo: https://github.com/NVlabs/imaginaire.

• First, compile a snapshot of FlowNet2 by running python scripts/download_flownet2.py.
• Please first download example datasets by running python scripts/download_datasets.py.
• The following scripts are examples of using one GPU. For multi-GPU training, simply increase the batch sizes.

Training with pose datasets

• Example DensePose and OpenPose results are included. If you plan to use your own dataset, please generate these results and put them in the same way the example dataset is provided.
• Run the example script (bash ./scripts/pose/train_g1.sh)

  python train.py --name pose --dataset_mode fewshot_pose --adaptive_spade --warp_ref --spade_combine --remove_face_labels --add_face_D --niter_single 100 --niter 200 --batchSize 2

• Please refer to More Training/Test Details for more explanations about training flags.

Training with face datasets

• Run the example script (bash ./scripts/face/train_g1.sh)

  python train.py --name face --dataset_mode fewshot_face --adaptive_spade --warp_ref --spade_combine --batchSize 8

Training with street dataset

• Run the example script (bash ./scripts/street/train_g1.sh)

python train.py --name street --dataset_mode fewshot_street --adaptive_spade --loadSize 512 --fineSize 512 --batchSize 6

Training with your own dataset

• If your input is a label map, please generate label maps which are one-channel whose pixel values correspond to the object labels (i.e. 0,1,...,N-1, where N is the number of labels). This is because we need to generate one-hot vectors from the label maps. Please use --label_nc N during both training and testing.
• If your input is not a label map, please specify --input_nc N where N is the number of input channels (The default is 3 for RGB images).
• The default setting for preprocessing is scale_width, which will scale the width of all training images to opt.loadSize while keeping the aspect ratio. If you want a different setting, please change it by using the --resize_or_crop option.

Testing

• After training, you can run inference by using the following scripts. The test results will be saved in: ./results/. Due to privacy concerns, the pretrained models are not released.

• Poses

  • To test the trained model (bash ./scripts/pose/test.sh):

    python test.py --name pose --dataset_mode fewshot_pose --adaptive_spade --warp_ref --spade_combine --remove_face_labels --finetune --seq_path [PATH_TO_SEQ] --ref_img_path [PATH_TO_REF_IMG]

• Faces

  • To test the model (bash ./scripts/face/test.sh):

    python test.py --name face --dataset_mode fewshot_face --adaptive_spade --warp_ref --spade_combine --seq_path [PATH_TO_SEQ] --ref_img_path [PATH_TO_REF_IMG]

• Street

  • To test the model (bash ./scripts/street/test.sh):

    python test.py --name street --dataset_mode fewshot_street --adaptive_spade --loadSize 512 --fineSize 512 --seq_path [PATH_TO_SEQ]--ref_img_path [PATH_TO_REF_IMG]

More Training/Test Details

• Difference of training methodology vs. vid2vid: instead of copying frames from one GPU to another, each GPU now handles separate batches. To fit into memory, the network only generates one frame at a time (n_frames_per_gpu = 1), and keep this frame fixed when generating the next frame in the sequence. We found this is usually sufficient to modify the current frame only, and is more efficient and easier to maintain.

• Training schedule: after switching to using SPADE, the network now consists of two sub-networks: one for single image generation (the SPADE generator) and the flow estimation network. By default, the training will start with training the single frame generator only (i.e. n_frames_total = 1) for niter_single epochs. After that, the network will start to train the flow network to generate videos, and temporal losses are introduced. Similar to vid2vid, we double the training sequence length for every niter_step epochs after starting training videos.

• Important flags regarding network arch:

  • adaptive_spade: adaptively generate network weights for SPADE modules.
  • no_adaptive_embed: do not dynamically generate weights for the label embedding network.
  • n_adaptive_layers: number of adaptive layers in the generator.
  • warp_ref: add an additional flow network to warp the reference image to the current frame and combine with it.
  • spade_combine: instead of linearly blending hallucinated and warped frames to generate the final frame, use warped frame as a guidance image in an additional SPADE module during the synthesis process.

• Important flags regarding training:

  • n_frames_G: the number of input frames to feed into the generator network; i.e., n_frames_G - 1 is the number of frames we look into the past.
  • n_frames_total: the total number of frames in a sequence we want to train with. We gradually increase this number during training.
  • niter_single: the number of epochs we train the single frame generator before starting training videos.
  • niter_step: for how many epochs do we double n_frames_total. The default is 10.
  • batchSize: the number of training batches. If it is not divisible by number of GPUs, the first GPU (which is usually more memory heavy) will do fewer batches.

• For other flags, please see options/train_options.py and options/base_options.py for all the training flags; see options/test_options.py and options/base_options.py for all the test flags.

• Additional flags for pose examples:

  • remove_face_labels: remove densepose results for face, so the network can get more robust during inference on different subjects.
  • basic_point_only: if specified, only use basic joint keypoints for OpenPose output, without using any hand or face keypoints.
  • add_face_D: add an additional discriminator that only works on the face region.
  • refine_face: add an additional network to refine the face region.

• Additional flags for face examples:

  • no_upper_face: by default, we add artificial edges for the upper part of face by symmetry. This flag disables it.

Citation

If you find this useful for your research, please cite the following paper.

@inproceedings{wang2019fewshotvid2vid,
   author    = {Ting-Chun Wang and Ming-Yu Liu and Andrew Tao 
                and Guilin Liu and Jan Kautz and Bryan Catanzaro},
   title     = {Few-shot Video-to-Video Synthesis},
   booktitle = {Conference on Neural Information Processing Systems (NeurIPS)},   
   year      = {2019},
}

Acknowledgments

We thank Karan Sapra for generating the segmentation maps for us.
This code borrows heavily from pix2pixHD and vid2vid.

few-shot-vid2vid's Issues

Posted by mistake

Posted by mistake

Unable to donwload the pretrained flownet2 weights and dataset from google drive

My network problem from you know why
But any other possible ways to share these files?

about pose training details

1. how many dancing videos collected for training pose
2. how many frames extracted from each training video
3. is it necessary that each video extract same number of frames

Thanks~

Rough estimation of training time required before training

Hello, in your paper you mentioned to have used a NVIDIA DGX-1 machine with 8 32GB V100 GPUs and 15,000 clips for training the pose model, however, could you please be more specific about the amount of pictures, resolution and time it took to train your model? If I have something I can compare to, I can at least roughly estimate if it's a suicide mission to train something comparable/similar on a way less powerful machine, I have a Titan RTX with 24GB Vram and just to prepare my data (643.630 FullHD pictures) with OpenPose w/ maximum precision configuration and DensePose is likely going to take 5/7 days, just for that. Also, did your data require any cleaning/post-processing as occasional errors will always happen with data collected in the wild (such as incomplete hands/face, wrong bone positioning, glitches, etcetera)? Or as long as the data is reasonably good (not outright perfect) the error margin is big enough that it can be left ignored? Thanks for your amazing work!

Wait for your code

When will the code be released?

Great work!!!!
Any timeline to release the code?

TypeError: vis_print() takes 2 positional arguments but 3 were given

I tried running pose example provided and when trying to predict getting the following error

CustomDatasetDataLoader
dataset [PoseDataset] was created
---------- Networks initialized -------------
---------- Optimizers initialized -------------
Pretrained network G has fewer layers; The following are not initialized:
['flow_network_temp.conv_flow', 'flow_network_temp.conv_w', 'flow_network_temp.down_flow', 'flow_network_temp.res_flow', 'flow_network_temp.up_flow', 'img_prev_embedding.conv_first', 'img_prev_embedding.down_0', 'img_prev_embedding.down_1', 'img_prev_embedding.down_2', 'img_prev_embedding.down_3', 'img_prev_embedding.down_4']
model [Vid2VidModel] was created
first image
Traceback (most recent call last):
File "test.py", line 41, in
synthesized_image, _, _, _, _, _ = model(data_list)
File "/home/Vikas/.conda/envs/vid2vid/lib/python3.7/site-packages/torch/nn/modules/module.py", line 547, in call
result = self.forward(*input, **kwargs)
File "/home/Vikas/few-shot-vid2vid/models/vid2vid_model.py", line 55, in forward
return self.inference(tgt_label, ref_labels, ref_images)
File "/home/Vikas/few-shot-vid2vid/models/vid2vid_model.py", line 188, in inference
self.finetune(ref_labels, ref_images)
File "/home/Vikas/few-shot-vid2vid/models/vid2vid_model.py", line 205, in finetune
params, _ = self.get_train_params(self.netG, train_names)
File "/home/Vikas/few-shot-vid2vid/models/base_model.py", line 163, in get_train_params
Visualizer.vis_print(self.opt, 'training layers: ', train_list)
TypeError: vis_print() takes 2 positional arguments but 3 were given

Training a model with my own dataset

After successfully using the provided images for training, I’d like to create my own larger dataset from the FaceForensics footage. So I generated image sequences with ffmpeg and keypoints with dlib. When I try to start the training script, I get the following error. What exactly is the problem?

CustomDatasetDataLoader
485 sequences
dataset [FaceDataset] was created
Resuming from epoch 1 at iteration 0
create web directory ./checkpoints/face/web...
---------- Networks initialized -------------
---------- Optimizers initialized -------------
./checkpoints/face/latest_net_G.pth not exists yet!
./checkpoints/face/latest_net_D.pth not exists yet!
model [Vid2VidModel] was created
Traceback (most recent call last):
  File "train.py", line 73, in <module>
    train()
  File "train.py", line 40, in train
    for idx, data in enumerate(dataset, start=trainer.epoch_iter):
  File "/home/keno/.local/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 819, in __next__
    return self._process_data(data)
  File "/home/keno/.local/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 846, in _process_data
    data.reraise()
  File "/home/keno/.local/lib/python3.7/site-packages/torch/_utils.py", line 369, in reraise
    raise self.exc_type(msg)
IndexError: Caught IndexError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/home/keno/.local/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 178, in _worker_loop
    data = fetcher.fetch(index)
  File "/home/keno/.local/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/keno/.local/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/keno/repos/few-shot-vid2vid/data/fewshot_face_dataset.py", line 103, in __getitem__
    Li = self.get_face_image(keypoints, transform_L, ref_img.size)
  File "/home/keno/repos/few-shot-vid2vid/data/fewshot_face_dataset.py", line 168, in get_face_image
    x = keypoints[sub_edge, 0]
IndexError: index 82 is out of bounds for axis 0 with size 82

How can I resume training on colab from a particular epoch?

For effective results, 3k-5k frames are required. My colab training session gets disconnected due to high usage. How do I resume training. I am already saving the checkpoints to my drive.

[Pose] Unsatisfactory training results

Hi, thanks a lot for your awesome work.

I try to reproduce the results of pose in the paper. So I

• download 358 solo dance videos with simple backgrounds
• extract a slice of 30 seconds from each video
• obtain the poses with OpenPose and Densepose for all frames
• run train_g8.sh with 8 Telsa P40

However, after training for 108 epochs, I still cannot reproduce the results and the face regions are extremely terrible.

Cound you please give me some advice?

Another question. The option n_shot is set to 1 in base_option.py. Should I increase it so that the attention network can be trained during training?

Looking forward to your reply. Thanks again.

The approximate length of each video for training？10 seconds? 1 minutes? or longer?

Great work!
Can I ask What the approximate length of each video for training is? 10 seconds? 1 minutes? or longer?
Thanks a lot!

Pretrained network G has fewer layers

I was running the model using a sample image and sequence after training it.
I get the following error. Help!

python test.py --name face --dataset_mode fewshot_face --adaptive_spade --warp_ref --spade_combine --seq_path input_data/sequence.mp4 --ref_img_path example.png

CustomDatasetDataLoader
dataset [FaceDataset] was created
---------- Networks initialized -------------
---------- Optimizers initialized -------------
Pretrained network G has fewer layers; The following are not initialized:
['flow_network_temp.conv_flow', 'flow_network_temp.conv_w', 'flow_network_temp.down_flow', 'flow_network_temp.res_flow', 'flow_network_temp.up_flow']
model [Vid2VidModel] was created

Again the same: Pretrained network G has fewer layers

I was running the model using a sample image and sequence after training it.
I get the following error.

I can't fix it. Also I can't understand the reason. Sorry for my question, but I can't understand the answer in previous issue.

GPU Driver specification

I am using Google Colab (GPU runtime) to replicate the results of the paper. Pytorch version is 1.14, CUDA v10, and Nvidia-smi driver 418.67, Numpy version 1.17. Torchvision 0.5

On executing train.py I am getting the following error :
"CUDA driver version is insufficient for CUDA runtime version"

In google colab: usr/local/cuda is used, which is v10. So, what does CUDA runtime version imply?

Can share the whole youtube dance dataset‘s URLs?

I scan the downloaded example dataset, in dance dataset, it just has 2 clips(which contains about 10 images totally), so the how can i get urls of whole youtube dance dataset to reproduce performance in paper? dear author, could you share it ? I'll appreciate it very much

Training gets stuck after epoch 50 is complete

I've cloned the latest repo and been training it for the talking heads face model, the outputs and tests are producing really great results throughout, however after epoch 50 it gets stuck training. I've tried using bacthsize 1 and I still get the same problem. Tried rolling back the iter.txt file to try and run from epoch 49, it still gets stuck after saving the latest model at epoch 50. Below is the command I'm using to start training it and the output its stuck on.

python train.py --name face --dataset_mode fewshot_face --adaptive_spade --warp_ref --spade_combine --loadSize 512 --batchSize 2 --continue_train --gpu_id 1,0
CustomDatasetDataLoader
3 sequences
dataset [FaceDataset] was created
Resuming from epoch 51 at iteration 0
create web directory ./checkpoints/face/web...
---------- Networks initialized -------------
---------- Optimizers initialized -------------
update learning rate: 0.000400 -> 0.000040
---------- Now start training multiple frames -------------
Pretrained network G has fewer layers; The following are not initialized:
['flow_network_temp.conv_flow', 'flow_network_temp.conv_w', 'flow_network_temp.down_flow', 'flow_network_temp.res_flow', 'flow_network_temp.up_flow', 'img_prev_embedding.conv_first', 'img_prev_embedding.down_0', 'img_prev_embedding.down_1', 'img_prev_embedding.down_2', 'img_prev_embedding.down_3', 'img_prev_embedding.down_4', 'img_prev_embedding.up_0', 'img_prev_embedding.up_1', 'img_prev_embedding.up_2', 'img_prev_embedding.up_3', 'img_prev_embedding.up_4']
network loaded from ./checkpoints/face/latest_net_D.pth
./checkpoints/face/latest_net_DT.pth not exists yet!
model [Vid2VidModel] was created
update learning rate: 0.000040 -> 0.000039
--- Updating training sequence length to 4 ---
/home/vision/venv2/lib/python3.6/site-packages/torch/nn/functional.py:2479: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
"See the documentation of nn.Upsample for details.".format(mode))

I'm running on 2 RTX 2080 Ti GPUs with 11gb VRAM each...

Add parameter '--tf_log' error

Traceback (most recent call last):
File "train.py", line 69, in
train()
File "train.py", line 58, in train
if trainer.end_of_iter(loss_dict, generated + data_list + data_ref_list, model):
File "/home/ww/myGit/few-shot-vid2vid/models/trainer.py", line 58, in end_of_iter
self.visualizer.display_current_results(visuals, epoch, total_steps)
File "/home/ww/myGit/few-shot-vid2vid/util/visualizer.py", line 98, in display_current_results
if len(image_numpy.shape) >= 4:
AttributeError: 'NoneType' object has no attribute 'shape'

Problems with optical flow

I checked the optical flow between two neighboring frames in a video, and the result is:

[tensor([[[[[ 10.2950,   0.7181,  -4.7409,  ...,   3.1726,   6.1062,
             -8.1125],
           [ -3.6843,  -6.1567,  -7.4636,  ...,   3.6077,   2.9878,
              5.3008],
           [ -7.2534,  -6.3700,  -5.7260,  ...,   6.1592,   4.6107,
              6.7087],
           ...,
           [-13.3908, -15.8770, -17.8833,  ...,   9.0593,   9.0684,
              7.5918],
           [-14.2425, -15.4943, -16.1309,  ...,   8.8715,   7.9123,
              5.1580],
           [-15.3805, -18.3000, -18.8595,  ...,   9.3637,   3.7064,
              7.0863]],

          [[-31.9163, -33.5100, -46.0788,  ..., -57.1201, -48.2610,
            -35.4169],
           [-31.5390, -48.7966, -54.5237,  ..., -60.5762, -55.1705,
            -48.9994],
           [-46.4077, -52.8530, -54.1906,  ..., -56.5632, -57.1629,
            -55.7577],
           ...,
           [-29.6735, -33.5224, -34.8830,  ..., -34.7638, -35.6107,
            -30.5754],
           [-30.2230, -33.5731, -35.8306,  ..., -34.3179, -34.2813,
            -22.6298],
           [-15.9992, -27.0066, -34.1596,  ..., -34.1373, -32.0856,
            -19.3197]]]]], device='cuda:0'), None]

Is there any problem with this result?

How to run inference on the face dataset?

I trained the network with the example script and ran the test successfully.

Now I want to use my own video to run. So I split the video into frames and gave the path in seq_path argument, but it seems to not be enough, as no new frames are added to the result path. The output is only:

dataset [FaceDataset] was created
---------- Networks initialized -------------
---------- Optimizers initialized -------------
Pretrained network G has fewer layers; The following are not initialized:
['flow_network_temp.conv_flow', 'flow_network_temp.conv_w', 'flow_network_temp.down_flow', 'flow_network_temp.res_flow', 'flow_network_temp.up_flow']
model [Vid2VidModel] was created

Do I need to create the checkpoints myself ?
How do I go about inputting a sequence and an image, to output a new video sequence ?

F_Flow value and W value become nan during/after epoch 4

I am training with the pose example dataset. After epoch 4. F_Flow and W have became nan.

Can you explain this problem?

log:

Spec:

Thanks.

Error while running downlaod_flownet2.py

The only change was changing the bash file to use python3 instead of python. Here's the error.It is only some of it.
`running install
running bdist_egg
running egg_info
writing correlation_cuda.egg-info/PKG-INFO
writing dependency_links to correlation_cuda.egg-info/dependency_links.txt
writing top-level names to correlation_cuda.egg-info/top_level.txt
reading manifest file 'correlation_cuda.egg-info/SOURCES.txt'
writing manifest file 'correlation_cuda.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_ext
building 'correlation_cuda' extension
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/home/k4rth33k/.local/lib/python3.6/site-packages/torch/include -I/home/k4rth33k/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -I/home/k4rth33k/.local/lib/python3.6/site-packages/torch/include/TH -I/home/k4rth33k/.local/lib/python3.6/site-packages/torch/include/THC -I/usr/include/python3.6m -c correlation_cuda.cc -o build/temp.linux-x86_64-3.6/correlation_cuda.o -std=c++11 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=correlation_cuda -D_GLIBCXX_USE_CXX11_ABI=0
/usr/bin/nvcc -I/home/k4rth33k/.local/lib/python3.6/site-packages/torch/include -I/home/k4rth33k/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -I/home/k4rth33k/.local/lib/python3.6/site-packages/torch/include/TH -I/home/k4rth33k/.local/lib/python3.6/site-packages/torch/include/THC -I/usr/include/python3.6m -c correlation_cuda_kernel.cu -o build/temp.linux-x86_64-3.6/correlation_cuda_kernel.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options '-fPIC' -gencode arch=compute_50,code=sm_50 -gencode arch=compute_52,code=sm_52 -gencode arch=compute_60,code=sm_60 -gencode arch=compute_61,code=sm_61 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_70,code=compute_70 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=correlation_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11
/home/k4rth33k/.local/lib/python3.6/site-packages/torch/include/c10/core/TensorTypeSet.h(44): warning: integer conversion resulted in a change of sign

/home/k4rth33k/.local/lib/python3.6/site-packages/torch/include/c10/core/TensorTypeSet.h(44): warning: integer conversion resulted in a change of sign

/home/k4rth33k/.local/lib/python3.6/site-packages/torch/include/c10/core/TensorTypeSet.h(44): warning: integer conversion resulted in a change of sign

/home/k4rth33k/.local/lib/python3.6/site-packages/torch/include/c10/core/TensorTypeSet.h(44): warning: integer conversion resulted in a change of sign

/home/k4rth33k/.local/lib/python3.6/site-packages/torch/include/c10/core/TensorTypeSet.h(44): warning: integer conversion resulted in a change of sign

/home/k4rth33k/.local/lib/python3.6/site-packages/torch/include/c10/core/TensorTypeSet.h(44): warning: integer conversion resulted in a change of sign

/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<, _Elements>::_MoveConstructibleTuple() [with _UElements = {std::tuple<at::Tensor, at::Tensor, at::Tensor>}; bool = true; _Elements = {at::Tensor, at::Tensor, at::Tensor}]’:
/usr/include/c++/6/tuple:626:248: required by substitution of ‘template<class ... _UElements, typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor, at::Tensor, at::Tensor>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (3ul >= 1)), bool>::type > constexpr std::tuple< >::tuple(_UElements&& ...) [with _UElements = {std::tuple<at::Tensor, at::Tensor, at::Tensor>}; typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor, at::Tensor, at::Tensor>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (3ul >= 1)), bool>::type = ]’
/home/k4rth33k/.local/lib/python3.6/site-packages/torch/include/ATen/core/TensorMethods.h:5084:246: required from here
/usr/include/c++/6/tuple:483:67: error: mismatched argument pack lengths while expanding ‘std::is_constructible<_Elements, _UElements&&>’
return _and<is_constructible<_Elements, _UElements&&>...>::value;
^~~~~
/usr/include/c++/6/tuple:484:1: error: body of constexpr function ‘static constexpr bool std::_TC<, _Elements>::_MoveConstructibleTuple() [with _UElements = {std::tuple<at::Tensor, at::Tensor, at::Tensor>}; bool = true; _Elements = {at::Tensor, at::Tensor, at::Tensor}]’ not a return-statement
}
^
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<, _Elements>::_ImplicitlyMoveConvertibleTuple() [with _UElements = {std::tuple<at::Tensor, at::Tensor, at::Tensor>}; bool = true; _Elements = {at::Tensor, at::Tensor, at::Tensor}]’:
/usr/include/c++/6/tuple:626:362: required by substitution of ‘template<class ... _UElements, typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor, at::Tensor, at::Tensor>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (3ul >= 1)), bool>::type > constexpr std::tuple< >::tuple(_UElements&& ...) [with _UElements = {std::tuple<at::Tensor, at::Tensor, at::Tensor>}; typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor, at::Tensor, at::Tensor>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (3ul >= 1)), bool>::type = ]’
/home/k4rth33k/.local/lib/python3.6/site-packages/torch/include/ATen/core/TensorMethods.h:5084:246: required from here
/usr/include/c++/6/tuple:489:65: error: mismatched argument pack lengths while expanding ‘std::is_convertible<_UElements&&, _Elements>’
return _and<is_convertible<_UElements&&, _Elements>...>::value;
^~~~~
/usr/include/c++/6/tuple:490:1: error: body of constexpr function ‘static constexpr bool std::_TC<, _Elements>::_ImplicitlyMoveConvertibleTuple() [with _UElements = {std::tuple<at::Tensor, at::Tensor, at::Tensor>}; bool = true; _Elements = {at::Tensor, at::Tensor, at::Tensor}]’ not a return-statement
}
^
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<, _Elements>::_NonNestedTuple() [with _SrcTuple = const std::tuple<at::Tensor, at::Tensor, at::Tensor>&; bool = true; _Elements = {at::Tensor, at::Tensor, at::Tensor}]’:
/usr/include/c++/6/tuple:662:419: required by substitution of ‘template<class ... _UElements, class _Dummy, typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor>::_ConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor>::_ImplicitlyConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor, at::Tensor, at::Tensor>::_NonNestedTuple<const tuple<_Elements ...>&>()), bool>::type > constexpr std::tuple< >::tuple(const std::tuple<_Args1 ...>&) [with _UElements = {at::Tensor, at::Tensor, at::Tensor}; _Dummy = void; typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor>::_ConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor>::_ImplicitlyConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor, at::Tensor, at::Tensor>::_NonNestedTuple<const tuple<_Elements ...>&>()), bool>::type = ]’
/home/k4rth33k/.local/lib/python3.6/site-packages/torch/include/ATen/core/TensorMethods.h:5084:246: required from here
/usr/include/c++/6/tuple:495:244: error: wrong number of template arguments (4, should be 2)
return _and<_not<is_same<tuple<_Elements...>,
^
/usr/include/c++/6/type_traits:1558:8: note: provided for ‘template<class _From, class _To> struct std::is_convertible’
struct is_convertible
^~~~~~~~~~~~~~
/usr/include/c++/6/tuple:502:1: error: body of constexpr function ‘static constexpr bool std::_TC<, _Elements>::_NonNestedTuple() [with _SrcTuple = const std::tuple<at::Tensor, at::Tensor, at::Tensor>&; bool = true; _Elements = {at::Tensor, at::Tensor, at::Tensor}]’ not a return-statement
}
^
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<, _Elements>::_NonNestedTuple() [with _SrcTuple = std::tuple<at::Tensor, at::Tensor, at::Tensor>&&; bool = true; _Elements = {at::Tensor, at::Tensor, at::Tensor}]’:
/usr/include/c++/6/tuple:686:422: required by substitution of ‘template<class ... _UElements, class _Dummy, typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor>::_MoveConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor, at::Tensor, at::Tensor>::_NonNestedTuple<tuple<_Elements ...>&&>()), bool>::type > constexpr std::tuple< >::tuple(std::tuple<_Args1 ...>&&) [with _UElements = {at::Tensor, at::Tensor, at::Tensor}; _Dummy = void; typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor>::_MoveConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor, at::Tensor, at::Tensor>::_NonNestedTuple<tuple<_Elements ...>&&>()), bool>::type = ]’
/home/k4rth33k/.local/lib/python3.6/site-packages/torch/include/ATen/core/TensorMethods.h:5084:246: required from here
/usr/include/c++/6/tuple:495:244: error: wrong number of template arguments (4, should be 2)
return _and<_not<is_same<tuple<_Elements...>,
^
/usr/include/c++/6/type_traits:1558:8: note: provided for ‘template<class _From, class _To> struct std::is_convertible’
struct is_convertible
^~~~~~~~~~~~~~
/usr/include/c++/6/tuple:502:1: error: body of constexpr function ‘static constexpr bool std::_TC<, _Elements>::_NonNestedTuple() [with _SrcTuple = std::tuple<at::Tensor, at::Tensor, at::Tensor>&&; bool = true; _Elements = {at::Tensor, at::Tensor, at::Tensor}]’ not a return-statement
}
^
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<, _Elements>::_MoveConstructibleTuple() [with _UElements = {std::tuple<at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>}; bool = true; _Elements = {at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor}]’:
/usr/include/c++/6/tuple:626:248: required by substitution of ‘template<class ... _UElements, typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (5ul >= 1)), bool>::type > constexpr std::tuple< >::tuple(_UElements&& ...) [with _UElements = {std::tuple<at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>}; typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (5ul >= 1)), bool>::type = ]’
/home/k4rth33k/.local/lib/python3.6/site-packages/torch/include/ATen/Functions.h:1476:450: required from here
/usr/include/c++/6/tuple:483:67: error: mismatched argument pack lengths while expanding ‘std::is_constructible<_Elements, _UElements&&>’
return _and<is_constructible<_Elements, _UElements&&>...>::value;
^~~~~
/usr/include/c++/6/tuple:484:1: error: body of constexpr function ‘static constexpr bool std::_TC<, _Elements>::_MoveConstructibleTuple() [with _UElements = {std::tuple<at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>}; bool = true; _Elements = {at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor}]’ not a return-statement
}
^
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<, _Elements>::_ImplicitlyMoveConvertibleTuple() [with _UElements = {std::tuple<at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>}; bool = true; _Elements = {at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor}]’:
/usr/include/c++/6/tuple:626:362: required by substitution of ‘template<class ... _UElements, typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (5ul >= 1)), bool>::type > constexpr std::tuple< >::tuple(_UElements&& ...) [with _UElements = {std::tuple<at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>}; typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (5ul >= 1)), bool>::type = ]’
/home/k4rth33k/.local/lib/python3.6/site-packages/torch/include/ATen/Functions.h:1476:450: required from here
/usr/include/c++/6/tuple:489:65: error: mismatched argument pack lengths while expanding ‘std::is_convertible<_UElements&&, _Elements>’
return _and<is_convertible<_UElements&&, _Elements>...>::value;
^~~~~
/usr/include/c++/6/tuple:490:1: error: body of constexpr function ‘static constexpr bool std::_TC<, _Elements>::_ImplicitlyMoveConvertibleTuple() [with _UElements = {std::tuple<at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>}; bool = true; _Elements = {at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor}]’ not a return-statement
}
^
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<, _Elements>::_NonNestedTuple() [with _SrcTuple = const std::tuple<at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>&; bool = true; _Elements = {at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor}]’:
/usr/include/c++/6/tuple:662:419: required by substitution of ‘template<class ... _UElements, class _Dummy, typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_ConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_ImplicitlyConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_NonNestedTuple<const tuple<_Elements ...>&>()), bool>::type > constexpr std::tuple< >::tuple(const std::tuple<_Args1 ...>&) [with _UElements = {at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor}; _Dummy = void; typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_ConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_ImplicitlyConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_NonNestedTuple<const tuple<_Elements ...>&>()), bool>::type = ]’
/home/k4rth33k/.local/lib/python3.6/site-packages/torch/include/ATen/Functions.h:1476:450: required from here
/usr/include/c++/6/tuple:495:244: error: wrong number of template arguments (6, should be 2)
return _and<_not<is_same<tuple<_Elements...>,
^
/usr/include/c++/6/type_traits:1558:8: note: provided for ‘template<class _From, class _To> struct std::is_convertible’
struct is_convertible
^~~~~~~~~~~~~~
/usr/include/c++/6/tuple:502:1: error: body of constexpr function ‘static constexpr bool std::_TC<, _Elements>::_NonNestedTuple() [with _SrcTuple = const std::tuple<at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>&; bool = true; _Elements = {at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor}]’ not a return-statement
}
^
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<, _Elements>::_NonNestedTuple() [with _SrcTuple = std::tuple<at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>&&; bool = true; _Elements = {at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor}]’:
/usr/include/c++/6/tuple:686:422: required by substitution of ‘template<class ... _UElements, class _Dummy, typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_MoveConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_NonNestedTuple<tuple<_Elements ...>&&>()), bool>::type > constexpr std::tuple< >::tuple(std::tuple<_Args1 ...>&&) [with _UElements = {at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor}; _Dummy = void; typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_MoveConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>::_NonNestedTuple<tuple<_Elements ...>&&>()), bool>::type = ]’
/home/k4rth33k/.local/lib/python3.6/site-packages/torch/include/ATen/Functions.h:1476:450: required from here
/usr/include/c++/6/tuple:495:244: error: wrong number of template arguments (6, should be 2)
return _and<_not<is_same<tuple<_Elements...>,
^
/usr/include/c++/6/type_traits:1558:8: note: provided for ‘template<class _From, class _To> struct std::is_convertible’
struct is_convertible
^~~~~~~~~~~~~~
/usr/include/c++/6/tuple:502:1: error: body of constexpr function ‘static constexpr bool std::_TC<, _Elements>::_NonNestedTuple() [with _SrcTuple = std::tuple<at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor>&&; bool = true; _Elements = {at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor}]’ not a return-statement
}
^
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<, _Elements>::_MoveConstructibleTuple() [with _UElements = {std::tuple<at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor > >}; bool = true; _Elements = {at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor >}]’:
/usr/include/c++/6/tuple:626:248: required by substitution of ‘template<class ... _UElements, typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor > >::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor > >::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor > >::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (4ul >= 1)), bool>::type > constexpr std::tuple< >::tuple(_UElements&& ...) [with _UElements = {std::tuple<at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor > >}; typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor > >::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor > >::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor > >::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (4ul >= 1)), bool>::type = ]’
/home/k4rth33k/.local/lib/python3.6/site-packages/torch/include/ATen/Functions.h:1492:115: required from here
/usr/include/c++/6/tuple:483:67: error: mismatched argument pack lengths while expanding ‘std::is_constructible<_Elements, _UElements&&>’
return _and<is_constructible<_Elements, _UElements&&>...>::value;
^~~~~
/usr/include/c++/6/tuple:484:1: error: body of constexpr function ‘static constexpr bool std::_TC<, _Elements>::_MoveConstructibleTuple() [with _UElements = {std::tuple<at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor > >}; bool = true; _Elements = {at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor >}]’ not a return-statement
}
^
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<, _Elements>::_ImplicitlyMoveConvertibleTuple() [with _UElements = {std::tuple<at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor > >}; bool = true; _Elements = {at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor >}]’:
/usr/include/c++/6/tuple:626:362: required by substitution of ‘template<class ... _UElements, typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor > >::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor > >::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor > >::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (4ul >= 1)), bool>::type > constexpr std::tuple< >::tuple(_UElements&& ...) [with _UElements = {std::tuple<at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor > >}; typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor > >::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor > >::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor > >::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (4ul >= 1)), bool>::type = ]’
/home/k4rth33k/.local/lib/python3.6/site-packages/torch/include/ATen/Functions.h:1492:115: required from here
/usr/include/c++/6/tuple:489:65: error: mismatched argument pack lengths while expanding ‘std::is_convertible<_UElements&&, _Elements>’
return _and<is_convertible<_UElements&&, _Elements>...>::value;
^~~~~
/usr/include/c++/6/tuple:490:1: error: body of constexpr function ‘static constexpr bool std::_TC<, _Elements>::_ImplicitlyMoveConvertibleTuple() [with _UElements = {std::tuple<at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor > >}; bool = true; _Elements = {at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor >}]’ not a return-statement
}
^
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<, _Elements>::_NonNestedTuple() [with _SrcTuple = const std::tuple<at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor > >&; bool = true; _Elements = {at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor >}]’:
/usr/include/c++/6/tuple:662:419: required by substitution of ‘template<class ... _UElements, class _Dummy, typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor > >::_ConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor > >::_ImplicitlyConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor > >::_NonNestedTuple<const tuple<_Elements ...>&>()), bool>::type > constexpr std::tuple< >::tuple(const std::tuple<_Args1 ...>&) [with _UElements = {at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor >}; _Dummy = void; typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor > >::_ConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor > >::_ImplicitlyConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor > >::_NonNestedTuple<const tuple<_Elements ...>&>()), bool>::type = ]’
/home/k4rth33k/.local/lib/python3.6/site-packages/torch/include/ATen/Functions.h:1492:115: required from here
/usr/include/c++/6/tuple:495:244: error: wrong number of template arguments (5, should be 2)
return _and<_not<is_same<tuple<_Elements...>,
^
/usr/include/c++/6/type_traits:1558:8: note: provided for ‘template<class _From, class _To> struct std::is_convertible’
struct is_convertible
^~~~~~~~~~~~~~
/usr/include/c++/6/tuple:502:1: error: body of constexpr function ‘static constexpr bool std::_TC<, _Elements>::_NonNestedTuple() [with _SrcTuple = const std::tuple<at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor > >&; bool = true; _Elements = {at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor >}]’ not a return-statement
}
^
/usr/include/c++/6/tuple: In instantiation of ‘static constexpr bool std::_TC<, _Elements>::_NonNestedTuple() [with _SrcTuple = std::tuple<at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor > >&&; bool = true; _Elements = {at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor >}]’:
/usr/include/c++/6/tuple:686:422: required by substitution of ‘template<class ... _UElements, class _Dummy, typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor > >::_MoveConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor > >::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor > >::_NonNestedTuple<tuple<_Elements ...>&&>()), bool>::type > constexpr std::tuple< >::tuple(std::tuple<_Args1 ...>&&) [with _UElements = {at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor >}; _Dummy = void; typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor > >::_MoveConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor > >::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), at::Tensor, at::Tensor, at::Tensor, std::vector<at::Tensor, std::allocatorat::Tensor > >::_NonNestedTuple<tuple<_Elements ...>&&>()), bool>::type = ]’
`

The CUDA version

the CUDA version of my pc is 9.0

When i compile the snapshot of FlowNet2 by running python scripts/download_flownet2.py, the error is "correlation_cuda.cc:4:35: fatal error: ATen/cuda/CUDAContext.h: No such file or directory"

I think the reason is that the CUDA version does not match.

Could you please tell the CUDA version?

potential problem in code

I think t_step = np.random.randint(max_t_step) + 1 in ./data/base_dataset.py would throw an error when cur_seq_len equals to 1.

Maybe an if-clause like if max_t_step==0: t_step=1 should be added to avoid crash while training?

flownet2_pytorch resample2d_cuda error

Has anyone encountered this error?
ImportError: /root/miniconda/envs/py36/lib/python3.6/site-packages/resample2d_cuda-0.0.0-py3.6-linux-x86_64.egg/resample2d_cuda.cpython-36m-x86_64-linux-gnu.so: undefined symbol: _ZN2at19NonVariableTypeMode10is_enabledEv

my environment:
Docker:nvidia/cuda:9.2-cudnn7-devel-ubuntu18.04-py3.6.10
torch:1.2.0

有关colab训练模型的坑及解决方案（Pits and solutions about training on colab）

1，cuda出错，no kernel image和no memory：
colab主要有四种显卡：k80、p4、T4、p100。
其中p100和T4，在安装运行flownet2_pytorh时（每次都需要装），不会有问题，直接按照官方read me运行，或者进入few-shot-vid2vid/models/networks/flownet2_pytorch/后，!bash install.sh。
而k80和p4则会报错，cuda kernel：no kernel image……（一个千年未解决的cuda老问题）
最佳解决方案为：
直接重置所有代码执行程序，换主机，换到T4或p100为止，最省力，最高效，呵呵。
其次的方案为修改你的
/content/drive/My Drive/few-shot-vid2vid/models/networks/flownet2_pytorch/networks/channelnorm_package/setup.py；
/content/drive/My Drive/few-shot-vid2vid/models/networks/flownet2_pytorch/networks/correlation_package/setup.py；
/content/drive/My Drive/few-shot-vid2vid/models/networks/flownet2_pytorch/networks/resample2d_package/setup.py；
在三个文件中添加上适配的环境变量：
nvcc_args = [
'-gencode', 'arch=compute_30,code=sm_30',
'-gencode', 'arch=compute_35,code=sm_35',
'-gencode', 'arch=compute_37,code=sm_37',
'-gencode', 'arch=compute_50,code=sm_50',
'-gencode', 'arch=compute_52,code=sm_52',
'-gencode', 'arch=compute_60,code=sm_60',
'-gencode', 'arch=compute_61,code=sm_61',
'-gencode', 'arch=compute_70,code=sm_70',
'-gencode', 'arch=compute_70,code=compute_70'
]

k80请强行指定pytorch==0.41。

2，继续训练时web预览图片读取错误：input/output：epoch……
这个错误，本地不会发生。只有colab与谷歌云盘。
原因：谷歌云盘的文件夹内文件过多，colab无法读入（也是老问题）。
解决方案：
进入/content/drive/My Drive/few-shot-vid2vid/checkpoints/face/web/
删掉整个images文件夹中，再生成一个空的images就可以了。
或者在训练时加上‘--no_html‘参数（我未测试，因为我需要预览）

注意：继续训练，iters不一定为整数，不影响结果。

3，50个epoch后，seq length to XX，out of memory内存溢出。
解决方案：只有换P100，T4不行。
70个epoch后，seq length to 16。
还没想出方案，诶，flownet2升级验证太大了。

注意：第3条的问题，已经由程序主-@tcwang0509 升级修正了，现在跑起来很流畅。就是70个epoch后，colab的主机速度比较慢，这是没办法的，免费啊！——2020.2.13

English（machine translation，forgive me）：
1, cuda error, no kernel image and no memory: colab mainly has four kinds of graphics cards: k80, p4, p100,T4.
Among them, p100 and T4, when installing and running flownet2_pytorh (need to be installed each time), there will be no problem, run directly according to the official read me, or enter fee-shot-vid2vid / models / networks / flownet2_pytorch /, and bash install.sh.
But k80 and p4 will report an error, cuda kernel: no kernel image ... (an unsolved old problem of cuda for a thousand years)
The best solution is: directly reset all code execution programs, change the host, change to p100 or T4, the most labor-saving , The most efficient, huh, huh.
The second solution is to modify your
/ content / drive / My Drive / few-shot-vid2vid / models / networks / flownet2_pytorch / networks / channelnorm_package / setup.py;
/ content / drive / My Drive / few-shot-vid2vid / models /networks/flownet2_pytorch/networks/correlation_package/setup.py;
/ content / drive / My Drive / few-shot-vid2vid / models / networks / flownet2_pytorch / networks / resample2d_package / setup.py;
add adaptations to the three files Environment variables:
nvcc_args = ['-gencode', 'arch = compute_30, code = sm_30', '-gencode', 'arch = compute_35, code = sm_35', '-gencode', 'arch = compute_37, code = sm_37 ',' -gencode ',' arch = compute_50, code = sm_50 ',' -gencode ',' arch = compute_52, code = sm_52 ',' -gencode ',' arch = compute_60, code = sm_60 ',' -gencode ',' arch = compute_61, code = sm_61 ',' -gencode ',' arch = compute_70, code = sm_70 ',' -gencode ',' arch = compute_70, code = compute_70 '] k80 Please specify pytorch == 0.41 .

2，Web preview image reading error when continuing training: input / output: epoch ...
This error does not occur locally.
Only Colab and Google Cloud Disk.
Cause: There are too many files in the Google Cloud Disk folder, and Colab cannot read them (also an old problem).
Solution: Go to / content / drive / My Drive / few-shot-vid2vid / checkpoints / face / web/
Delete the entire images folder and generate an empty images.
Or add ‘--no_html’ parameter during training (I have n’t tested it because I need to preview it)

Note: If you continue to train, iters may not be an integer and will not affect the result.

3，After 50 epochs, seq length to XX, out of memory.
Solution: Change P100 instead of T4.

After 70 epochs, seq length to 16.
Haven't figured out a solution yet.

Note: The problem of Article 3 has been upgraded and corrected by the program owner-@ tcwang0509, and now it runs smoothly. After 70 epochs, the host of Colab is relatively slow. There is no way to do it, it's free!——2020.2.13

程序很棒，感谢开发者，感谢NVlabs（我刚发了疯做空了NV的股票，以为few-v2v和stylegan2赶不上年底的………………诶，天啊！）
祝大家顺利，愉快！
The program is great, thanks to the developers, thanks to NVlabs (I just went crazy and shorted the stock of NV, thinking that few-v2v and stylegan2 can't keep up with the end of the year ... oh, my God!)
I wish you all a smooth and happy!

Flows become strange when set n_shot >= 1

I train the few-vid2vid with FaceForensics dataset, and set n_shot to 2 or 4 to try to use multiple reference image during training, but the flows become dissimilar to the flow result when n_shot is default (= 1). Can someone explain if this is a bug? Thanks in advance for your answers!

Flows of n_shot=1:

Warping result:

Flows of n_shot=2:

Warping result:

channelnorm_cuda Module import error

I am implementing few shot vid2vid in google colab. And I am getting this error on executing train.py

File "/content/few-shot-vid2vid/models/networks/flownet2_pytorch/networks/channelnorm_package/channelnorm.py", line 3, in
import channelnorm_cuda
ModuleNotFoundError: No module named 'channelnorm_cuda'

Is there possible to run this code using pytorch 1.1.0?

My current working environment can only use Pytorch 1.1.0. How can I modify the code to adapt to Pytorch 1.1.0.?

Iterations for benchmark?

I've reviewed the paper and readme but can't find any reference to how many iterations were used to achieve the results as displayed on the gifs of this repo.
I can see in the paper that training was conducted using an NVIDIA DGX-1 machine with 8 32GB V100 GPUs, but how many iterations were done and how long did it took to complete the training?

[Face] Results getting worse after 50 epochs, model starts diverging

Logs:
loss_log.txt

opt.txt
opt.txt

Upto 50 epochs results were satisfactory but then the results started to get worse.

After 50 Epochs:

After 55 Epochs:

After 60 Epochs:

After 65 Epochs:

After 75 Epoch:

Need dance data

Thank you for your work! I can't access YouTube because of the Internet,Who can provide relevant dance data sets, My mailbox: [email protected]

Identical pictures after inference.

I have trained your model for 5 epoches on pose dataset with script ./scripts/pose/train_g1.sh. After this I ran ./scripts/pose/test.sh and got 30 identical pictures in ./results/pose/test_lates_finetune/02-0_01/synthesized/. Is it ok and I have just need to wait more? How many epoches I need to wait, before I could see any changes?

How to run 'n_shot' inference?

I have noticed that there is an "n_shot" parameter in the base_options.py. When testing the pose datasets, I have 5 reference images to be pose transfered, so I set "n_shot" to 5. Howevery, I got only one reference image in the result directory. I wonder if this means that only one reference image is used for testing？Is there any advice for me？

pretrained model?

Any pretrained model for prediction?

GPU requirements?

What sort of GPU would be required to run few-shot-vid2vid? Would a Geforce 1050 or 1080 be sufficient?

How to train on multi-nodes?

I noticed the parameter 'distributed' in base_options.py. Howevery, when I add this parameter to train on multiple nodes and multiple GPUs, the following error ocurs. Could anyone tell me how to solve it?

Have problem after epoch 50.

After notification : "---------- Now start training multiple frames -------------"
i wait approximately 12 hours, but it do not run to next epoch. Can you explain this problem. i run on AWS instance have 2 GPU Tesla M60 8gb.

Problem at background in face video output

Hi guys,
I have some problems when using few shot vid2vid. After face traning, I run the testing with driving video in FaceForensics dataset. At the video output, face is good but the background
oscillate, not stable. Anyone meet my problem and how can I fix that?
Thankyou

Import error during training

Hi, when I train the model as recommended in the README (python train.py --name pose --dataset_mode fewshot_pose --adaptive_spade --warp_ref --spade_combine --remove_face_labels --add_face_D --niter_single 100 --niter 200 --batchSize 2), I get the error import resample2d_cuda due to an undefined symbol (full output below). Does anyone have tips on how to fix that? I followed the recommended steps to compile a snapshot of FlowNet2.

File "train.py", line 34, in train
    model, flowNet = create_model(opt, trainer.start_epoch)
  File "/home/alberto/Documents/few-shot-vid2vid/models/models.py", line 27, in create_model
    flowNet.initialize(opt)
  File "/home/alberto/Documents/few-shot-vid2vid/models/flownet.py", line 23, in initialize
    from .networks.flownet2_pytorch import models as flownet2_models
  File "/home/alberto/Documents/few-shot-vid2vid/models/networks/flownet2_pytorch/models.py", line 8, in <module>
    from .networks.resample2d_package.resample2d import Resample2d
  File "/home/alberto/Documents/few-shot-vid2vid/models/networks/flownet2_pytorch/networks/resample2d_package/resample2d.py", line 3, in <module>
    import resample2d_cuda
ImportError: /home/alberto/.local/lib/python3.6/site-packages/resample2d_cuda-0.0.0-py3.6-linux-x86_64.egg/resample2d_cuda.cpython-36m-x86_64-linux-gnu.so: undefined symbol: _ZN3c105ErrorC1ENS_14SourceLocationERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE

Very bad results

Getting very bad results even for the first 50 epochs.

At 15 epochs:

At 30 epochs:

At 50 epochs:

From start my f_flow value is very high but I am not sure if it has something to do with the issue and if yes what exactly is wrong.

Log File:
loss_log.txt

fake_raw_image is None in training pose

I run bash ./scripts/pose/train_g1.sh and the raw_image (fake_raw_image) is None.
Is it normal ？

Dance videos for multiple people

Hi, you said random dance videos from youtube, but does it have to be a solo dance? Or is it okay to train or validate with dance videos that have multiple people? Thank you for your time in advance.

TypeError: 'numpy.float64' object cannot be interpreted as an integer

I am on Ubuntu 18.04. It seems that there are some issues with DataLoader?

(few-shot-new) ~/few-shot-vid2vid$ bash ./scripts/pose/train_g1.sh
CustomDatasetDataLoader
2 sequences
dataset [PoseDataset] was created
5000
Resuming from epoch 1 at iteration 0
create web directory ./checkpoints/pose/web...
---------- Networks initialized -------------
---------- Optimizers initialized -------------
./checkpoints/pose/latest_net_G.pth not exists yet!
./checkpoints/pose/latest_net_D.pth not exists yet!
./checkpoints/pose/latest_net_Df.pth not exists yet!
model [Vid2VidModel] was created
Traceback (most recent call last):
  File "train.py", line 73, in <module>
    train()
  File "train.py", line 40, in train
    for idx, data in enumerate(dataset, start=trainer.epoch_iter):
  File "/home/chuan/anaconda3/envs/few-shot-new/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 345, in __next__
    data = self._next_data()
  File "/home/chuan/anaconda3/envs/few-shot-new/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 856, in _next_data
    return self._process_data(data)
  File "/home/chuan/anaconda3/envs/few-shot-new/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 881, in _process_data
    data.reraise()
  File "/home/chuan/anaconda3/envs/few-shot-new/lib/python3.7/site-packages/torch/_utils.py", line 394, in reraise
    raise self.exc_type(msg)
TypeError: Caught TypeError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/home/chuan/anaconda3/envs/few-shot-new/lib/python3.7/site-packages/numpy/core/function_base.py", line 117, in linspace
    num = operator.index(num)
TypeError: 'numpy.float64' object cannot be interpreted as an integer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/chuan/anaconda3/envs/few-shot-new/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 178, in _worker_loop
    data = fetcher.fetch(index)
  File "/home/chuan/anaconda3/envs/few-shot-new/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/chuan/anaconda3/envs/few-shot-new/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/chuan/few-shot-vid2vid/data/fewshot_pose_dataset.py", line 91, in __getitem__
    idx, ref_size, img_params, self.ref_crop_coords[i])
  File "/home/chuan/few-shot-vid2vid/data/fewshot_pose_dataset.py", line 129, in get_images
    O, op, crop_coords, face_pts = self.get_image(op_path, size, params, crop_coords, input_type='openpose', ref_face_pts=ref_face_pts)
  File "/home/chuan/few-shot-vid2vid/data/fewshot_pose_dataset.py", line 149, in get_image
    opt.basic_point_only, opt.remove_face_labels, ppl_idx, ref_face_pts)
  File "/home/chuan/few-shot-vid2vid/data/keypoint2img.py", line 48, in read_keypoints
    pose_img = connect_keypoints(opt, pts, edge_lists, size, random_drop_prob, distort_points, basic_point_only, remove_face_labels)
  File "/home/chuan/few-shot-vid2vid/data/keypoint2img.py", line 92, in connect_keypoints
    curve_x, curve_y = interpPoints(x, y)
  File "/home/chuan/few-shot-vid2vid/data/keypoint2img.py", line 309, in interpPoints
    curve_x = np.linspace(x[0], x[-1], (x[-1]-x[0]))
  File "<__array_function__ internals>", line 6, in linspace
  File "/home/chuan/anaconda3/envs/few-shot-new/lib/python3.7/site-packages/numpy/core/function_base.py", line 121, in linspace
    .format(type(num)))
TypeError: object of type <class 'numpy.float64'> cannot be safely interpreted as an integer.

./scripts/pose/train_g1.sh: line 10: 20085 Segmentation fault      (core dumped) python train.py --name pose --dataset_mode fewshot_pose --adaptive_spade --warp_ref --spade_combine --remove_face_labels --add_face_D --niter_single 100 --niter 200 --batchSize 2 --continue_train

Have anybody successfully reproduced the performance as in example videos?

I tried to train the face model step by step exactly follow the guide, with the default hyper params several times, but all models collapsed with producing bad images after 50 epochs.
I also tried to adjust some hyper params like lr, batchsize, niter_step, still failed.
Kindly asking for somebody who have successfully trained a face model share some experience with me . Best regards!

tile cannot extend outside image

I am trying to run few shot pose on my data, I tried running it on small batches of data, it ran successfully. But, when I tried to train it on full dataset then it gave me the following error

Run command

python train.py --dataroot ./datasets/cvdata/09 --name 09_data --dataset_mode fewshot_pose --adaptive_spade --warp_ref --spade_combine --remove_face_labels --add_face_D --niter_single 100 --niter 200 --batchSize 4 --gpu_ids 2

Error output

CustomDatasetDataLoader
1 sequences
dataset [PoseDataset] was created
create web directory ./checkpoints/09_data/web...
---------- Networks initialized -------------
---------- Optimizers initialized -------------
model [Vid2VidModel] was created
/home/developer/anaconda3/envs/fewshot-dev/lib/python3.7/site-packages/torch/nn/functional.py:2479: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
  "See the documentation of nn.Upsample for details.".format(mode))
Traceback (most recent call last):
  File "train.py", line 73, in <module>
    train()
  File "train.py", line 40, in train
    for idx, data in enumerate(dataset, start=trainer.epoch_iter):
  File "/home/developer/anaconda3/envs/fewshot-dev/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 819, in __next__
    return self._process_data(data)
  File "/home/developer/anaconda3/envs/fewshot-dev/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 846, in _process_data
    data.reraise()
  File "/home/developer/anaconda3/envs/fewshot-dev/lib/python3.7/site-packages/torch/_utils.py", line 369, in reraise
    raise self.exc_type(msg)
ValueError: Caught ValueError in DataLoader worker process 1.
Original Traceback (most recent call last):
  File "/home/developer/anaconda3/envs/fewshot-dev/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 178, in _worker_loop
    data = fetcher.fetch(index)
  File "/home/developer/anaconda3/envs/fewshot-dev/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/developer/anaconda3/envs/fewshot-dev/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/developer/workspace/dance/few-shot-vid2vid/data/fewshot_pose_dataset.py", line 109, in __getitem__
    crop_coords, self.ref_face_pts)
  File "/home/developer/workspace/dance/few-shot-vid2vid/data/fewshot_pose_dataset.py", line 129, in get_images
    O, op, crop_coords, face_pts = self.get_image(op_path, size, params, crop_coords, input_type='openpose', ref_face_pts=ref_face_pts)
  File "/home/developer/workspace/dance/few-shot-vid2vid/data/fewshot_pose_dataset.py", line 152, in get_image
    A_img, crop_coords = self.crop_person_region(A_img, crop_coords, pose_pts, size)
  File "/home/developer/workspace/dance/few-shot-vid2vid/data/fewshot_pose_dataset.py", line 182, in crop_person_region
    A_img = Image.fromarray(A_img[ys:ye, xs:xe, :])
  File "/home/developer/anaconda3/envs/fewshot-dev/lib/python3.7/site-packages/PIL/Image.py", line 2670, in fromarray
    return frombuffer(mode, size, obj, "raw", rawmode, 0, 1)
  File "/home/developer/anaconda3/envs/fewshot-dev/lib/python3.7/site-packages/PIL/Image.py", line 2613, in frombuffer
    return frombytes(mode, size, data, decoder_name, args)
  File "/home/developer/anaconda3/envs/fewshot-dev/lib/python3.7/site-packages/PIL/Image.py", line 2546, in frombytes
    im.frombytes(data, decoder_name, args)
  File "/home/developer/anaconda3/envs/fewshot-dev/lib/python3.7/site-packages/PIL/Image.py", line 829, in frombytes
    d.setimage(self.im)
ValueError: tile cannot extend outside image

I know there is some issue with my data but I am not able to figure out the exact problem, since it worked on small sample batches of my data.
Any help will be appreciated

Sizes of Tensors must match except in dimension 0

Been getting the following error around epoch 70, 71. Just updated with the latest commit. Can anybody tell me what it means?

Traceback (most recent call last):
  File "/home/videx/Desktop/vid2vid/few-shot-vid2vid/train.py", line 73, in <module>
    train()
  File "/home/videx/Desktop/vid2vid/few-shot-vid2vid/train.py", line 40, in train
    for idx, data in enumerate(dataset, start=trainer.epoch_iter):
  File "/usr/local/lib/python3.5/dist-packages/torch/utils/data/dataloader.py", line 637, in __next__
    return self._process_next_batch(batch)
  File "/usr/local/lib/python3.5/dist-packages/torch/utils/data/dataloader.py", line 658, in _process_next_batch
    raise batch.exc_type(batch.exc_msg)
RuntimeError: Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/torch/utils/data/dataloader.py", line 138, in _worker_loop
    samples = collate_fn([dataset[i] for i in batch_indices])
  File "/usr/local/lib/python3.5/dist-packages/torch/utils/data/dataloader.py", line 229, in default_collate
    return {key: default_collate([d[key] for d in batch]) for key in batch[0]}
  File "/usr/local/lib/python3.5/dist-packages/torch/utils/data/dataloader.py", line 229, in <dictcomp>
    return {key: default_collate([d[key] for d in batch]) for key in batch[0]}
  File "/usr/local/lib/python3.5/dist-packages/torch/utils/data/dataloader.py", line 209, in default_collate
    return torch.stack(batch, 0, out=out)
RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 0. Got 16 and 15 in dimension 1 at /pytorch/aten/src/TH/generic/THTensorMoreMath.cpp:1333

Thanks!

multi gpu training hangs in python threading

Using the training configuration below:

--name face_no_warpref_64 --dataset_mode fewshot_face --adaptive_spade --gpu_ids 0,1,2,3,4,5 --batchSize 224 --nThreads 12 --tf_log --niter_single 3000 --loadSize 128 --fineSize 128 --gan_mode ls --continue_train --which_epoch latest

It will hang in python threading incidentally, whether at the very begaining of the training process, after the first epoch, or sometime after several epochs

'''
Traceback (most recent call last):
File "/home/xxx/image_augmentation/gan/few-shot-vid2vid-master/train.py", line 72, in
train()
File "/home/xxx/image_augmentation/gan/few-shot-vid2vid-master/train.py", line 56, in train
d_losses = model(data_list_t, mode='discriminator')
File "/home/xxx/miniconda3/envs/py3.6_torch1.1.0/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/home/xxx/image_augmentation/gan/few-shot-vid2vid-master/models/models.py", line 86, in forward
outputs = self.model(*inputs, **kwargs, dummy_bs=self.pad_bs)
File "/home/liusiyao/miniconda3/envs/py3.6_torch1.1.0/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/home/xxx/miniconda3/envs/py3.6_torch1.1.0/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 152, in forward
outputs = self.parallel_apply(replicas, inputs, kwargs)
File "/home/xxx/miniconda3/envs/py3.6_torch1.1.0/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 162, in parallel_apply
return parallel_apply(replicas, inputs, kwargs, self.device_ids[:len(replicas)])
File "/home/xxx/miniconda3/envs/py3.6_torch1.1.0/lib/python3.6/site-packages/torch/nn/parallel/parallel_apply.py", line 75, in parallel_apply
thread.join()
File "/home/xxx/miniconda3/envs/py3.6_torch1.1.0/lib/python3.6/threading.py", line 1056, in join
self._wait_for_tstate_lock()
File "/home/xxx/miniconda3/envs/py3.6_torch1.1.0/lib/python3.6/threading.py", line 1072, in _wait_for_tstate_lock
elif lock.acquire(block, timeout):
KeyboardInterrupt
'''

I think this is a pytorch bug? I'm using pytorch 1.1.0 (because I'm not permitted to update the gpu driver) instead of 1.3.0, having 6 gtx titan v
Does anyone else have the same problem with me?

error running train.py, resample2d_cuda.forward() has incompatible function arguments.

/home/daniel/.local/lib/python3.6/site-packages/torch/nn/functional.py:2494: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
  "See the documentation of nn.Upsample for details.".format(mode))
Traceback (most recent call last):
  File "train.py", line 74, in <module>
    train()
  File "train.py", line 46, in train
    flow_gt, conf_gt = flowNet(data_list, epoch)
  File "/home/daniel/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/daniel/project/010_GAN/201_vid2vid/few-shot-vid2vid/models/models.py", line 90, in forward
    outputs = self.model(*inputs, **kwargs, dummy_bs=self.pad_bs)
  File "/home/daniel/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/daniel/.local/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 150, in forward
    return self.module(*inputs[0], **kwargs[0])
  File "/home/daniel/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/daniel/project/010_GAN/201_vid2vid/few-shot-vid2vid/models/flownet.py", line 48, in forward
    flow_gt_ref, conf_gt_ref = self.flowNet_forward(image_now, image_ref.expand_as(image_now))              
  File "/home/daniel/project/010_GAN/201_vid2vid/few-shot-vid2vid/models/flownet.py", line 60, in flowNet_forward
    flow, conf = self.compute_flow_and_conf(input_A, input_B)
  File "/home/daniel/project/010_GAN/201_vid2vid/few-shot-vid2vid/models/flownet.py", line 75, in compute_flow_and_conf
    flow1 = self.flowNet(data1)
  File "/home/daniel/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/daniel/project/010_GAN/201_vid2vid/few-shot-vid2vid/models/networks/flownet2_pytorch/models.py", line 130, in forward
    resampled_img1 = self.resample1(x[:,3:,:,:], flownetc_flow)
  File "/home/daniel/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/daniel/project/010_GAN/201_vid2vid/few-shot-vid2vid/models/networks/flownet2_pytorch/networks/resample2d_package/resample2d.py", line 46, in forward
    return Resample2dFunction.apply(input1_c, input2, self.kernel_size)
  File "/home/daniel/project/010_GAN/201_vid2vid/few-shot-vid2vid/models/networks/flownet2_pytorch/networks/resample2d_package/resample2d.py", line 19, in forward
    resample2d_cuda.forward(input1, input2, output, kernel_size)
TypeError: forward(): incompatible function arguments. The following argument types are supported:
    1. (arg0: at::Tensor, arg1: at::Tensor, arg2: at::Tensor, arg3: int, arg4: bool) -> int

error in correlation_forward_cuda_kernel: invalid device function (URGENT HELP REQ)

Please help me, its urgent. My CUDA version is 10.1 , pytorch 1.3.1 , python 3.6.9 .Whenever I try to run the train script it returns the following error:

error in correlation_forward_cuda_kernel: invalid device function
Traceback (most recent call last):
File "train.py", line 73, in
train()
File "train.py", line 45, in train
flow_gt, conf_gt = flowNet(data_list, epoch)
File "/HPS/DFD_19/work/anaconda3/envs/myenv/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in call
result = self.forward(*input, **kwargs)
File "/HPS/DFD_19/work/few-shot-vid2vid/models/models.py", line 90, in forward
outputs = self.model(*inputs, **kwargs, dummy_bs=self.pad_bs)
File "/HPS/DFD_19/work/anaconda3/envs/myenv/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in call
result = self.forward(*input, **kwargs)
File "/HPS/DFD_19/work/anaconda3/envs/myenv/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 150, in forward
return self.module(*inputs[0], **kwargs[0])
File "/HPS/DFD_19/work/anaconda3/envs/myenv/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in call
result = self.forward(*input, **kwargs)
File "/HPS/DFD_19/work/few-shot-vid2vid/models/flownet.py", line 48, in forward
flow_gt_ref, conf_gt_ref = self.flowNet_forward(image_now, image_ref.expand_as(image_now))
File "/HPS/DFD_19/work/few-shot-vid2vid/models/flownet.py", line 60, in flowNet_forward
flow, conf = self.compute_flow_and_conf(input_A, input_B)
File "/HPS/DFD_19/work/few-shot-vid2vid/models/flownet.py", line 75, in compute_flow_and_conf
flow1 = self.flowNet(data1)
File "/HPS/DFD_19/work/anaconda3/envs/myenv/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in call
result = self.forward(*input, **kwargs)
File "/HPS/DFD_19/work/few-shot-vid2vid/models/networks/flownet2_pytorch/models.py", line 126, in forward
flownetc_flow2 = self.flownetc(x)[0]
File "/HPS/DFD_19/work/anaconda3/envs/myenv/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in call
result = self.forward(*input, **kwargs)
File "/HPS/DFD_19/work/few-shot-vid2vid/models/networks/flownet2_pytorch/networks/FlowNetC.py", line 86, in forward
out_corr = self.corr(out_conv3a, out_conv3b) # False
File "/HPS/DFD_19/work/anaconda3/envs/myenv/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in call
result = self.forward(*input, kwargs)
File "/HPS/DFD_19/work/few-shot-vid2vid/models/networks/flownet2_pytorch/networks/correlation_package/correlation.py", line 69, in forward
self.stride1, self.stride2, self.corr_multiply)
File "/HPS/DFD_19/work/few-shot-vid2vid/models/networks/flownet2_pytorch/networks/correlation_package/correlation.py", line 34, in forward
ctx.pad_size, ctx.kernel_size, ctx.max_displacement,ctx.stride1, ctx.stride2, ctx.corr_multiply)
RuntimeError: CUDA call failed (correlation_forward_cuda at correlation_cuda.cc:82)
frame #0: c10::Error::Error(c10::SourceLocation, std::string const&) + 0x47 (0x7f7e28ef2687 in /HPS/DFD_19/work/anaconda3/envs/myenv/lib/python3.6/site-packages/torch/lib/libc10.so)
frame #1: correlation_forward_cuda(at::Tensor&, at::Tensor&, at::Tensor&, at::Tensor&, at::Tensor&, int, int, int, int, int, int) + 0x58f (0x7f7d7e4c62af in /home/jnehvi/.local/lib/python3.6/site-packages/correlation_cuda-0.0.0-py3.6-linux-x86_64.egg/correlation_cuda.cpython-36m-x86_64-linux-gnu.so)
frame #2: + 0x1c425 (0x7f7d7e4d4425 in /home/jnehvi/.local/lib/python3.6/site-packages/correlation_cuda-0.0.0-py3.6-linux-x86_64.egg/correlation_cuda.cpython-36m-x86_64-linux-gnu.so)
frame #3: + 0x1c6ae (0x7f7d7e4d46ae in /home/jnehvi/.local/lib/python3.6/site-packages/correlation_cuda-0.0.0-py3.6-linux-x86_64.egg/correlation_cuda.cpython-36m-x86_64-linux-gnu.so)
frame #4: + 0x19851 (0x7f7d7e4d1851 in /home/jnehvi/.local/lib/python3.6/site-packages/correlation_cuda-0.0.0-py3.6-linux-x86_64.egg/correlation_cuda.cpython-36m-x86_64-linux-gnu.so)

frame #11: THPFunction_apply(_object, _object) + 0x9ff (0x7f7e2995490f in /HPS/DFD_19/work/anaconda3/envs/myenv/lib/python3.6/site-packages/torch/lib/libtorch_python.so)

Segmentation fault

Could anybody provide me the pretrained models?

especially the pose model? my graphic card is not good enough to train data, tks a lot!
u could mail [email protected]