nbei/deep-flow-guided-video-inpainting

pytorch implementation for "Deep Flow-Guided Video Inpainting"(CVPR'19)

deep-flow-guided-video-inpainting's Issues

Please check the results!!!

I used following command to test the code.
python tools/frame_inpaint.py --test_img xxx.png --test_mask xxx.png

I am getting the following results? Can you tell me why the glass is not properly in-painted?

Similarly look here:

About learning method of original video

please tell me.

1. I want to learn from my input data (frame_dir, masks_dir). Can I learn the image size 960 x 540 of the input data?

2. Should I give an image list (frames cut out of the video) and a mask (corresponding to a frame) for learning?

3. If you want to train, please tell me how to use command_line arguments using train_initial.py and train_refine.py in the tools directory.

$ python tool/train_initial.py ...　or $ python tool/train_refine.py ...

I do not know how. please tell me.

the compile environment error

I excute the following the shell and the error always happened.would you like to help me solve it.

bash install_scripts.sh

Traceback (most recent call last):
File "build.py", line 31, in
ffi.build()
File "C:\Users\IBM_ADMIN\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\utils\ffi_init_.py", line 184, in build
build_extension(ffi, cffi_wrapper_name, target_dir, verbose)
File "C:\Users\IBM_ADMIN\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\utils\ffi_init.py", line 108, in _build_extension
outfile = ffi.compile(tmpdir=tmpdir, verbose=verbose, target=libname)
File "C:\Users\IBM_ADMIN\AppData\Local\Programs\Python\Python36\lib\site-packages\cffi\api.py", line 723, in compile
compiler_verbose=verbose, debug=debug, **kwds)
File "C:\Users\IBM_ADMIN\AppData\Local\Programs\Python\Python36\lib\site-packages\cffi\recompiler.py", line 1526, in recompile
compiler_verbose, debug)
File "C:\Users\IBM_ADMIN\AppData\Local\Programs\Python\Python36\lib\site-packages\cffi\ffiplatform.py", line 22, in compile
outputfilename = _build(tmpdir, ext, compiler_verbose, debug)
File "C:\Users\IBM_ADMIN\AppData\Local\Programs\Python\Python36\lib\site-packages\cffi\ffiplatform.py", line 58, in _build
raise VerificationError('%s: %s' % (e.class.name, e))
cffi.VerificationError: LinkError: command 'C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\link.exe' failed with exit status 1120

about flownet

I run this code and the following errors occurred.
I want to know if NVIDIA display card(hardware) is necessary. My computer is Intel card.

This Video has 80 Images
The optical flow list has been generated: ./demo\video_flow.txt
====> Loading ./pretrained_models/FlowNet2_checkpoint.pth.tar
THCudaCheck FAIL file=..\aten\src\THC\THCGeneral.cpp line=74 error=35 : CUDA driver version is insufficient for CUDA runtime version
Traceback (most recent call last):
File "tools/video_inpaint.py", line 184, in
main()
File "tools/video_inpaint.py", line 165, in main
extract_flow(args)
File "tools/video_inpaint.py", line 85, in extract_flow
output_file = infer(args)
File "C:\diskD\deep-flow\Deep-Flow-Guided-Video-Inpainting-master\tools\infer_flownet2.py", line 42, in infer
Flownet.to(device)
File "C:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 379, in to
return self._apply(convert)
File "C:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 185, in _apply
module._apply(fn)
File "C:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 185, in _apply
module._apply(fn)
File "C:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 185, in _apply
module._apply(fn)
File "C:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 191, in _apply
param.data = fn(param.data)
File "C:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 377, in convert
return t.to(device, dtype if t.is_floating_point() else None, non_blocking)
RuntimeError: cuda runtime error (35) : CUDA driver version is insufficient for CUDA runtime version at ..\aten\src\THC\THCGeneral.cpp:74

Installing Flownet

While installing Flownet getting this error

Compiling resample2d kernels by nvcc... rm: cannot remove 'Resample2d_kernel.o': No such file or directory Compiling correlation kernels by nvcc... rm: cannot remove 'correlation_cuda_kernel.o': No such file or directory Compiling channelnorm kernels by nvcc... rm: cannot remove 'ChannelNorm_kernel.o': No such file or directory

nvcc version:
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2015 NVIDIA Corporation
Built on Tue_Aug_11_14:27:32_CDT_2015
Cuda compilation tools, release 7.5, V7.5.17

can you tell me how to train this model?

About generating mask

Thanks for doing such a great job!
And I have a little question about mask. I found that the project contained deep flow model, and I checked some intermediate results (heatmaps or something) of deep flow model, some of them are perfectly just like ground truth mask. Have you tried to use deep flow model in this project to generating mask? Is this method a feasible one?

How to train the deepfill model?

I didn't see the deepfill in training guide,it seems only about training flownet.
Could you tell me what should i do if i want to train the deepfill model alone?

Questions about the network structure

Hi, thanks for your great work and sharing the code.

I'm confused about the network structure. For subnetwork1, the output channel is 2, it's the complete flow of center frame right? But the input channel of subnetwork2 is 66, specified in train_refine.py. Also, the paper writes "For each pair of the consecutive frames, i frame and i+1 frame, the second subnetwork takes a sequence of estimated bidirectional flow", and said the flow is generated by first subnetwork? So where this bidirectional flow come from? Is the flow generated by FlowNet2? And you replace the flow of center frame by the result of subnetwork1?

Looking forward to your reply!

where can I find the PRETRAINED_MODEL_2 and PRETRAINED_MODEL_3

I can download PRETRAINED_MODEL_1 which is resnet101_movie.pth
but I don't konw which is PRETRAINED_MODEL_2 and which is PRETRAINED_MODEL_3
thanks!

picture number of inpaint_res folder

I found that the the picture number of video inpainting.py's output do not match the input of frames and masks.
Such as the demo ,the input picture number is 81 but the output picture number is 80,could you tell me the reason?

how can i get the dataset?

The input image resolution

Hey, your works are amazing!
I tried the data you provides in the demo folder, and I wonder what's the resolution requirements for input image with "python tools/video_inpaint.py"?

I believe your original image is 480x854, but in your example command "python tools/video_inpaint.py --frame_dir ./demo/frames --MASK_ROOT ./demo/masks --image_size 512 832 --FlowNet2 --DFC --ResNet101 --Propagation", it's 512x832.

So I tried the original resolution, with the command "python tools/video_inpaint.py --frame_dir ./demo/frames --MASK_ROOT ./demo/masks --image_size 480 854 --FlowNet2 --DFC --ResNet101 --Propagation"
but I got this error:

Traceback (most recent call last):
File "tools/video_inpaint.py", line 183, in
main()
File "tools/video_inpaint.py", line 164, in main
extract_flow(args)
File "tools/video_inpaint.py", line 84, in extract_flow
output_file = infer(args)
File "/data/flow_inpaint/tools/infer_flownet2.py", line 53, in infer
flow = Flownet(f1, f2)
File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 491, in call
result = self.forward(*input, **kwargs)
File "/data/flow_inpaint/models/FlowNet2_Models/FlowNet2.py", line 131, in forward
flownetc_flow2 = self.flownetc(x)[0]
File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 491, in call
result = self.forward(*input, **kwargs)
File "/data/flow_inpaint/models/FlowNet2_Models/FlowNetC.py", line 110, in forward
concat5 = torch.cat((out_conv5,out_deconv5,flow6_up),1)
RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 1. Got 16 and 15 in dimension 2 at /pytorch/aten/src/THC/generic/THCTensorMath.cu:111

problem when install flownet

when i run command bash install_scripts.sh, I get the following error messages:

Compiling resample2d kernels by nvcc...
rm: cannot remove 'Resample2d_kernel.o': No such file or directory
rm: cannot remove '../_ext': No such file or directory
./make.sh: line 9: nvcc: command not found
Traceback (most recent call last):
File "build.py", line 3, in
import torch.utils.ffi
File "/home/ubuntu/.local/lib/python3.6/site-packages/torch/utils/ffi/init.py", line 1, in
raise ImportError("torch.utils.ffi is deprecated. Please use cpp extensions instead.")
ImportError: torch.utils.ffi is deprecated. Please use cpp extensions instead.
Compiling correlation kernels by nvcc...
rm: cannot remove 'correlation_cuda_kernel.o': No such file or directory
rm: cannot remove '../_ext': No such file or directory
./make.sh: line 11: nvcc: command not found
Traceback (most recent call last):
File "build.py", line 3, in
import torch.utils.ffi
File "/home/ubuntu/.local/lib/python3.6/site-packages/torch/utils/ffi/init.py", line 1, in
raise ImportError("torch.utils.ffi is deprecated. Please use cpp extensions instead.")
ImportError: torch.utils.ffi is deprecated. Please use cpp extensions instead.
Compiling channelnorm kernels by nvcc...
rm: cannot remove 'ChannelNorm_kernel.o': No such file or directory
rm: cannot remove '../_ext': No such file or directory
./make.sh: line 9: nvcc: command not found
Traceback (most recent call last):
File "build.py", line 3, in
import torch.utils.ffi
File "/home/ubuntu/.local/lib/python3.6/site-packages/torch/utils/ffi/init.py", line 1, in
raise ImportError("torch.utils.ffi is deprecated. Please use cpp extensions instead.")
ImportError: torch.utils.ffi is deprecated. Please use cpp extensions instead.

how can i do fixed region inpainting

for object removing ,i can use this:

To use our video inpainting tool for object removing, we recommend that the frames should be put into xxx/video_name/frames and the mask of each frame should be put into xxx/video_name/masks. And please download the resources of the demo and model weights from here. An example demo containing frames and masks has been put into the demo and running the following command will get the result:
python tools/video_inpaint.py --frame_dir ./demo/frames --MASK_ROOT ./demo/masks --image_size 512 832 --FlowNet2 --DFC --ResNet101 --Propagation

but what about fixed region inpainting?

Cuda version and Operating System?

Hi, can you specify which CUDA toolkit you installed (8,9,10) and the operating system that you used. I'm having constant issues trying to get the demo running and I think it's due to the wrong cuda version for pytorch 0.4.0

problem when install scripts.sh

Hello! I am a beginner, can you tell me how to solve it?
Thanks a lot!

roo@lab-Lenovo:~/下载/Deep-Flow-Guided-Video-Inpainting-master$ bash install_scripts.sh
Traceback (most recent call last):
File "", line 1, in
ImportError: No module named torch
Compiling resample2d kernels by nvcc...
rm: 无法删除'Resample2d_kernel.o': 没有那个文件或目录
rm: 无法删除'../_ext': 没有那个文件或目录
./make.sh: 行 9: nvcc: 未找到命令
Traceback (most recent call last):
File "build.py", line 2, in
import torch
ImportError: No module named torch
Traceback (most recent call last):
File "", line 1, in
ImportError: No module named torch
Compiling correlation kernels by nvcc...
rm: 无法删除'correlation_cuda_kernel.o': 没有那个文件或目录
rm: 无法删除'../_ext': 没有那个文件或目录
./make.sh: 行 11: nvcc: 未找到命令
Traceback (most recent call last):
File "build.py", line 2, in
import torch
ImportError: No module named torch
install_scripts.sh: 行 7: ./make.sh: 权限不够

RuntimeError: cuda runtime error (30)

THCudaCheck FAIL file=/pytorch/aten/src/THC/THCGeneral.cpp line=70 error=30 : unknown error
Traceback (most recent call last):
File "tools/video_inpaint.py", line 184, in
main()
File "tools/video_inpaint.py", line 165, in main
extract_flow(args)
File "tools/video_inpaint.py", line 85, in extract_flow
output_file = infer(args)
File "/home/Deep-Flow-Guided-Video-Inpainting-master/tools/infer_flownet2.py", line 42, in infer
Flownet.to(device)
File "/home/anaconda3/envs/deep_flow_guided/lib/python3.6/site-packages/torch/nn/modules/module.py", line 393, in to
return self._apply(lambda t: t.to(device))
File "/home/anaconda3/envs/deep_flow_guided/lib/python3.6/site-packages/torch/nn/modules/module.py", line 176, in _apply
module._apply(fn)
File "/home/anaconda3/envs/deep_flow_guided/lib/python3.6/site-packages/torch/nn/modules/module.py", line 176, in _apply
module._apply(fn)
File "/home/anaconda3/envs/deep_flow_guided/lib/python3.6/site-packages/torch/nn/modules/module.py", line 176, in _apply
module._apply(fn)
File "/home/anaconda3/envs/deep_flow_guided/lib/python3.6/site-packages/torch/nn/modules/module.py", line 182, in _apply
param.data = fn(param.data)
File "/home/anaconda3/envs/deep_flow_guided/lib/python3.6/site-packages/torch/nn/modules/module.py", line 393, in
return self._apply(lambda t: t.to(device))
File "/home/anaconda3/envs/deep_flow_guided/lib/python3.6/site-packages/torch/cuda/init.py", line 161, in _lazy_init
torch._C._cuda_init()
RuntimeError: cuda runtime error (30) : unknown error at /pytorch/aten/src/THC/THCGeneral.cpp:70

AttributeError: 'ResNet' object has no attribute 'isRefine'

When I tried to run the demo code with the pretrained model, I got this error "AttributeError: 'ResNet' object has no attribute 'isRefine' ".
I am a newbie and I don't know how to solve it.
Is there anyone who knows how to solve this problem? Thanks!

How can I train this model pytorch version > 0.4.0?

I work with RTX20.., which is unable to install pytorch0.4.0, any suggestion will be helpful

Why do you set require_grad to false in BatchNorm layers when training?

bn1,bn2,bn3 In resnet_models.Bottleneck

cv2.error: OpenCV(4.1.0) /io/opencv/modules/imgproc/src/resize.cpp:3718: error: (-215:Assertion failed) !ssize.empty() in function 'resize'

when I run
python ./tools/video_inpaint.py --frame_dir ./demo/demo/spider-man/origin/2/ --MASK_ROOT ./demo/demo/spider-man/masks/2/ --img_size 512 832 --FlowNet2 --DFC --ResNet101 --Propagation

This Video has 75 Images
The optical flow list has been generated: ./demo/demo/spider-man/origin/2/video_flow.txt
====> Loading ./pretrained_models/FlowNet2_checkpoint.pth.tar
[ ] 0/148, elapsed: 0s, ETA:/home/hhl/.conda/envs/video/lib/python3.6/site-packages/torch/nn/functional.py:1749: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
"See the documentation of nn.Upsample for details.".format(mode))
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 148/148, 4.6 task/s, elapsed: 32s, ETA: 0sFlowNet2 Inference has been finished~!
Extracted Flow has been save in ./demo/demo/spider-man/origin/2/Flow
Generate datalist for initial step
Flow Completion in First Step
Freezing BatchNorm2D.
Load Pretrained Model from ./pretrained_models/resnet101_movie.pth
[ ] 0/149, elapsed: 0s, ETA:/home/hhl/.conda/envs/video/lib/python3.6/site-packages/torch/nn/functional.py:1749: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
"See the documentation of nn.Upsample for details.".format(mode))
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 149/149, 7.6 task/s, elapsed: 19s, ETA: 0sInitial Results Saved in ./demo/demo/spider-man/origin/2/Flow_res/initial_res
Load Deepfill Model from ./pretrained_models/imagenet_deepfill.pth
./demo/demo/spider-man/origin/2/
[512, 832]
./demo/demo/spider-man/masks/2/
Flow Start no 238
80 frames need to be inpainted.
The output shape is: [512, 832]
Iter 0 Forward Propagation
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ] 75/80, 2.5 task/s, elapsed: 30s, ETA: 2sTraceback (most recent call last):
File "./tools/video_inpaint.py", line 184, in
main()
File "./tools/video_inpaint.py", line 180, in main
flow_guided_propagation(args)
File "./tools/video_inpaint.py", line 156, in flow_guided_propagation
frame_inapint_model=deepfill_model)
File "/home/hhl/Deep-Flow-Guided-Video-Inpainting/tools/propagation_inpaint.py", line 126, in propagation
image = cv2.resize(image, (shape[1], shape[0]))
cv2.error: OpenCV(4.1.0) /io/opencv/modules/imgproc/src/resize.cpp:3718: error: (-215:Assertion failed) !ssize.empty() in function 'resize'

mmcv error

My environment is macOS Mojave 10.14.4, Anaconda 4.4.0,Python 3.6.1.
When I type "pip install -r requirements.txt", I got:
"Running setup.py clean for mmcv
Failed to build mmcv
Installing collected packages: mmcv
Running setup.py install for mmcv ... error" and :
"In file included from ./mmcv/video/optflow_warp/flow_warp.cpp:1:
./mmcv/video/optflow_warp/flow_warp.hpp:3:10: fatal error: 'iostream' file not found
#include "
this error also happens when I directly use "pip install mmcv"
Anybody help? Thank you very much.

Why does this work design a module for missing flow complement?

Hi, this work is very impressive, but I have some questions.
For video object removal (the main goal of this work), usually we have the original video and video with missing hole. The optimal flow can be obtained through original video. Why do we need to design a model for missing flow complement?

Thanks so much,

SIGSEGV: Segmentation fault

A “SIGSEGV: Segmentation fault” error occurred while running video_inpaint.py

About Video Mask

When I tested the demo video, I found that the mask pictures in the "/demo/masks" folder look like they have "border", not just directly use the video frames and masks in DAVIS dataset.
Can you tell me how you design the mask area for inpainting ? Thanks!

Experimental Details

The results are very impressive, I like it very much and thank you for sharing!
Would you mind giving me more details as I missed them in your paper or this repo:

1. What's the spatial size during training for each dataset (Youtube-VOS and DAVIS)?
2. What's the spatial size for numeric evaluation and visual results shown in the paper for each dataset (Youtube-VOS and DAVIS)?
3. How many frames of each testing video do you use for numeric evaluation?
4. You said in the paper that you only use a subset of 90 videos of DAVIS for testing, would you mind sharing the list?
5. Do you use all test split of YouTube-VOS for numeric evaluation?
6. Actually, YouTube-VOS has dense object segmentation masks for training samples, why not split a subset from the training list for object removal evaluation?

Looking forward to your response and big thanks again!!

Input of flow-completion network

I want to use the resnet101 model to complete the input flow.
I think the input of the network is [flow0-x,flow0-y,flow1-x,flow1-y,flow2-x,flow2-y...........,mask0,mask1,mask2......]
However, I get the wrong completion output with the pre-trained model.
Is there any problem with the order of my input? Thanks for your help.

compile environment error

when i run command bash install_scripts.sh, I get the following error messages:
Compiling resample2d kernels by nvcc...
rm: 无法删除"Resample2d_kernel.o": 没有那个文件或目录
rm: 无法删除"../_ext": 没有那个文件或目录
In file included from /home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH/THVector.h:5:0,
from /home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH/TH.h:12,
from _resample2d.c:492:
/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH/THMath.h: 在函数‘TH_polevl’中:
/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH/THMath.h:134:3: 错误：只允许在 C99 模式下使用‘for’循环初始化声明
for (size_t i = 0; i <= len; i++) {
^
/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH/THMath.h:134:3: 附注：使用 -std=c99 或 -std=gnu99 来编译您的代码
/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH/THMath.h: 在函数‘TH_polevlf’中:
/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH/THMath.h:142:3: 错误：只允许在 C99 模式下使用‘for’循环初始化声明
for (size_t i = 0; i <= len; i++) {
^
/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH/THMath.h: 在函数‘TH_trigamma’中:
/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH/THMath.h:260:3: 错误：只允许在 C99 模式下使用‘for’循环初始化声明
for (int i = 0; i < 6; ++i) {
^
/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH/THMath.h: 在函数‘TH_trigammaf’中:
/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH/THMath.h:278:3: 错误：只允许在 C99 模式下使用‘for’循环初始化声明
for (int i = 0; i < 6; ++i) {
^
Traceback (most recent call last):
File "/home/c201/anaconda3/envs/flow/lib/python3.6/distutils/unixccompiler.py", line 118, in _compile
extra_postargs)
File "/home/c201/anaconda3/envs/flow/lib/python3.6/distutils/ccompiler.py", line 909, in spawn
spawn(cmd, dry_run=self.dry_run)
File "/home/c201/anaconda3/envs/flow/lib/python3.6/distutils/spawn.py", line 36, in spawn
_spawn_posix(cmd, search_path, dry_run=dry_run)
File "/home/c201/anaconda3/envs/flow/lib/python3.6/distutils/spawn.py", line 159, in _spawn_posix
% (cmd, exit_status))
distutils.errors.DistutilsExecError: command 'gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/cffi/ffiplatform.py", line 51, in _build
dist.run_command('build_ext')
File "/home/c201/anaconda3/envs/flow/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/home/c201/anaconda3/envs/flow/lib/python3.6/distutils/command/build_ext.py", line 339, in run
self.build_extensions()
File "/home/c201/anaconda3/envs/flow/lib/python3.6/distutils/command/build_ext.py", line 448, in build_extensions
self._build_extensions_serial()
File "/home/c201/anaconda3/envs/flow/lib/python3.6/distutils/command/build_ext.py", line 473, in _build_extensions_serial
self.build_extension(ext)
File "/home/c201/anaconda3/envs/flow/lib/python3.6/distutils/command/build_ext.py", line 533, in build_extension
depends=ext.depends)
File "/home/c201/anaconda3/envs/flow/lib/python3.6/distutils/ccompiler.py", line 574, in compile
self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
File "/home/c201/anaconda3/envs/flow/lib/python3.6/distutils/unixccompiler.py", line 120, in _compile
raise CompileError(msg)
distutils.errors.CompileError: command 'gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "build.py", line 31, in
ffi.build()
File "/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/torch/utils/ffi/init.py", line 184, in build
_build_extension(ffi, cffi_wrapper_name, target_dir, verbose)
File "/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/torch/utils/ffi/init.py", line 108, in _build_extension
outfile = ffi.compile(tmpdir=tmpdir, verbose=verbose, target=libname)
File "/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/cffi/api.py", line 723, in compile
compiler_verbose=verbose, debug=debug, **kwds)
File "/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/cffi/recompiler.py", line 1526, in recompile
compiler_verbose, debug)
File "/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/cffi/ffiplatform.py", line 22, in compile
outputfilename = _build(tmpdir, ext, compiler_verbose, debug)
File "/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/cffi/ffiplatform.py", line 58, in _build
raise VerificationError('%s: %s' % (e.class.name, e))
cffi.VerificationError: CompileError: command 'gcc' failed with exit status 1
Compiling correlation kernels by nvcc...
rm: 无法删除"correlation_cuda_kernel.o": 没有那个文件或目录
rm: 无法删除"../_ext": 没有那个文件或目录
In file included from /home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH/THVector.h:5:0,
from /home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH/TH.h:12,
from _correlation.c:492:
/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH/THMath.h: 在函数‘TH_polevl’中:
/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH/THMath.h:134:3: 错误：只允许在 C99 模式下使用‘for’循环初始化声明
for (size_t i = 0; i <= len; i++) {
^
/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH/THMath.h:134:3: 附注：使用 -std=c99 或 -std=gnu99 来编译您的代码
/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH/THMath.h: 在函数‘TH_polevlf’中:
/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH/THMath.h:142:3: 错误：只允许在 C99 模式下使用‘for’循环初始化声明
for (size_t i = 0; i <= len; i++) {
^
/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH/THMath.h: 在函数‘TH_trigamma’中:
/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH/THMath.h:260:3: 错误：只允许在 C99 模式下使用‘for’循环初始化声明
for (int i = 0; i < 6; ++i) {
^
/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH/THMath.h: 在函数‘TH_trigammaf’中:
/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH/THMath.h:278:3: 错误：只允许在 C99 模式下使用‘for’循环初始化声明
for (int i = 0; i < 6; ++i) {
^
Traceback (most recent call last):
File "/home/c201/anaconda3/envs/flow/lib/python3.6/distutils/unixccompiler.py", line 118, in _compile
extra_postargs)
File "/home/c201/anaconda3/envs/flow/lib/python3.6/distutils/ccompiler.py", line 909, in spawn
spawn(cmd, dry_run=self.dry_run)
File "/home/c201/anaconda3/envs/flow/lib/python3.6/distutils/spawn.py", line 36, in spawn
_spawn_posix(cmd, search_path, dry_run=dry_run)
File "/home/c201/anaconda3/envs/flow/lib/python3.6/distutils/spawn.py", line 159, in _spawn_posix
% (cmd, exit_status))
distutils.errors.DistutilsExecError: command 'gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/cffi/ffiplatform.py", line 51, in _build
dist.run_command('build_ext')
File "/home/c201/anaconda3/envs/flow/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/home/c201/anaconda3/envs/flow/lib/python3.6/distutils/command/build_ext.py", line 339, in run
self.build_extensions()
File "/home/c201/anaconda3/envs/flow/lib/python3.6/distutils/command/build_ext.py", line 448, in build_extensions
self._build_extensions_serial()
File "/home/c201/anaconda3/envs/flow/lib/python3.6/distutils/command/build_ext.py", line 473, in _build_extensions_serial
self.build_extension(ext)
File "/home/c201/anaconda3/envs/flow/lib/python3.6/distutils/command/build_ext.py", line 533, in build_extension
depends=ext.depends)
File "/home/c201/anaconda3/envs/flow/lib/python3.6/distutils/ccompiler.py", line 574, in compile
self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
File "/home/c201/anaconda3/envs/flow/lib/python3.6/distutils/unixccompiler.py", line 120, in _compile
raise CompileError(msg)
distutils.errors.CompileError: command 'gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "build.py", line 31, in
ffi.build()
File "/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/torch/utils/ffi/init.py", line 184, in build
_build_extension(ffi, cffi_wrapper_name, target_dir, verbose)
File "/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/torch/utils/ffi/init.py", line 108, in _build_extension
outfile = ffi.compile(tmpdir=tmpdir, verbose=verbose, target=libname)
File "/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/cffi/api.py", line 723, in compile
compiler_verbose=verbose, debug=debug, **kwds)
File "/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/cffi/recompiler.py", line 1526, in recompile
compiler_verbose, debug)
File "/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/cffi/ffiplatform.py", line 22, in compile
outputfilename = _build(tmpdir, ext, compiler_verbose, debug)
File "/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/cffi/ffiplatform.py", line 58, in _build
raise VerificationError('%s: %s' % (e.class.name, e))
cffi.VerificationError: CompileError: command 'gcc' failed with exit status 1
Compiling channelnorm kernels by nvcc...
rm: 无法删除"ChannelNorm_kernel.o": 没有那个文件或目录
rm: 无法删除"../_ext": 没有那个文件或目录
In file included from /home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH/THVector.h:5:0,
from /home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH/TH.h:12,
from _channelnorm.c:492:
/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH/THMath.h: 在函数‘TH_polevl’中:
/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH/THMath.h:134:3: 错误：只允许在 C99 模式下使用‘for’循环初始化声明
for (size_t i = 0; i <= len; i++) {
^
/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH/THMath.h:134:3: 附注：使用 -std=c99 或 -std=gnu99 来编译您的代码
/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH/THMath.h: 在函数‘TH_polevlf’中:
/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH/THMath.h:142:3: 错误：只允许在 C99 模式下使用‘for’循环初始化声明
for (size_t i = 0; i <= len; i++) {
^
/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH/THMath.h: 在函数‘TH_trigamma’中:
/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH/THMath.h:260:3: 错误：只允许在 C99 模式下使用‘for’循环初始化声明
for (int i = 0; i < 6; ++i) {
^
/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH/THMath.h: 在函数‘TH_trigammaf’中:
/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH/THMath.h:278:3: 错误：只允许在 C99 模式下使用‘for’循环初始化声明
for (int i = 0; i < 6; ++i) {
^
Traceback (most recent call last):
File "/home/c201/anaconda3/envs/flow/lib/python3.6/distutils/unixccompiler.py", line 118, in _compile
extra_postargs)
File "/home/c201/anaconda3/envs/flow/lib/python3.6/distutils/ccompiler.py", line 909, in spawn
spawn(cmd, dry_run=self.dry_run)
File "/home/c201/anaconda3/envs/flow/lib/python3.6/distutils/spawn.py", line 36, in spawn
_spawn_posix(cmd, search_path, dry_run=dry_run)
File "/home/c201/anaconda3/envs/flow/lib/python3.6/distutils/spawn.py", line 159, in _spawn_posix
% (cmd, exit_status))
distutils.errors.DistutilsExecError: command 'gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/cffi/ffiplatform.py", line 51, in _build
dist.run_command('build_ext')
File "/home/c201/anaconda3/envs/flow/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/home/c201/anaconda3/envs/flow/lib/python3.6/distutils/command/build_ext.py", line 339, in run
self.build_extensions()
File "/home/c201/anaconda3/envs/flow/lib/python3.6/distutils/command/build_ext.py", line 448, in build_extensions
self._build_extensions_serial()
File "/home/c201/anaconda3/envs/flow/lib/python3.6/distutils/command/build_ext.py", line 473, in _build_extensions_serial
self.build_extension(ext)
File "/home/c201/anaconda3/envs/flow/lib/python3.6/distutils/command/build_ext.py", line 533, in build_extension
depends=ext.depends)
File "/home/c201/anaconda3/envs/flow/lib/python3.6/distutils/ccompiler.py", line 574, in compile
self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
File "/home/c201/anaconda3/envs/flow/lib/python3.6/distutils/unixccompiler.py", line 120, in _compile
raise CompileError(msg)
distutils.errors.CompileError: command 'gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "build.py", line 31, in
ffi.build()
File "/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/torch/utils/ffi/init.py", line 184, in build
_build_extension(ffi, cffi_wrapper_name, target_dir, verbose)
File "/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/torch/utils/ffi/init.py", line 108, in _build_extension
outfile = ffi.compile(tmpdir=tmpdir, verbose=verbose, target=libname)
File "/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/cffi/api.py", line 723, in compile
compiler_verbose=verbose, debug=debug, **kwds)
File "/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/cffi/recompiler.py", line 1526, in recompile
compiler_verbose, debug)
File "/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/cffi/ffiplatform.py", line 22, in compile
outputfilename = _build(tmpdir, ext, compiler_verbose, debug)
File "/home/c201/anaconda3/envs/flow/lib/python3.6/site-packages/cffi/ffiplatform.py", line 58, in _build
raise VerificationError('%s: %s' % (e.class.name, e))
cffi.VerificationError: CompileError: command 'gcc' failed with exit status 1

when install scripts.sh i got a problem

Hello! I'm a beginner, could you tell me how to solve it?
Thanks a lot!

(deep_flow_guided) throne@throne-GL553VD:~/Deep-Flow-Guided-Video-Inpainting-master (2)$ sudo bash install_scripts.sh
Compiling resample2d kernels by nvcc...
rm: 无法删除'Resample2d_kernel.o': 没有那个文件或目录
./make.sh: 行 9: nvcc: 未找到命令
x86_64-linux-gnu-gcc: error: /home/throne/Deep-Flow-Guided-Video-Inpainting-master (2)/models/FlowNet2_Models/resample2d_package/src/Resample2d_kernel.o: 没有那个文件或目录
Traceback (most recent call last):
File "/usr/lib/python3.5/distutils/unixccompiler.py", line 207, in link
self.spawn(linker + ld_args)
File "/usr/lib/python3.5/distutils/ccompiler.py", line 909, in spawn
spawn(cmd, dry_run=self.dry_run)
File "/usr/lib/python3.5/distutils/spawn.py", line 36, in spawn
_spawn_posix(cmd, search_path, dry_run=dry_run)
File "/usr/lib/python3.5/distutils/spawn.py", line 159, in _spawn_posix
% (cmd, exit_status))
distutils.errors.DistutilsExecError: command 'x86_64-linux-gnu-gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/cffi/ffiplatform.py", line 51, in _build
dist.run_command('build_ext')
File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/usr/lib/python3.5/distutils/command/build_ext.py", line 338, in run
self.build_extensions()
File "/usr/lib/python3.5/distutils/command/build_ext.py", line 447, in build_extensions
self._build_extensions_serial()
File "/usr/lib/python3.5/distutils/command/build_ext.py", line 472, in _build_extensions_serial
self.build_extension(ext)
File "/usr/lib/python3.5/distutils/command/build_ext.py", line 557, in build_extension
target_lang=language)
File "/usr/lib/python3.5/distutils/ccompiler.py", line 717, in link_shared_object
extra_preargs, extra_postargs, build_temp, target_lang)
File "/usr/lib/python3.5/distutils/unixccompiler.py", line 209, in link
raise LinkError(msg)
distutils.errors.LinkError: command 'x86_64-linux-gnu-gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "build.py", line 31, in
ffi.build()
File "/usr/local/lib/python3.5/dist-packages/torch/utils/ffi/init.py", line 184, in build
_build_extension(ffi, cffi_wrapper_name, target_dir, verbose)
File "/usr/local/lib/python3.5/dist-packages/torch/utils/ffi/init.py", line 108, in _build_extension
outfile = ffi.compile(tmpdir=tmpdir, verbose=verbose, target=libname)
File "/usr/local/lib/python3.5/dist-packages/cffi/api.py", line 723, in compile
compiler_verbose=verbose, debug=debug, **kwds)
File "/usr/local/lib/python3.5/dist-packages/cffi/recompiler.py", line 1526, in recompile
compiler_verbose, debug)
File "/usr/local/lib/python3.5/dist-packages/cffi/ffiplatform.py", line 22, in compile
outputfilename = _build(tmpdir, ext, compiler_verbose, debug)
File "/usr/local/lib/python3.5/dist-packages/cffi/ffiplatform.py", line 58, in _build
raise VerificationError('%s: %s' % (e.class.name, e))
cffi.VerificationError: LinkError: command 'x86_64-linux-gnu-gcc' failed with exit status 1
Compiling correlation kernels by nvcc...
rm: 无法删除'correlation_cuda_kernel.o': 没有那个文件或目录
./make.sh: 行 11: nvcc: 未找到命令
x86_64-linux-gnu-gcc: error: /home/throne/Deep-Flow-Guided-Video-Inpainting-master (2)/models/FlowNet2_Models/correlation_package/src/correlation_cuda_kernel.o: 没有那个文件或目录
Traceback (most recent call last):
File "/usr/lib/python3.5/distutils/unixccompiler.py", line 207, in link
self.spawn(linker + ld_args)
File "/usr/lib/python3.5/distutils/ccompiler.py", line 909, in spawn
spawn(cmd, dry_run=self.dry_run)
File "/usr/lib/python3.5/distutils/spawn.py", line 36, in spawn
_spawn_posix(cmd, search_path, dry_run=dry_run)
File "/usr/lib/python3.5/distutils/spawn.py", line 159, in _spawn_posix
% (cmd, exit_status))
distutils.errors.DistutilsExecError: command 'x86_64-linux-gnu-gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/cffi/ffiplatform.py", line 51, in _build
dist.run_command('build_ext')
File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/usr/lib/python3.5/distutils/command/build_ext.py", line 338, in run
self.build_extensions()
File "/usr/lib/python3.5/distutils/command/build_ext.py", line 447, in build_extensions
self._build_extensions_serial()
File "/usr/lib/python3.5/distutils/command/build_ext.py", line 472, in _build_extensions_serial
self.build_extension(ext)
File "/usr/lib/python3.5/distutils/command/build_ext.py", line 557, in build_extension
target_lang=language)
File "/usr/lib/python3.5/distutils/ccompiler.py", line 717, in link_shared_object
extra_preargs, extra_postargs, build_temp, target_lang)
File "/usr/lib/python3.5/distutils/unixccompiler.py", line 209, in link
raise LinkError(msg)
distutils.errors.LinkError: command 'x86_64-linux-gnu-gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "build.py", line 31, in
ffi.build()
File "/usr/local/lib/python3.5/dist-packages/torch/utils/ffi/init.py", line 184, in build
_build_extension(ffi, cffi_wrapper_name, target_dir, verbose)
File "/usr/local/lib/python3.5/dist-packages/torch/utils/ffi/init.py", line 108, in _build_extension
outfile = ffi.compile(tmpdir=tmpdir, verbose=verbose, target=libname)
File "/usr/local/lib/python3.5/dist-packages/cffi/api.py", line 723, in compile
compiler_verbose=verbose, debug=debug, **kwds)
File "/usr/local/lib/python3.5/dist-packages/cffi/recompiler.py", line 1526, in recompile
compiler_verbose, debug)
File "/usr/local/lib/python3.5/dist-packages/cffi/ffiplatform.py", line 22, in compile
outputfilename = _build(tmpdir, ext, compiler_verbose, debug)
File "/usr/local/lib/python3.5/dist-packages/cffi/ffiplatform.py", line 58, in _build
raise VerificationError('%s: %s' % (e.class.name, e))
cffi.VerificationError: LinkError: command 'x86_64-linux-gnu-gcc' failed with exit status 1
Compiling channelnorm kernels by nvcc...
rm: 无法删除'ChannelNorm_kernel.o': 没有那个文件或目录
./make.sh: 行 9: nvcc: 未找到命令
x86_64-linux-gnu-gcc: error: /home/throne/Deep-Flow-Guided-Video-Inpainting-master (2)/models/FlowNet2_Models/channelnorm_package/src/ChannelNorm_kernel.o: 没有那个文件或目录
Traceback (most recent call last):
File "/usr/lib/python3.5/distutils/unixccompiler.py", line 207, in link
self.spawn(linker + ld_args)
File "/usr/lib/python3.5/distutils/ccompiler.py", line 909, in spawn
spawn(cmd, dry_run=self.dry_run)
File "/usr/lib/python3.5/distutils/spawn.py", line 36, in spawn
_spawn_posix(cmd, search_path, dry_run=dry_run)
File "/usr/lib/python3.5/distutils/spawn.py", line 159, in _spawn_posix
% (cmd, exit_status))
distutils.errors.DistutilsExecError: command 'x86_64-linux-gnu-gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/cffi/ffiplatform.py", line 51, in _build
dist.run_command('build_ext')
File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/usr/lib/python3.5/distutils/command/build_ext.py", line 338, in run
self.build_extensions()
File "/usr/lib/python3.5/distutils/command/build_ext.py", line 447, in build_extensions
self._build_extensions_serial()
File "/usr/lib/python3.5/distutils/command/build_ext.py", line 472, in _build_extensions_serial
self.build_extension(ext)
File "/usr/lib/python3.5/distutils/command/build_ext.py", line 557, in build_extension
target_lang=language)
File "/usr/lib/python3.5/distutils/ccompiler.py", line 717, in link_shared_object
extra_preargs, extra_postargs, build_temp, target_lang)
File "/usr/lib/python3.5/distutils/unixccompiler.py", line 209, in link
raise LinkError(msg)
distutils.errors.LinkError: command 'x86_64-linux-gnu-gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "build.py", line 31, in
ffi.build()
File "/usr/local/lib/python3.5/dist-packages/torch/utils/ffi/init.py", line 184, in build
_build_extension(ffi, cffi_wrapper_name, target_dir, verbose)
File "/usr/local/lib/python3.5/dist-packages/torch/utils/ffi/init.py", line 108, in _build_extension
outfile = ffi.compile(tmpdir=tmpdir, verbose=verbose, target=libname)
File "/usr/local/lib/python3.5/dist-packages/cffi/api.py", line 723, in compile
compiler_verbose=verbose, debug=debug, **kwds)
File "/usr/local/lib/python3.5/dist-packages/cffi/recompiler.py", line 1526, in recompile
compiler_verbose, debug)
File "/usr/local/lib/python3.5/dist-packages/cffi/ffiplatform.py", line 22, in compile
outputfilename = _build(tmpdir, ext, compiler_verbose, debug)
File "/usr/local/lib/python3.5/dist-packages/cffi/ffiplatform.py", line 58, in _build
raise VerificationError('%s: %s' % (e.class.name, e))
cffi.VerificationError: LinkError: command 'x86_64-linux-gnu-gcc' failed with exit status 1

RuntimeError: invalid argument 2: out of range at /opt/conda/conda-bld/pytorch_1524584710464/work/aten/src/THC/generic/THCTensor.c:23

when I run CUDA_VISIBLE_DEVICES=2 python3 tools/train_initial.py --model_name stage1 --FIX_MASK --MASK_MODE bbox --TRAIN_LIST /home/ytwei/Inpainting/datalist.txt --DATA_ROOT /home/ytwei/Inpainting/Flow --INITIAL_HOLE --batch_size 8, I met this error.
Is there anybody who can help me ?

links of backbone and tools are 404

Could you please check these results?

Hi, I used the pre-trained model on DAVIS dataset and got the following results. It looks not so good. Is that the expected results?

memory requirement?

I was running the demo (video inpainting) and successfully extracted the flow. However, I encountered the following error while loading pre-trained resnet model. It seems that it is a memory problem. Any clue about the memory requirement and how to solve it? Thanks!

Generate datalist for initial step
Flow Completion in First Step
Freezing BatchNorm2D.
Load Pretrained Model from ./pretrained_models/resnet101_movie.pth

[ ] 0/159, elapsed: 0s, ETA:ERROR: Unexpected bus error encountered in worker. This might be caused by insufficient shared memory (shm).
Traceback (most recent call last):
File "tools/video_inpaint.py", line 183, in
main()
File "tools/video_inpaint.py", line 167, in main
flow_completion(args)
File "tools/video_inpaint.py", line 115, in flow_completion
test_initial_stage(args)
File "/app/tools/test_scripts.py", line 101, in test_initial_stage
for i, item in enumerate(eval_dataloader):
File "/opt/conda/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 286, in next
return self._process_next_batch(batch)
File "/opt/conda/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 307, in _process_next_batch
raise batch.exc_type(batch.exc_msg)
RuntimeError: Traceback (most recent call last):
File "/opt/conda/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 57, in _worker_loop
samples = collate_fn([dataset[i] for i in batch_indices])
File "/opt/conda/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 138, in default_collate
return [default_collate(samples) for samples in transposed]
File "/opt/conda/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 138, in
return [default_collate(samples) for samples in transposed]
File "/opt/conda/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 113, in default_collate
storage = batch[0].storage()._new_shared(numel)
File "/opt/conda/lib/python3.6/site-packages/torch/storage.py", line 116, in _new_shared
return cls._new_using_fd(size)
RuntimeError: unable to write to file </torch_91_481538039> at /pytorch/aten/src/TH/THAllocator.c:383

how long will it take to open your codes?

how long will it take to open your codes?
i'm fully waiting for your project codes
your project is damn amazing
i watched the demo video

anyway, can i get to know the day to open?

Link of pretrained model is 404

Thanks for your codes!
However, I found the link of the pretrained model was 404. Would you please provide a new link?

about frames and masks

make.sh can't run successfully

When I bash make.sh in Deep-Flow-Guided-Video-Inpainting-master\annotation_edition\models\FlowNet2_Models\resample2d_package ,I find there is not Resample2d_kernel.o in /src。

How to use README Is the command argument wrong?

Hi Readme's arguments and source code's arguments are wrong, --image_size 512 832 → --img_size 512 832, right?

requirements.txt

requirements.txt这个文件是不是忘记放了。。。

Warning: NaN or Inf found in input tensor.

when I train this model with batch size 1, the error occurred at 41st samples.

Flow inpainting along the motion boundary

First of all, thanks for the great work and making the code available.

I have a question regarding the flow inpainting stage. The provided DFC weight is mainly for inpainting the flow of the foreground object, which is relatively easier because there is no motion boundary issue. However, if the hole has an arbitrary shape, then we do need to pay attention to the motion boundary. And I believe this motivates the key contribution of this paper, which is the coarse-to-fine flow completion. Thus, I was wondering if you plan to release the weights for the all subnetworks, so that we can all benefit from it.

Also, could you please provide some additional guidance on training the DFC-NET? Again, thanks for your great work. Looking forward to hearing from you.

How to install Pytorch 0.4.0

How to install pytorch 0.4.0? Cause there are some errors when compile the code with pytorch version > 0.4.0

Looking forward to the release

how can I get the data list?

Could you please tell me how to get the data list?

1. generate by myself? if so, what does this list need to contain?
2. if I can use your code to generate this list, which file should I refer to?

Thanks in advance

Tensorflow?

@nbei @metalbubble is there a tensorflow version in works?

bug in program

when I use your program “ video_inpaint.py”
I want to use stage2 and stage3 ， and the program is :

  if args.MS:
      from tools.test_scripts import test_refine_stage
      args.PRETRAINED_MODEL = args.PRETRAINED_MODEL_2
      args.IMAGE_SHAPE = [320, 600]
      args.RES_SHAPE = [320, 600]
      args.DATA_ROOT = args.output_root
      args.output_root = os.path.join(args.dataset_root, 'Flow_res', 'stage2_res')`

      stage2_data_list = os.path.join(data_list_dir, 'stage2_test_list.txt')
      from dataset.data_list import gen_flow_refine_test_mask_list
      gen_flow_refine_test_mask_list(flow_root=args.DATA_ROOT,
                                   output_txt_path=stage2_data_list)
      args.EVAL_LIST = stage2_data_list      
      test_refine_stage(args)

      args.PRETRAINED_MODEL = args.PRETRAINED_MODEL_3
      args.IMAGE_SHAPE = [480, 840]
      args.RES_SHAPE = [480, 840]
      args.DATA_ROOT = args.output_root
      args.output_root = os.path.join(args.dataset_root, 'Flow_res', 'stage3_res')

      stage3_data_list = os.path.join(data_list_dir, 'stage3_test_list.txt')
      from dataset.data_list import gen_flow_refine_test_mask_list
      gen_flow_refine_test_mask_list(flow_root=args.DATA_ROOT,
                                   output_txt_path=stage3_data_list)
      args.EVAL_LIST = stage3_data_list
      test_refine_stage(args)
      args.flow_root =  args.output_root

And we know, I should activate the variable “MS”， so I use the command "python tools/video_inpaint.py --frame_dir ./demo/frames --MASK_ROOT ./demo/masks --img_size 512 832 --FlowNet2 --DFC --ResNet101 --Propagation --MS"

but I cannot run the program currently and get error below:

Traceback (most recent call last): File "tools/video_inpaint.py", line 196, in <module> main() File "tools/video_inpaint.py", line 177, in main flow_completion(args) File "tools/video_inpaint.py", line 140, in flow_completion test_refine_stage(args) File "/media/zhu/software/Deep-Flow-Guided-Video-Inpainting/tools/test_scripts.py", line 127, in test_refine_stage eval_dataset = FlowRefine.FlowSeq(args, isTest=True) File "/media/zhu/software/Deep-Flow-Guided-Video-Inpainting/dataset/FlowRefine.py", line 31, in __init__ if self.config.GT_FLOW_ROOT is not None: AttributeError: 'Namespace' object has no attribute 'GT_FLOW_ROOT'

how to fix the problem? thanks!

The zip file has been corrupted：“FlowNet2_checkpoint.pth.tar”

File corrupted. Can't use it?

how to visualizing the flow map

sorry to bother you, I wonder how to visualizing the generated flow map like your paper.

watermark removal mask in video

This is really an exciting project. Thank you for sharing. I want to delogo use it for video, watermark removal in your demo video. I want to ask how the watermark mask is generated. can you tell me? thank you