lukas-blecher/deep-flow-guided-video-inpainting

pytorch implementation for "Deep Flow-Guided Video Inpainting"(CVPR'19)

deep-flow-guided-video-inpainting's Issues

unexpected keyword argument 'align_corners'

pretrain_model download from Weights for LiteFlowNet are hosted by sniklaus: default, kitti, sintel.
like network-default.pytorch is different from './pretrained_models/default.pth'

The following is the error message:

====> Loading /home/xieyl/Deep-Flow-Guided-Video-Inpainting/pretrained_models/network-default.pytorch
0%| | 0/158 [00:00<?, ?it/s]
Traceback (most recent call last):
File "tools/infer_liteflownet.py", line 150, in
main()
File "tools/infer_liteflownet.py", line 146, in main
infer(args)
File "tools/infer_liteflownet.py", line 81, in infer
flow = estimate(Flownet, f1, f2)
File "tools/infer_liteflownet.py", line 51, in estimate
tensorFlow = torch.nn.functional.interpolate(input=model(tensorPreprocessedFirst, tensorPreprocessedSecond), size=(intHeight, intWidth), mode='bilinear', align_corners=False)
File "/home/xieyl/anaconda3/envs/flow/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/home/xieyl/Deep-Flow-Guided-Video-Inpainting/models/LiteFlowNet_Models/LiteFlowNet.py", line 296, in forward
tensorFlow = self.moduleSubpixel[intLevel](tensorFirst[intLevel], tensorSecond[intLevel], tensorFeaturesFirst[intLevel], tensorFeaturesSecond[intLevel], tensorFlow)
File "/home/xieyl/anaconda3/envs/flow/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/home/xieyl/Deep-Flow-Guided-Video-Inpainting/models/LiteFlowNet_Models/LiteFlowNet.py", line 190, in forward
tensorFeaturesSecond = Backward(tensorInput=tensorFeaturesSecond, tensorFlow=tensorFlow * self.dblBackward)
File "/home/xieyl/Deep-Flow-Guided-Video-Inpainting/models/LiteFlowNet_Models/LiteFlowNet.py", line 25, in Backward
return torch.nn.functional.grid_sample(input=tensorInput, grid=(Backward_tensorGrid[str(tensorFlow.size())] + tensorFlow).permute(0, 2, 3, 1), mode='bilinear', padding_mode='zeros', align_corners=True)
TypeError: grid_sample() got an unexpected keyword argument 'align_corners'

train DeepFill v1 model

I want to train my DeepFill v1 model with my dataset.Can you tell me how to do it

No such file or directory: './pretrained_models/sintel.pth'

Thanks for your efforts to make this project run under windows, could you please provide the sintel.pth file

pretrained_model_liteflownet = (sintel.pth)

Hello, thanks for sharing this implementation.
Can you also share your pretrained model of liteflownet? My email address is [email protected]. Thank you!

No pretrained liteflow model

When running python tools/video_inpaint.py --frame_dir ./demo/frames --MASK_ROOT ./demo/masks --img_size 512 832 --LiteFlowNet --DFC --ResNet101 --Propagation,it showed:
FileNotFoundError: [Errno 2] No such file or directory: './pretrained_models/sintel.pth'

It seems that i dont have the sintel.pth file.Could you help me?thx.

FileNotFoundError: [Errno 2] No such file or directory: './pretrained_models/sintel.pth'

Where can i find the weights?

issues when get the result

thanks for cv2 detail,I had solve it but after that i get a new problem.

here are error details:
(lpz) mobax@nvidia-quadro-vws-ubuntu-18-1-vm:~/lpz/Deep-Flow-Guided-Video-Inpainting-master$ python tools/video_inpaint.py --frame_dir ./demo/frames --MASK_ROOT ./demo/masks --img_size 512 832 --LiteFlowNet --DFC --ResNet101 --Propagation
This Video has 80 Images
The optical flow list has been generated: ./demo/video_flow.txt
====> Loading ./pretrained_models/sintel.pth
Traceback (most recent call last):
File "tools/video_inpaint.py", line 185, in
main()
File "tools/video_inpaint.py", line 166, in main
extract_flow(args)
File "tools/video_inpaint.py", line 85, in extract_flow
output_file = infer(args)
File "/home/mobax/lpz/Deep-Flow-Guided-Video-Inpainting-master/tools/infer_liteflownet.py", line 69, in infer
Flownet = LiteFlowNet(args.pretrained_model_liteflownet)
File "/home/mobax/lpz/Deep-Flow-Guided-Video-Inpainting-master/models/LiteFlowNet_Models/LiteFlowNet.py", line 269, in init
self.load_state_dict(torch.load(model_path))
File "/home/mobax/anaconda3/envs/lpz/lib/python3.6/site-packages/torch/serialization.py", line 419, in load
f = open(f, 'rb')
FileNotFoundError: [Errno 2] No such file or directory: './pretrained_models/sintel.pth'

Something wrong with results of video inpainting demo.

I used the following command to test.
python tools/video_inpaint.py --frame_dir ./demo/frames --MASK_ROOT ./demo/masks --img_size 512 832 --LiteFlowNet --DFC --ResNet101 --Propagation

Results turn out to be really bad. Even running deepfill1 independently provides better results than video inpainting.

cv2.error ： ssize.empty() in function 'resize'

I have a problem when using my own data. I put my frames in /demo/sur/frames and masks in /demo/sur/masks

the masks caused the error?

The following is the error message:

This Video has 30 Images
The optical flow list has been generated: ./demo/sur/video_flow.txt
====> Loading /home/xieyl/Deep-Flow-Guided-Video-Inpainting/pretrained_models/network-default.pytorch
100%|███████████████████████████████████████████████████████████████████████████████████████| 58/58 [00:07<00:00, 7.93it/s]

LiteFlowNet Inference has been finished~!
Extracted Flow has been save in ./demo/sur/Flow
Generate datalist for initial step
Flow Completion in First Step
Freezing BatchNorm2D.
Load Pretrained Model from ./pretrained_models/resnet101_movie.pth
0%| | 0/59 [00:00<?, ?it/s]Traceback (most recent call last):
File "tools/video_inpaint.py", line 185, in
main()
File "tools/video_inpaint.py", line 169, in main
flow_completion(args)
File "tools/video_inpaint.py", line 116, in flow_completion
test_initial_stage(args)
File "/home/xieyl/Deep-Flow-Guided-Video-Inpainting/tools/test_scripts.py", line 101, in test_initial_stage
for i, item in tqdm(enumerate(eval_dataloader), total=len(eval_dataset)):
File "/home/xieyl/anaconda3/envs/flow/lib/python3.7/site-packages/tqdm/std.py", line 1081, in iter
for obj in iterable:
File "/home/xieyl/anaconda3/envs/flow/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 582, in next
return self._process_next_batch(batch)
File "/home/xieyl/anaconda3/envs/flow/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 608, in _process_next_batch
raise batch.exc_type(batch.exc_msg)
cv2.error: Traceback (most recent call last):
File "/home/xieyl/anaconda3/envs/flow/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 99, in _worker_loop
samples = collate_fn([dataset[i] for i in batch_indices])
File "/home/xieyl/anaconda3/envs/flow/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 99, in
samples = collate_fn([dataset[i] for i in batch_indices])
File "/home/xieyl/Deep-Flow-Guided-Video-Inpainting/dataset/FlowInitial.py", line 78, in getitem
tmp_mask = self._mask_tf(tmp_mask)
File "/home/xieyl/Deep-Flow-Guided-Video-Inpainting/dataset/FlowInitial.py", line 130, in _mask_tf
interpolation=cv2.INTER_NEAREST)
cv2.error: OpenCV(4.1.1) /io/opencv/modules/imgproc/src/resize.cpp:3720: error: (-215:Assertion failed) !ssize.empty() in function 'resize'

Lack of './pretrained_models/sintel.pth'

Hi, all
When I tried to execute example, I encountered with this issues.
Could you please share this file with me?

Package Version

Hi,
When I tried to execute example, I encountered with this issues.
I think it's a problem with the torch and sensorboardX version. Do you have the same problem?