adamjm/sepconv

An implementation of Adaptive Separable Convolution for Video Frame Interpolation

Implementing Adaptive Separable Convolution for Video Frame Interpolation

This is a fully functional implementation of the work of Niklaus et al. [1] on Adaptive Separable Convolution, which claims high quality results on the video frame interpolation task. We apply the same network structure trained on a smaller dataset and experiment with various different loss functions, in order to determine the optimal approach in data-scarce scenarios.

For detailed information, please see our report on arXiv:1809.07759.

The video below is an example of the capabilities of this implementation. Our pretrained model (used in this instance) can be downloaded from here.

---

Installation

Note that the following instructions apply in the case of a fresh Ubuntu 17 machine with a CUDA-enabled GPU. In other scenarios (ex. if you want to work in a virtual environment or prefer to use the CPU), you may need to skip or change some of the commands.

Install pip3:

sudo apt-get install python3-setuptools
sudo easy_install3 pip

Install GCC & friends:

sudo apt-get install gcc libdpkg-perl python3-dev

Install FFmpeg:

sudo apt-get install ffmpeg

Install CUDA:

curl -O https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/cuda-repo-ubuntu1604_8.0.61-1_amd64.deb
sudo dpkg -i ./cuda-repo-ubuntu1604_8.0.61-1_amd64.deb
sudo apt-get update
sudo apt-get install cuda-8-0 -y

Install NVCC:

sudo apt-get install nvidia-cuda-toolkit

Install dependencies:

sudo pip3 install -r ./path/to/requirements.txt

---

How to setup the project and train the network:

Move to the project directory:

cd ./sepconv

Create a new configuration file:

echo -e "from src.default_config import *\r\n\r\n# ...custom constants here" > ./src/config.py

To train the network, run main.py as a module:

python3 -m src.main

---

License

This project is released under the MIT license. See LICENSE for more information.

Third-party Libraries

The following dependencies are bundled with this project, but are under terms of a separate license:

• pytorch-sepconv by sniklaus
• CUDA Gradient for sepconv by ekgibbons

References

[1] Video Frame Interpolation via Adaptive Separable Convolution, Niklaus 2017, arXiv:1708.01692