zokooo/unsupflownet

Unsupervised Flownet

Back to Basics: Unsupervised Learning of Optical Flow via Brightness Constancy and Motion Smoothness

This repository contains a TensorFlow implementation from the original work: Back to Basics: Unsupervised Learning of Optical Flow via Brightness Constancy and Motion Smoothness (ECCV 2016 Workshops, Part 3).

Authors: Jason J. Yu, Adam W. Harley, Konstantinos G. Derpanis

Project Page

Slides

Citation

@inproceedings{
    jjyu2016unsupflow,
    author = {Jason J. Yu and Adam W. Harley and Konstantinos G. Derpanis},
    title = {Back to Basics: Unsupervised Learning of Optical Flow via Brightness Constancy and Motion Smoothness},
    booktitle = {Computer Vision - ECCV 2016 Workshops, Part 3},
    year = {2016}
}

Description

The code provided has been improved beyond what was described in the original paper. The original work focused on only a loss based on photometric constancy and motion smoothness. This can be achieved by disabling additional loss terms in the hyper-parameter file. This code has only been tested on python 2.7 and TensorFlow 1.8.0.

Benchmarks

Coming soon!

Usage

Hardware

It is recommended to train using an NVIDIA GPU with minimum 8GB of VRAM.

Software and Libraries

• Python 2.7
• TensorFlow 1.8.0 (And its requirements)
• Pillow
• NumPy
• SciPy

Scripts and Modules

train.py: Script to train network
test.py: Script for tested trained network, paths for datasets should be modified
util/TrainingData.py: handles reading, queuing, and processing of data, paths for datasets should be modified here
components: Helpers to build graph for network and losses
util: Larger components such as the network, losses, etc.
hyperparams.json: Contains all hyper-parameters, flags for different loss terms, and settings for training

Example data lists

example_data: an example of how a dataset should be structured
example_data/data: a single example image pair and its corresponding ground truth flow
example_data/datalists: example datalists for the data
example_data/datalists2 example datalists used for training with KITTI

Pre-trained Models

model_download_scripts contains scripts for downloading pre-trained models along with their corresponding hyper-parameters. Currently there is only one model available trained with photometric and smoothness loss terms. Run these scripts from their respective directories e.g. photometric_smoothness.