zordaan/openface

OpenFace++ – an extended variant of OpenFace. Includes facial expression detection, Android support and more.

OpenFace++

Introduction

OpenFace++ was created for my diploma thesis. It is based on the original OpenFace codebase from August 2016. OpenFace++ contains several modifications and extensions described below.

---

Comparison to OpenFace

OpenFace++ does NOT contain any major changes to the facial data extraction techniques already used in OpenFace. However, OpenFace++ includes the following changes compared to OpenFace.

• Facial Expression Detection (using Action Units)
• Attention Estimation (using Eye-Gaze)
• Android Support
• Background Worker Thread
• Simplified API
• Code Refactoring
• Static Linking and Optimized Building

For more detailed information please refer to the thesis document.

---

Downloads

Project	Platform	Distribution	Download
MiniExample	Windows	ZIP	Link
FaceSensor	Windows	ZIP	Link
OpenFaceAndroid	Android 4.4+	APK	Link

---

License

OpenFace++ was created for research purposes and does not have any additional license requirements. However, you must respect the original OpenFace license as well as the licenses of dependencies listed below.

---

Dependencies

The sources of following third party components have been added to this repository. These components are linked statically into the final application. You must respect their licenses. Unlike OpenFace, OpenFace++ does NOT require boost.

Name	Description	License	Website
OpenCV	Computer Vision Framework	Link	http://opencv.org
dlib	Machine Learning	Link	http://dlib.net
LibPNG	PNG-Fileformat	Link	http://www.libpng.org/pub/png/libpng.html
LibSVM	Support Vector Machines	Link	http://www.csie.ntu.edu.tw/~cjlin/libsvm/
zlib	ZIP-Compression	zlib	http://www.zlib.net
Intel TBB	Multi Threading Toolkit	Apache 2.0	https://www.threadingbuildingblocks.org/

---

Build Instructions

Visual Studio 2015 is required. Make sure to selected C++ and Android NDK related components at setup stage.

Engine

• Engine folder contains all dependencies and OpenFace++
• All libraries are compiled statically from sources, no precompiled binaries!
• Build the engine for Windows or Android using the Visual Studio 2015 solutions below.
• Recommendation: Build engine for all available configurations (e.g. x86/debug being only one!).
• This step must be done only once; or in case there are changes in the dependencies.
• Compiled libraries are placed in: _Engine/lib/[platform]_
• Debug builds include additional filename appendix: _\_d_

Windows

VS2015 Solution:	Engine.sln
Platforms:	x86, x64

Android

VS2015 Solution:	Engine-Android.sln
Platforms:	ARM, ARM64, x86, x64
Target:	android-19

Note: The Android solution includes the additional openface-jni project. This is the Java JNI wrapper used to access OpenFace from a managed Java APP on Android. This project generates the SO library file which can be loaded and called in an Android App using the native keyword in Java.

MiniExample

• Folder
• Minimal example as presented in my thesis.
• For Windows only; requires Webcam.
• Building requires successfully compiled engine for Windows!
• Includes the openface library project. You can change, rebuild and test from here!
• VS2015 Solution: MiniExample.sln

FaceSensor

• Folder
• Application used for the user study in my thesis
• For Windows only; requires Webcam.
• Building requires successfully compiled engine for Windows!
• Streams facial data data using OSC network protocol.
• Includes the openface library project. You can change, rebuild and test from here!
• VS2015 Solution: FaceSensor.sln

OpenFaceAndroid

• Folder
• Test project for Android Studio
• (!) Requires compatible SO library from _openface-jni_
• (!) Contains duplicated resources from _Engine/res_