langjunman/head-pose-estimation

Real-time head pose estimation built with OpenCV and dlib

head-pose-estimation

Real-time head pose estimation built with OpenCV and dlib

2D:
Using dlib for facial features tracking, modified from http://dlib.net/webcam_face_pose_ex.cpp.html
The algorithm behind it is described in http://www.csc.kth.se/~vahidk/papers/KazemiCVPR14.pdf
It applies cascaded regression trees to predict shape(feature locations) change in every frame.
Splitting nodes of trees are trained in random, greedy, maximizing variance reduction fashion.
The well trained model can be downloaded from http://dlib.net/files/shape_predictor_68_face_landmarks.dat.bz2
Training set is based on i-bug 300-W datasets. It's annotation is shown below:



3D:
To match with 2D image points(facial features) we need their corresponding 3D model points.
http://aifi.isr.uc.pt/Downloads/OpenGL/glAnthropometric3DModel.cpp provides a similar 3D facial feature model.
It's annotation is shown below:



Finally, with solvepnp function in OpenCV, we can achieve real-time head pose estimation.