panda-lab/face-landmark

caffe 68 points face-landmark

face-landmark-localization

This is a project predict face landmarks (68 points).

• Model Size : 3.3MB.
• Speed : 5ms per face on i5-2.7GHz CPU.

Install

• caffe
• dlib face detector
• add libdlib.a to your project lib path
• add face_lardmark.cpp to your caffe project example folder
• opencv

Usage

• CPP Demo : ./face_lardmark
• Python Demo: python face_landmark.py (created by jiangwqcooler)

notice

• This model was trained on dlib face detector. Other face detector may not get landmark result as good as dlib

Result

face-landmark's Issues

是否能提供一份训练数据？

你好，目前我使用公开的数据库，例如ibug，helen，300w数据库训练，loss值很大，不收敛。是否能提供一份训练数据到网盘下载，谢谢！

使用opencv加载caffe模型，预测关键点特别不准

cv::Mat inputBlob = cv::dnn::blobFromImage(roi, std_val, cv::Size(60, 60), cv::Scalar(127.5));
net.setInput(inputBlob, "data");
cv::Mat prob = net.forward("Dense3");
for (int i = 0; i < 136; i += 2) {
int x = int(prob.at(0, i)*roi.cols);
int y = int(prob.at(0, i + 1)*roi.rows);
cv::Point curr_pt = cv::Point(x,y);
cv::circle(roi, curr_pt, 2, cv::Scalar(0, 255, 0));
}

EliteBook 840 G3笔记本，检测和识别耗时700ms

比dlib demo还慢，求优化。

训练h5文件的对应顺序

你好，请问下，训练的h5文件对应的是该图片的
xmin ymin xmax ymax x1 y1 x2 y2 .....x68, 68吗？
多谢了，想自己学习训练下

关于转换成HD5的问题

博主你好，我在用mainloop，转换成HD5格式的文件的时候。出现了如下错误：

ivan@ivan-System-Product-Name:~/face-landmark-master/train$ python mainloop.py
../train_data/WFLW/test-1-2487.jpg
Traceback (most recent call last):
File "mainloop.py", line 166, in
dataRowsTrain_CSV = createDataRowsFromCSV(CSV_TRAIN, DataRow.DataRowFromNameBoxInterlaved, DATA_PATH)
File "/home/ivan/face-landmark-master/train/DataRow.py", line 45, in createDataRowsFromCSV
d = csvParseFunc(row, DATA_PATH)
File "/home/ivan/face-landmark-master/train/DataRow.py", line 874, in DataRowFromNameBoxInterlaved
d.p1 = (float(row[1]), float(row[2]))
ValueError: could not convert string to float:

请问是哪里出错了么？我是图片放在一个文件夹，然后对应标注的信息都放在landmrk68.txt文件夹里面。每一行的结构是：图片名+BBox+landmark 。请教下哪里出错了呢

怎么在Ubuntu编译这个工程呢。。。

谢谢。。。

请问数据集我看着是10个关键点，是怎么做到识别60多个的？

同上

Paper

Have you written a research paper about this code?

have a question

Hello, writer.
First of all, I saw your impressive code well.

I'm going to build your code on my computer.
So, I want to know the environment that you have implemented.

For example, Python 3.x, openv-python, numpy, dlib, caffe, etc.

As you know, there are sensitive libraries depending on the version.
I want to know their version.

Thank you !

face landmark 多个点相当于多标签？

大佬好，想在一些数据上重新训练特征点的模型。
以68点为例子，68点相当于 68*2=136个标签？
以人脸框为单位，将点坐标进行归一化，归一化到（-1,1）或者（0,1）之间，每个标签下（一个标签相当于一个点的x或者y坐标）的数值都是小数了，这样没问题吧？
是不是分类问题标签是 0 1 2等整数
回归问题标签可以是小数

看着数据集只有5个关键点，兄台是怎么做到训练识别60多个的

同上

链接无法下载AFW/ AFLW数据；Can't download the datasets

https://www.ics.uci.edu/~xzhu/face/AFW.zip
http://mmlab.ie.cuhk.edu.hk/archive/CNN/data/train.zip
这两个链接都无法下载数据，请问怎么解决的呢？

The links can not be used to download the datasets.
How to solve this question?

scaledLM = dataRow.landmarksScaledMinus05_plus05()请教原理

DataRow源码中的如下代码不太理解：

 dataRow = dataRowOrig.copyCroppedByBBox(dataRowOrig.fbbox)  #这里将图片已经resize为60*60了，而且landmark也已经进行了相应的缩放操作，debug这里输出图像和点是正确的

        scaledLM = dataRow.landmarksScaledMinus05_plus05()#不明白上面已经得到了正确的landmark，为什么还要将landmark进行处理，难道是和图片一样要做归一化操作？如果是，那么归一化方法应该和图片的对应吧，为什么要除以60？

#如下是图片的预处理和归一化操作
        image = dataRow.image.astype('f4')
        # image = (image-meanTrainSet)/(1.e-6 + stdTrainSet)
        image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY).reshape(1, IMAGE_SIZE, IMAGE_SIZE)
        m, s = cv2.meanStdDev(image)
        image = (image - m) / (1.e-6 + s)

        HD5Images[i, :] = image
        HD5Landmarks[i, :] = scaledLM

landmark的归一化操作请解释一下原理，谢谢~

model size

I found that your model size is only 3.5M compared with dilb's 98M shape_predictor_68_face_landmarks. Why is your model so small and what's the accuracy about your model?

如何优化视频中关键点抖动

您好，请问对连续视频帧输入产生的预测关键点抖动问题，有什么比较好解决方法吗

how to add python layer

使用ibug，helen，300W数据训练，loss很大，不收敛

我使用ibug，helen，300W数据训练，loss一直稳定在1.5左右，不收敛，请问如何解决，能否提供一份完整的训练代码或数据做参考， 谢谢！

你好，能给出模型相关方法的原论文吗？

如题

关于 train_prototxt

layer {
type: "HDF5Data"
top: "X" # same name as given in create_dataset!
top: "landmarks"
hdf5_data_param {
source: "/home/ly/workspace/Vanilla-40/caffeData/test.txt" # do not give the h5 files directly, but the list.

请教一下：里面的 top: "X" # same name as given in create_dataset!X指的是啥

Bug in DataRow

DataRow.py 1237行，在copyMirrored()为了获取y轴翻转图像的代码：

ret.image=cv2.flip(self.image.copy(),1)
        # Now we mirror the landmarks and swap left and right
        width = ret.image.shape[0] 

由于源码是对60*60图像的翻转，所以刚好碰对了
ret.image.shape[0] 是获取的图像的行数，指的是height不是width
所以应该修改width = ret.image.shape[1]才是正确的
请核实后修改代码，谢谢~

数据集

有个小小的请求，群主能把这些公开的数据集百度分享下么。。。我下了300W，为啥解压不了，说文件损坏了。。

Would you like to provide a more standard version?

Hello,

when test your network, it says that

I0918 15:13:54.308856 12776 net.cpp:137] Memory required for data: 998273536
I0918 15:13:54.308861 12776 layer_factory.hpp:77] Creating layer loss
[libprotobuf FATAL google/protobuf/stubs/common.cc:61] This program requires version 3.6.0 of the Protocol Buffer runtime library, but the installed version is 2.6.1. Please update your library. If you compiled the program yourself, make sure that your headers are from the same version of Protocol Buffers as your link-time library. (Version verification failed in "google/protobuf/descriptor.pb.cc".)
terminate called after throwing an instance of 'google::protobuf::FatalException'
what(): This program requires version 3.6.0 of the Protocol Buffer runtime library, but the installed version is 2.6.1. Please update your library. If you compiled the program yourself, make sure that your headers are from the same version of Protocol Buffers as your link-time library. (Version verification failed in "google/protobuf/descriptor.pb.cc".)
*** Aborted at 1537254835 (unix time) try "date -d @1537254835" if you are using GNU date ***
PC: @ 0x7f3f6b76c428 gsignal
*** SIGABRT (@0x3e8000031e8) received by PID 12776 (TID 0x7f3f6effe740) from PID 12776; stack trace: ***
@ 0x7f3f6b76c4b0 (unknown)
@ 0x7f3f6b76c428 gsignal
@ 0x7f3f6b76e02a abort
@ 0x7f3f6eb2184d __gnu_cxx::__verbose_terminate_handler()
@ 0x7f3f6eb1f6b6 (unknown)
@ 0x7f3f6eb1f701 std::terminate()
@ 0x7f3f6eb1f919 __cxa_throw
@ 0x7f3f6c742647 google::protobuf::internal::LogMessage::Finish()
@ 0x7f3f6c74287d google::protobuf::internal::VerifyVersion()
@ 0x7f3f148ee007 protobuf_google_2fprotobuf_2fdescriptor_2eproto::InitDefaultsUninterpretedOption_NamePart()
@ 0x7f3f149859d1 google::protobuf::internal::(anonymous namespace)::InitSCC_DFS()
@ 0x7f3f149859be google::protobuf::internal::(anonymous namespace)::InitSCC_DFS()
@ 0x7f3f149859be google::protobuf::internal::(anonymous namespace)::InitSCC_DFS()
@ 0x7f3f149859be google::protobuf::internal::(anonymous namespace)::InitSCC_DFS()
@ 0x7f3f149859be google::protobuf::internal::(anonymous namespace)::InitSCC_DFS()
@ 0x7f3f149859be google::protobuf::internal::(anonymous namespace)::InitSCC_DFS()
@ 0x7f3f149859be google::protobuf::internal::(anonymous namespace)::InitSCC_DFS()
@ 0x7f3f14985f50 google::protobuf::internal::InitSCCImpl()
@ 0x7f3f148e9b15 protobuf_google_2fprotobuf_2fdescriptor_2eproto::InitDefaults()
@ 0x7f3f148e9ca9 protobuf_google_2fprotobuf_2fdescriptor_2eproto::AddDescriptorsImpl()
@ 0x7f3f5ecd0a99 __pthread_once_slow
@ 0x7f3f14912f92 ZSt9call_onceIRFvvEJEEvRSt9once_flagOT_DpOT0
@ 0x7f3f6ee246ba (unknown)
@ 0x7f3f6ee247cb (unknown)
@ 0x7f3f6ee298e2 (unknown)
@ 0x7f3f6ee24564 (unknown)
@ 0x7f3f6ee28da9 (unknown)
@ 0x7f3f5eabef09 (unknown)
@ 0x7f3f6ee24564 (unknown)
@ 0x7f3f5eabf571 (unknown)
@ 0x7f3f5eabefa1 dlopen
@ 0x7f3f6bdf48dd _PyImport_GetDynLoadFunc
Aborted (core dumped)

it doesn't appear on others, just in face-landmark( run in same caffe, and I can make sure it had recompiled after use NormlizedMSE, and NMSE can be import both in python and this train process).
so would you like to provide a more standard version than compiling from multiple languages one? It enviroment is too hard to be fixed.

face landmark 训练收敛到face landmark的均值点，怎么解决？

where is the absval_layer?

I can't find it。Could you please show me?

I have found it in the original caffe, thanks.

How to train ?

Hi, @lsy17096535 . Thanks for sharing your code. Could you provide more infor about your training ?

Where to get the dataset?

1. In mainloop.py, trainSetHD5 creates the train data, but the var CSV_TRAIN is not set.
2. You mention the original model in #2 . The link also includes two datasets(MTFL and MAFL), but both only provide five facial landmarks. Do you use their alignment tool to obtains 68 facial landmarks as your offered model outputs 68 landmarks ?

Do you use the same solver prototxt as VanillaCNN?

How can i test your amazing work?

@lsy17096535 hello, i forked your project, but your installing tutorial is too rough， can you tell me the detail of installing face-landmark, thank you very much!

landmark68.txt文件

请问生成这个文件　里面的格式是：每一行　图片路径＋名字＋xmin ymin xmax ymax landmark..........　需要加图片的绝对路径么？

How to make training prediction on one's own data set

Hello, I have a face data with 194 face key points. I want to use your network to train my data so that I can predict 194 face key points, but there is no explanation in your explanation. Please tell me how should I start?

Is it right? gary_img = cv2.cvtColor(roi, cv2.COLOR_RGB2GRAY) in face_landmark.py line 65

gary_img = cv2.cvtColor(roi, cv2.COLOR_RGB2GRAY)
why not
gary_img = cv2.cvtColor(roi, cv2.COLOR_BGR2GRAY)

眼睛检测不准确

@lsy17096535 你好，想问一下这个模型在移动端眼睛检测不准确，特别是闭眼的时候。请问怎么处理，或者有误无更新的模型？

five point

Hi,how to produce 5 points of face landmark!
thank you!

ImportError: No module named python.NormlizedMSE

when I use command: “caffe train --solver=solver.prototxt ", abort and get this message, how can I fix it? thanks.

脚本执行流程

您好，最近在搞特征点定位，看到了您的工程，但是train文件夹中的脚本文件有些多，，，，对于新手的我，您方便分享一下这些脚本执行的顺序吗？或是脚本执行流程？例子用到的数据集是aflw或是afw吗？哪里能够下载，谢谢！

转ＨＤ５格式要花好久

转换成ＨＤ５格式　，才７０００张图片，花了好久。

'hsi' is not a member of 'dlib::pixel_traits<int>'

@lsy17096535 昨天的问题我解决掉了，今天在caffe里编译face-flandmark.cpp时候又遇到新的问题，尝试多种办法无果，感觉像是dlib的版本问题，我用的是git clone dlid的版本，还请帮忙看一下

/usr/local/include/dlib/pixel.h:1313:52: error: 'lab' is not a member of 'dlib::pixel_traits'
typename enable_if_c<pixel_traits::lab && pixel_traits::hsi>::type
^
/usr/local/include/dlib/pixel.h:1313:52: error: 'hsi' is not a member of 'dlib::pixel_traits'
make: *** [.build_release/examples/face_landmark.o] Error 1

关于如何生成训练数据

@lsy17096535 你好，想问一下，mainloop.py中提到了'trainMean.png'和'trainSTD.png'，这两个文件怎么获得？有什么作用？

Module face_landmark_localization在移动端比如RK3288上检测出68个关键点需要多长时间？

另外，有这个模块的C++代码吗？方便迁移到Android上。

训练数据

麻烦问下训练数据采用的是啥呢？能提供下caffe solver吗

请问训练方法对应的是那一篇论文

RT

训练时用的 solver.prototxt 能参考一下吗？

想了解一下训练时的一些参数。对于刚入门 caffe，参数拿捏不准，而且训练又太耗时了，想参考一下。顺便问一下，训练一次花费多长时间啊？

关于mainloop生成HD5训练数据

我自己准备好了图像，以及对应的TXT文件后，是否只要运行mainloop里面的trainSetHD5就可以了，其他要跑么？

人脸关键点顺序

您好！打扰您了.....请问下您这人脸关键点的顺序跟dlib标注的人脸关键点的顺序是一样的吗？提供的代码后面的loss需要更改吗？还是直接训练就可以了，我这边训练loss一直在1左右，无法再收敛，得到的模型效果不好......有时间可以帮我解答疑惑吗? 非常感谢！

怎么用dlib才能训练出3M的模型呢？

使用dlib训练了，最小也到了15M，请问是怎么训练出3M模型的呢？

train model的caffe solver

hello，麻烦问下能提供下train model的caffe solver吗？非常感谢。