marekkowalski/deepalignmentnetwork

A deep neural network for face alignment

deepalignmentnetwork's Issues

Face Normalization

I can not see any explanation about Face Normalization. It this project can normalize the faces? Could you provide some details please?

I can't training on GPU

Hi:
I'm trying to use the DANtraining.py to complete the training process about the code,but i find the training is so slow,and it seems that it did not use GPU for training.Why is this happening?

Regarding licence of model file and dataset.

hi Marek,
I have gone through the repository and found that the license type used in this repository is "MIT".
1.Is the model file also covered under MIT license?
2.With your knowledge, any idea about the licensing of datasets you used?

The BoundingBox Problem with private test set

i use the TestSetPreparation.py scripe and save the image to disk, to find many pictures is all dark, and some picture has only half face.
and i check the boxes300WIndoor.pkl file and print it out, find some of the bounding box may has wrong order. as ibug says, the array is saved as [xmin, ymin, xmax, ymax], but in boxes300WIndoor.pkl, the dark image correspond to the box is like:
'indoor_153.png': array([1040.282, 725.013, 1228.23 , 925.087]),
'indoor_150.png': array([ 863.458, 293.431, 1058.745, 467.606]),
but the size of 150png is only 437*454

how can i implements landmark image layer in caffe

the title is my question ，thank you for your reply.

Need to first face detection?

Hi MarekKowalski,
Sorry for this question, but I am a newbie in this field, I wonder what if need to firstly face detection?
Thanks!

some questions about these pre-trainedmodels

Hi,
I have download these three models: DAN.npz, DAN-Menpo.npz, DAN-menpo-tracking.npz, but I don't know the difference between these models.
And I have download menpo training dataset, but it needs password to unzip, can you share the password or tell me how to get this?

about CropResizeRotate

hi,MarekKowalski ,In the CropResizeRotate，I don't understand the role of destShape.Why is this step to calculate the transformation from initshape to destshape? In the network,the initLandmarks(S0) is the destshape?thanks

how self.offsets + intLandmark possible?

I wonder how locations = self.offsets + intLandmark in LandmarkImageLayer.py is possible as I think self.offset = np.array(list(itertools.product(range(-self.half_size, self.half_size + 1), range(-self.half_size, self.half_size + 1)))), so it has size of (289,2), but intLandmark's size must be either (136,1) or (1,136).

I don't know how to debug theano to see the intermediate TensorValues, so I'm trying to convert it with only numpy. and I got the size mismatch error.

but ,Interestingly, I have no problem in running the theano code!

I'm totally puzzled!

Please let me know any hint.

Thank you in advance!

Can you share me the evalution code?

Hi,
I am a novice in face alignment, i don't very clear about 300W's "inter-pupil normalization" and "inter-ocular normalization". Can you share me the evalution code? Email-address: [email protected]
Thank you very much.

some questions about DAN_Model

hi :

1. i train the first stage only and then the second stage only；but when i trained the first stage, the validation set was chosen at random from train data sets, epoch=200, lr=0.001, is this process suitable?

looking forward to your reply, thanks.

How to train a model of 5 landmarks

Hi ,

Thanks for your codes.

Now I want to train model of 5 landmarks, then how can I do it?
Could you please give me some suggestion?

Thank you very much!

how to train the next stages?

I apply this method to our own dataset and work well. thanks for your work and code.
but only trained one stage,i wanna train more stages, could u give me some helps.
my method is that:
first of all, training = FaceAlignmenTraing(1,[0]).
then ,traning = FaceAlignmentTraing(2,[1]), and traing.loadNetwork('../my.npz')

it shows error that parameters dont match.

TransformParamsLayer class

Hi @MarekKowalski
I'm reading your code. I can't understand the code in class TransformParamsLayer.
Can you tell me which algorithm used in this class. Thanks!

can you provide inference script?

hello, interested in your code. But can you provide a inference script for testing an image?
thank you very much~~

some questions about TransformParamsLayer.py

Hi Marek, I see your code, and I kown TransformParamsLayer.py is used to get transform parameters from S1 to S0, but I can't understand why you subtract mean value here, and what's the meaning of a and b?

def bestFit(self, transformed_shape):
destination = self.mean_shape
source = transformed_shape.reshape((-1, 2))

  destMean = T.mean(destination, axis=0)
  srcMean = T.mean(source, axis=0)

  srcVec = (source - srcMean).flatten()
  destVec = (destination - destMean).flatten()
  a = T.dot(srcVec, destVec) / T.nlinalg.norm(srcVec, 2)**2
  b = 0
  for i in range(self.mean_shape.shape[0]):
      b += srcVec[2*i] * destVec[2*i+1] - srcVec[2*i+1] * destVec[2*i] 
  b = b / T.nlinalg.norm(srcVec, 2)**2  
  
  A = T.zeros((2, 2))
  A = T.set_subtensor(A[0, 0], a)
  A = T.set_subtensor(A[0, 1], b)
  A = T.set_subtensor(A[1, 0], -b)
  A = T.set_subtensor(A[1, 1], a)
  srcMean = T.dot(srcMean, A)  
  
  return T.concatenate((A.flatten(), destMean - srcMean))

how compute the meanshape s0 ?

hi,thanks for your work.
I am confused about s0. how compute the meanshape s0 ?can you share the code about it.Dose it related to the face box?
hope your reply

question about camera.py

Dear Marek:
I try to utilize different pre-trained model to test performance on camera. However, I found only when useTrackingModel = True and load DAN-Menpo-tracking.npz model could work, if I set the useTrackingModel = False and load DAN.npz or the model that I trained via myself, It would generate the following err:
File "/xxx/xxxx/DeepAlignmentNetwork-mast_back/DeepAlignmentNetwork /LandmarkImageLayer.py", line 15, in init
self.offsets = np.array(list(itertools.product(range(-self.half_size, self.half_size + 1), range(-self.half_size, self.half_size + 1))))
TypeError: 'float' object cannot be interpreted as an integer
Do you have any ideas about this err?
Thanks

Too difficult to reimplement your project !

Hi! I am very interesting in your project! But it seems that there are some difference between the Traindataset process script and the actual script in your paper. And can you provide me the training detail, such as the epoch for stage1 and stage2, and the learning rate. The network you provide on the Internet also cant reach the accuracy in your paper! Looking forward to you kind rely. Thank you!

some questions about GeneratePerturbations function

HI ! Thanks for your codes! But i am confused in the GeneratePerturbation function. I am poor in math , can you provide me some information or paper to let me know how the perturbation generate？ I have seen the other issue before, but the link is broken, can you provide it again! Thank you very much!

how to train model via the codes?

@MarekKowalski Thanks for sharing the codes. how to train model via the codes?

about DANtesting.py problem

Hi,When I run CameraDemo.py,there has a problem that
F:\code\DeepAlignmentNetwork-master\DeepAlignmentNetwork> python .\DANtesting.py
Loading network...
Input shape: (None, 1, 112, 112)
Network being tested: ..\DAN.npz
Normalization is set to: corners
Failure threshold is set to: 0.08
Processing common subset of the 300W public test set (test sets of LFPW and HELEN)
E:\anaconda\lib\site-packages\numpy\core\fromnumeric.py:2957: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
Average error: nan
Processing challenging subset of the 300W public test set (IBUG dataset)
Average error: nan
Showing results for the entire 300W pulic test set (IBUG dataset, test sets of LFPW and HELEN
Average error: nan
Traceback (most recent call last):
File ".\DANtesting.py", line 35, in
tests.AUCError(fullsetErrs, failureThreshold, showCurve=showCED)
File "F:\code\DeepAlignmentNetwork-master\DeepAlignmentNetwork\tests.py", line 52, in AUCError
ced = [float(np.count_nonzero([errors <= x])) / nErrors for x in xAxis]
File "F:\code\DeepAlignmentNetwork-master\DeepAlignmentNetwork\tests.py", line 52, in
ced = [float(np.count_nonzero([errors <= x])) / nErrors for x in xAxis]
ZeroDivisionError: float division by zero
What is the reason? Can you help me ?
Thank you !

about dataset and training

hi,MarekKowalski ，thank you for your code about deep alignment network.but i have some trouble about it ,i hope you could help me.
1、When I download the 300-W dataset based on the URL you gave and unzip the part2 of 300-W dataset,an error occurred .I don't know what's going on, I can't open the file, and I tried it many times without success.so ,Can you send Part2 data of 300-W to me, or can I use only other data sets( LFPW, HELEN, AFW and IBUG)without 300-W data set?what should I do
2、How to train the first and second stage？ Can you tell me the training steps?
I am looking forward to your answer, thank you.

question about AffineTransformLayer.py

hi, in my view, the AffineTransformLayer.py is used to apply the affine/similiarity transform on original training image, but why the affine transform matrix be inversed in lines 59-60?

How can input the picture in CameraDemo.py?

Hi,
I want to change the input to the picture path, but it always reminds me that " 'numpy.ndarray' has no attribute of 'read' ", which “vis = vidIn.read()[1]” in CameraDemo.py, I don't understand the meaning of "[1]" after "read". If I delete the "
vis = vidIn.read()[1]
if len(vis.shape) > 2:
img = np.mean(vis, axis=2).astype(np.uint8)
else: "
it reminds me that “ RuntimeError: affine matrix has wrong number of rows ”.
Thanks.

Question about bestFit function

Dear Marek:
Sorry to bother you, I could not understand the process in bestFit function. I read the paper and found it aims to align the input landmark to canonical shape. However, I could not know the math form utilized behind that. If it is possible, could you give a favor to let me know that?
Thanks for your help.

Separable Depthwise Layer?

Hi, heavy questioner is back.

today, I tried to apply popular separable dw conv layer introduced in theano 0.10.0(but not in lasagne)
to DAN, but as I'm a newbie in theano, I got lost....

below is my custom layer code for "SeparableDepthWiseConvolutionLayer"

[ SeparableDepthWiseConvolutionLayer.py ]

import theano
from lasagne.layers import Layer
import lasagne.init
from theano.tensor.nnet.abstract_conv import separable_conv2d
import numpy
rng = numpy.random

class SeparableDepthWiseConvolutionLayer(Layer):
    def __init__(self, input, input_shape, num_dw_channels=32, num_pw_filters=32, filter_size=[3, 3], stride=[1, 1], **kwargs):
        self.input_shape=input_shape
        self.num_dw_channels = num_dw_channels
        self.num_pw_filters  = num_pw_filters
        self.filter_size=filter_size
        self.stride=stride
        # dimension : self.num_dw_channels, 1, self.filter_size[0], self.filter_size[1]
        self.depthwise_filters= numpy.asarray(rng.randn(self.num_dw_channels * self.filter_size[0] * self.filter_size[1])).astype(theano.config.floatX)
        # dimension : (self.num_pw_filters, self.num_dw_channels, 1, 1)
        self.pointwise_filters= numpy.asarray(rng.randn(self.num_pw_filters * self.num_dw_channels)).astype(theano.config.floatX)
        super(SeparableDepthWiseConvolutionLayer, self).__init__(input, **kwargs)

    def get_output_shape_for(self, input_shape):
        return (1, input_shape[1]*int(self.stride[0]), input_shape[2]//self.stride[0],input_shape[3]//self.stride[0])

    def get_output_for(self, input, **kwargs):
        x_sym = theano.tensor.tensor4('x')
        dfilter_sym = theano.tensor.tensor4('d')
        pfilter_sym = theano.tensor.tensor4('p')
        sep_op = separable_conv2d(x_sym ,
                                  dfilter_sym ,
                                  pfilter_sym ,
                                  self.num_dw_channels,
                                  input_shape=self.input_shape,
                                  depthwise_filter_shape=(self.num_dw_channels,1,self.filter_size[0],self.filter_size[1]),
                                  pointwise_filter_shape=(self.num_pw_filters, self.num_dw_channels, 1, 1)
                                  )
        fun = theano.function([x_sym, dfilter_sym, pfilter_sym], sep_op, mode='FAST_RUN')
        output = fun(input ,self.depthwise_filters, self.pointwise_filters)
        return output

---

and then, I replaced all existing lasagnes' conv+batchnorm layers in createCNN() with the new layer as below;

[ DANtraining.py ]

from SepDWConvLayer import SeparableDepthWiseConvolutionLayer as SPDWConv
...
...
def createCNN(self):
...
...

net['s1_conv1_1'] = batch_norm(Conv2DLayer(net['input'], 64, 3, pad='same', W=GlorotUniform('relu')))
->> net['s1_conv1_1'] = batch_norm(SPDWConv(net['s1_conv0_1'], net['s1_conv0_1'].output_shape, 32, 64,[3, 3], stride=[1, 1] ))
net['s1_conv1_2'] = batch_norm(Conv2DLayer(net['s1_conv1_1'], 64, 3, pad='same', W=GlorotUniform('relu')))
->> net['s1_conv1_2'] = batch_norm(SPDWConv(net['s1_conv1_1'], net['s1_conv1_1'].output_shape, 64, 128,[3, 3], stride=[2, 2] ))
..
..
..
so on.

---

so the createCNN() looks like :

def createCNN(self):
    net = {}
    net['input'] = lasagne.layers.InputLayer(shape=(1, self.nChannels, self.imageHeight, self.imageWidth), input_var=self.data)
    print("Input shape: {0}".format(net['input'].output_shape))

    #STAGE 1
    net['s1_conv0_1'] = batch_norm(Conv2DLayer(net['input'], 32, 3, stride=(1,1), pad='same', W=GlorotUniform('relu')))
    print("s1_conv0_1 shape: {0}".format(net['s1_conv0_1'].output_shape))

    net['s1_conv1_1'] = batch_norm(SPDWConv(net['s1_conv0_1'], net['s1_conv0_1'].output_shape, 32,  64,[3, 3], stride=[1, 1] ))
    print("s1_conv1_1 shape: {0}".format(net['s1_conv1_1'].output_shape))
    net['s1_conv1_2'] = batch_norm(SPDWConv(net['s1_conv1_1'], net['s1_conv1_1'].output_shape, 64, 128,[3, 3], stride=[2, 2] ))
    print("s1_conv1_2 shape: {0}".format(net['s1_conv1_2'].output_shape))

    net['s1_conv2_1'] = batch_norm(SPDWConv(net['s1_conv1_2'], net['s1_conv1_2'].output_shape,128, 128,[3, 3], stride=[1, 1] ))
    print("s1_conv2_1 shape: {0}".format(net['s1_conv2_1'].output_shape))
    net['s1_conv2_2'] = batch_norm(SPDWConv(net['s1_conv2_1'], net['s1_conv2_1'].output_shape,128, 256,[3, 3], stride=[2, 2] ))
    print("s1_conv2_2 shape: {0}".format(net['s1_conv2_2'].output_shape))

    net['s1_conv3_1'] = batch_norm(SPDWConv(net['s1_conv2_2'], net['s1_conv2_2'].output_shape,256, 256,[3, 3], stride=[1, 1] ))
    print("s1_conv3_1 shape: {0}".format(net['s1_conv3_1'].output_shape))
    net['s1_conv3_2'] = batch_norm(SPDWConv(net['s1_conv3_1'], net['s1_conv3_1'].output_shape,256, 512,[3, 3], stride=[2, 2] ))
    print("s1_conv3_2 shape: {0}".format(net['s1_conv3_2'].output_shape))

    net['s1_conv4_1'] = batch_norm(SPDWConv(net['s1_conv3_2'], net['s1_conv3_2'].output_shape,512, 512,[3, 3], stride=[1, 1] ))
    print("s1_conv4_1 shape: {0}".format(net['s1_conv4_1'].output_shape))
    net['s1_conv4_2'] = batch_norm(SPDWConv(net['s1_conv4_1'], net['s1_conv4_1'].output_shape,512, 512,[3, 3], stride=[1, 1] ))
    print("s1_conv4_2 shape: {0}".format(net['s1_conv4_2'].output_shape))
    net['s1_conv4_3'] = batch_norm(SPDWConv(net['s1_conv4_2'], net['s1_conv4_2'].output_shape,512, 512,[3, 3], stride=[1, 1] ))
    print("s1_conv4_3 shape: {0}".format(net['s1_conv4_3'].output_shape))
    net['s1_conv4_4'] = batch_norm(SPDWConv(net['s1_conv4_3'], net['s1_conv4_3'].output_shape,512, 512,[3, 3], stride=[1, 1] ))
    print("s1_conv4_4 shape: {0}".format(net['s1_conv4_4'].output_shape))
    net['s1_conv4_5'] = batch_norm(SPDWConv(net['s1_conv4_4'], net['s1_conv4_4'].output_shape,512, 512,[3, 3], stride=[1, 1] ))
    print("s1_conv4_5 shape: {0}".format(net['s1_conv4_5'].output_shape))

    net['s1_conv5_1'] = batch_norm(SPDWConv(net['s1_conv4_5'], net['s1_conv4_5'].output_shape,512, 1024,[3, 3], stride=[2, 2]))
    print("s1_conv5_1 shape: {0}".format(net['s1_conv5_1'].output_shape))
    net['s1_conv5_2'] = batch_norm(SPDWConv(net['s1_conv5_1'], net['s1_conv5_1'].output_shape,1024,1024,[3, 3], stride=[1, 1]))
    print("s1_conv5_2 shape: {0}".format(net['s1_conv5_2'].output_shape))

    net['s1_pool6'] = lasagne.layers.Pool2DLayer(net['s1_conv5_2'], 1)
    print("s1_pool6 shape: {0}".format(net['s1_pool6'].output_shape))

    net['s1_fc1_dropout'] = lasagne.layers.DropoutLayer(net['s1_pool6'], p=0.5)
    print("s1_fc1_dropout shape: {0}".format(net['s1_fc1_dropout'].output_shape))
    net['s1_fc1'] = batch_norm(lasagne.layers.DenseLayer(net['s1_fc1_dropout'], num_units=256, W=GlorotUniform('relu')))
    print("s1_fc1 shape: {0}".format(net['s1_fc1'].output_shape))
    net['s1_output'] = lasagne.layers.DenseLayer(net['s1_fc1'], num_units=136, nonlinearity=None)
    print("s1_output shape: {0}".format(net['s1_output'].output_shape))
    net['s1_landmarks'] = LandmarkInitLayer(net['s1_output'], self.initLandmarks)
    print("s1_landmarks shape: {0}".format(net['s1_landmarks'].output_shape)) 

    for i in range(1, self.nStages):
        self.addDANStage(i + 1, net)

    net['output'] = net['s' + str(self.nStages) + '_landmarks']

    return net

my questions are,

1. above code makes an error in get_output_for :
   x_sym = theano.tensor.tensor4('x')
   " Expected an array-like object, but found a Variable: maybe you are trying to call a function on a (possibly shared) variable instead of a numeric array? "

( I'm referencing the theano test code "test_abstract_conv.py", https://github.com/Theano/Theano/blob/8dccbe6e1000239f57006e556fe8f737bb717aba/theano/tensor/nnet/tests/test_abstract_conv.py

There is def test_interface2d(self): in line 1683, and they test it with real numpy array values for input and deptwise/pointwise filters...

self.x = np.array([[[[1, 2, 3, 4, 5], [3, 2, 1, 4, 5], [3, 3, 1, 3, 6], [5, 3, 2, 1, 1], [4, 7, 1, 2, 1]],
[[3, 3, 1, 2, 6], [6, 5, 4, 3, 1], [3, 4, 5, 2, 3], [6, 4, 1, 3, 4], [2, 3, 4, 2, 5]]]]).astype(theano.config.floatX)
self.depthwise_filter = np.array([[[[3, 2, 1], [5, 3, 2], [6, 4, 2]]], [[[5, 5, 2], [3, 7, 4], [3, 5, 4]]],
[[[7, 4, 7], [5, 3, 3], [1, 3, 1]]], [[[4, 4, 4], [2, 4, 6], [0, 0, 7]]]]).astype(theano.config.floatX)
self.pointwise_filter = np.array([[[[4]], [[1]], [[3]], [[5]]], [[[2]], [[1]], [[2]], [[8]]]]).astype(theano.config.floatX)
x_sym = theano.tensor.tensor4('x')
dfilter_sym = theano.tensor.tensor4('d')
pfilter_sym = theano.tensor.tensor4('p')
sep_op = separable_conv2d(x_sym, dfilter_sym, pfilter_sym, self.x.shape[1])
fun = theano.function([x_sym, dfilter_sym, pfilter_sym], sep_op, mode='FAST_RUN')
top = fun(self.x, self.depthwise_filter, self.pointwise_filter)

but in my code, I'm passing "input"(is it TensorVariable?) to the theano.function

def get_output_for(self, input, **kwargs):
x_sym = theano.tensor.tensor4('x')
dfilter_sym = theano.tensor.tensor4('d')
pfilter_sym = theano.tensor.tensor4('p')
sep_op = separable_conv2d(x_sym , dfilter_sym , pfilter_sym , ...)
fun = theano.function([x_sym, dfilter_sym, pfilter_sym], sep_op, mode='FAST_RUN')
output = fun(input ,self.depthwise_filters, self.pointwise_filters)
return output

another try also failed :

def __init__(self, input, input_shape, num_dw_channels=32, num_pw_filters=32, filter_size=[3, 3], stride=[1, 1],
             **kwargs):
    self.input=inputs

...
...
def get_output_for(self, input, **kwargs):
x_sym = theano.tensor.tensor4('x_sym')
dfilter_sym = theano.tensor.tensor4('dfilter_sym')
pfilter_sym = theano.tensor.tensor4('pfilter_sym')
sep_op = separable_conv2d(x_sym ,
dfilter_sym,
pfilter_sym,... )
fun = theano.function([x_sym, dfilter_sym, pfilter_sym], sep_op, mode='FAST_RUN')
output = fun(self.input ,self.depthwise_filters, self.pointwise_filters)
return output

error : x_sym = theano.tensor.tensor4('x_sym')
float() argument must be a string or a number

is there a way to pass the real value to theano.function to avoid this error? I thouht, I should use symbols to build a graph for compile.

2. do you think the theano's new separable_conv2d op(which uses Abstract2D class) can replace the existing Conv2DLayer as I did? "Abstract" seems to me, it presents just an interface, so user should implement the actual mothod. But when I followed the theano codes, it seems there is an actual implementation for depthwise + pointwise conv in abtract_conv.py (https://github.com/Theano/Theano/blob/4d46e410bc765e9e288996c7da693146df69e3b9/theano/tensor/nnet/abstract_conv.py).

3. what method could you suggest for initializing depthwise/pointwise weight?

thank you in advance!

Why no weight decay?

i have a question about the code. Generally, when training the neural network, L2 normalization or the so called weight decay will be added to the total loss.
but in your code and the zjj implementation on Tensorflow, it seems like the loss is only the distance between the labels and prediction.
do i understand the code wrong? or i miss the weight decay factor?
i did not found any discuession in your paper. i am curious that if you add weight decay to your loss. if not, why not to add weight decay? or is this a trick on training network on regression problem?
thank you for your reply.

new with on stage is even better than which with two stage

Hi Marek,
i run DANtest.py and want to know the performance of different stage. However, the result of the network with one stage is even better. The data set is initialized as the paper said.below is the test result.(i used center distance)
OneStage
Processing common subset of the 300W public test set (test sets of LFPW and HELEN)
Average error: 0.428566349505
Processing challenging subset of the 300W public test set (IBUG dataset)
Average error: 0.598229653794
Showing results for the entire 300W pulic test set (IBUG dataset, test sets of LFPW and HELEN
Average error: 0.461809522334
TwoStage
Processing common subset of the 300W public test set (test sets of LFPW and HELEN)
Average error: 0.439570144885
Processing challenging subset of the 300W public test set (IBUG dataset)
Average error: 0.616254202783
Showing results for the entire 300W pulic test set (IBUG dataset, test sets of LFPW and HELEN
Average error: 0.474188937071

about trainging problem

hi，MarekKowalski，I used the code you provided to train the model. When I tested the image, it didn't respond. Using your model (DAN.npz) also meet same problem,but No problem with other images。I don't know what is going on.

2、load your network (DAN.npz),the validation error is 0.0177,the train error is 0.035395,When I train this network，the validation error is 0.04789,the train error is 0.0433754， If I continue training the network ,can I achieve the same result as you?How many epochs did you train（approximately）?Can these two stages train at the same time?

I hope you can help me ,thank you very much!

Different annotations

Dear colleagues,

First of all I would like to congratualate you for your excellent work.
I found a critical difference in the annotations. Where did you get these annotations? How did you compute the bounding box? Why these annotations are different from the 300w challenge?

For example:

Your annotations:
Image: lfpw/testset/image_0198.png
Bounding box: [ 152.95081243, 76.93296086, 312.66428949, 207.6076239 ]
Landmarks:
[ 152.196403, 105.019378],
[ 154.513368, 124.074788],
[ 158.260771, 143.04636 ],
[ 164.484855, 161.682191],
[ 175.107706, 178.684207],
[ 190.660592, 191.808697],
[ 208.852004, 201.198318],
[ 228.485914, 207.00529 ],
[ 248.668265, 207.756457],
[ 266.046124, 202.862818],
[ 277.387096, 189.872259],
[ 283.159731, 176.060588],
[ 292.860004, 161.712166],
[ 294.911208, 144.456357],
[ 293.630575, 127.963199],
[ 291.271658, 109.800198],
[ 288.96253 , 93.171127],
[ 190.076312, 92.038304],
[ 194.338551, 88.046244],
[ 202.254762, 85.73028 ],
[ 211.393757, 86.296981],
[ 224.596609, 89.32453 ],
[ 253.297234, 89.464433],
[ 261.183245, 83.927802],
[ 270.605391, 81.806645],
[ 279.759857, 82.998915],
[ 285.961887, 89.169947],
[ 241.312273, 100.02808 ],
[ 243.00472 , 111.089567],
[ 245.011426, 121.318851],
[ 246.926654, 131.786082],
[ 233.945945, 142.717822],
[ 241.51167 , 144.413906],
[ 248.327349, 144.695883],
[ 253.737043, 143.168839],
[ 258.116785, 140.864021],
[ 198.174932, 106.075823],
[ 207.115404, 101.455962],
[ 213.425674, 101.052585],
[ 220.853978, 104.883307],
[ 213.695762, 105.852023],
[ 207.363276, 105.984004],
[ 256.391043, 101.482455],
[ 262.630398, 97.135154],
[ 269.94615 , 95.466511],
[ 276.564927, 99.781536],
[ 270.194384, 101.803396],
[ 263.671057, 102.054199],
[ 213.475936, 165.978084],
[ 229.952814, 162.93299 ],
[ 239.840822, 160.84816 ],
[ 246.201577, 160.964465],
[ 253.865234, 159.977752],
[ 262.163952, 158.576558],
[ 273.439363, 158.312729],
[ 262.705848, 164.496517],
[ 254.703893, 167.144506],
[ 246.44157 , 168.89468 ],
[ 240.097913, 168.878754],
[ 230.195912, 167.371077],
[ 216.968855, 165.516275],
[ 239.998981, 163.842081],
[ 245.993046, 163.805091],
[ 254.203167, 163.10561 ],
[ 270.379387, 158.900874],
[ 254.203167, 163.10561 ],
[ 245.993046, 163.805091],
[ 239.998981, 163.842081]

LFPW bounding box:
[153.196403, 82.806645, 295.911208, 208.756457]

300w original annotations:
n_points: 68
{
153.196403 106.019378
155.513368 125.074788
159.260771 144.046360
165.484855 162.682191
176.107706 179.684207
191.660592 192.808697
209.852004 202.198318
229.485914 208.005290
249.668265 208.756457
267.046124 203.862818
278.387096 190.872259
284.159731 177.060588
293.860004 162.712166
295.911208 145.456357
294.630575 128.963199
292.271658 110.800198
289.962530 94.171127
191.076312 93.038304
195.338551 89.046244
203.254762 86.730280
212.393757 87.296981
225.596609 90.324530
254.297234 90.464433
262.183245 84.927802
271.605391 82.806645
280.759857 83.998915
286.961887 90.169947
242.312273 101.028080
244.004720 112.089567
246.011426 122.318851
247.926654 132.786082
234.945945 143.717822
242.511670 145.413906
249.327349 145.695883
254.737043 144.168839
259.116785 141.864021
199.174932 107.075823
208.115404 102.455962
214.425674 102.052585
221.853978 105.883307
214.695762 106.852023
208.363276 106.984004
257.391043 102.482455
263.630398 98.135154
270.946150 96.466511
277.564927 100.781536
271.194384 102.803396
264.671057 103.054199
214.475936 166.978084
230.952814 163.932990
240.840822 161.848160
247.201577 161.964465
254.865234 160.977752
263.163952 159.576558
274.439363 159.312729
263.705848 165.496517
255.703893 168.144506
247.441570 169.894680
241.097913 169.878754
231.195912 168.371077
217.968855 166.516275
240.998981 164.842081
246.993046 164.805091
255.203167 164.105610
271.379387 159.900874
255.203167 164.105610
246.993046 164.805091
240.998981 164.842081
}

I look formard to your response.

Best regards,
Roberto Valle

Communication group

q: 663196717

Unable to download pre-trained models

unable to download one of the two pre-trained models in the website, did you provide?

about meanShape

i have some question about meanshape.

should i Normalize meanshape in BBox?

I have not see you calc meanshape in your code.Can you share that?

Thanks!

what are the roles of heatmap and feat map？ especially feat map？

Hi，when I read your paper，I’m confused about the roles of the heatmap and featmap. What
is the motivation to use them?

how fast in average do it predict an image without gpu

Can not reach the performance reported on paper

I get same result as paper except challenging set and failure rate. Did I missed something?

how can I visualize heatmap ?

I'm trying to see how the heatmap is generated and focusing on draw_landmarks_helper.
def draw_landmarks_helper(self, landmark):
img = T.zeros((1, self.img_shape[0], self.img_shape[1]))

    intLandmark = landmark.astype('int32')
    locations = self.offsets + intLandmark
    dxdy = landmark - intLandmark

    offsetsSubPix = self.offsets - dxdy
    vals = 1 / (1 + T.sqrt(T.sum(offsetsSubPix * offsetsSubPix, axis=1) + 1e-6))

    img = T.set_subtensor(img[0, locations[:, 1], locations[:, 0]], vals)

    img2=T.cast(img,'uint8') 
    vals2=img2.eval()  << error!
    imshow("heatmap",vals2)
    return img

can you give me an advice on how to see the array or tensor?

thank you.

How to get my own meanFaceShape.npz

Hello:
If I want to get 194 landmarks, not 68 landmarks, how to get my own meanFaceShape.npz?

why did you split your network into two stages to train? what is the different between all to train and split train?

can't run CameraDemo.py

Hi,
When i run python i get this error

You can find the C code in this temporary file: /var/folders/w1/bt2ptyhn3ln60rxqg7wq68500000gn/T/theano_compilation_error_KB3_ox
Traceback (most recent call last):
File "CameraDemo.py", line 14, in
model.loadNetwork("../DAN-Menpo.npz")
File "/Users/kitegamesstudioltd/Desktop/My Resources/DeepAlignmentNetwork-master/DeepAlignmentNetwork/FaceAlignment.py", line 139, in loadNetwork
self.initializeNetwork()
File "/Users/kitegamesstudioltd/Desktop/My Resources/DeepAlignmentNetwork-master/DeepAlignmentNetwork/FaceAlignment.py", line 42, in initializeNetwork
self.generate_network_output = theano.function([self.data], [self.prediction])
File "/anaconda3/envs/dan_env/lib/python2.7/site-packages/theano/compile/function.py", line 317, in function
output_keys=output_keys)
File "/anaconda3/envs/dan_env/lib/python2.7/site-packages/theano/compile/pfunc.py", line 486, in pfunc
output_keys=output_keys)
File "/anaconda3/envs/dan_env/lib/python2.7/site-packages/theano/compile/function_module.py", line 1841, in orig_function
fn = m.create(defaults)
File "/anaconda3/envs/dan_env/lib/python2.7/site-packages/theano/compile/function_module.py", line 1715, in create
input_storage=input_storage_lists, storage_map=storage_map)
File "/anaconda3/envs/dan_env/lib/python2.7/site-packages/theano/gof/link.py", line 699, in make_thunk
storage_map=storage_map)[:3]
File "/anaconda3/envs/dan_env/lib/python2.7/site-packages/theano/gof/vm.py", line 1084, in make_all
impl=impl))
File "/anaconda3/envs/dan_env/lib/python2.7/site-packages/theano/scan_module/scan_op.py", line 866, in make_thunk
on_unused_input='ignore')
File "/anaconda3/envs/dan_env/lib/python2.7/site-packages/theano/compile/function.py", line 317, in function
output_keys=output_keys)
File "/anaconda3/envs/dan_env/lib/python2.7/site-packages/theano/compile/pfunc.py", line 486, in pfunc
output_keys=output_keys)
File "/anaconda3/envs/dan_env/lib/python2.7/site-packages/theano/compile/function_module.py", line 1841, in orig_function
fn = m.create(defaults)
File "/anaconda3/envs/dan_env/lib/python2.7/site-packages/theano/compile/function_module.py", line 1715, in create
input_storage=input_storage_lists, storage_map=storage_map)
File "/anaconda3/envs/dan_env/lib/python2.7/site-packages/theano/gof/link.py", line 699, in make_thunk
storage_map=storage_map)[:3]
File "/anaconda3/envs/dan_env/lib/python2.7/site-packages/theano/gof/vm.py", line 1084, in make_all
impl=impl))
File "/anaconda3/envs/dan_env/lib/python2.7/site-packages/theano/gof/op.py", line 955, in make_thunk
no_recycling)
File "/anaconda3/envs/dan_env/lib/python2.7/site-packages/theano/gof/op.py", line 858, in make_c_thunk
output_storage=node_output_storage)
File "/anaconda3/envs/dan_env/lib/python2.7/site-packages/theano/gof/cc.py", line 1217, in make_thunk
keep_lock=keep_lock)
File "/anaconda3/envs/dan_env/lib/python2.7/site-packages/theano/gof/cc.py", line 1157, in compile
keep_lock=keep_lock)
File "/anaconda3/envs/dan_env/lib/python2.7/site-packages/theano/gof/cc.py", line 1620, in cthunk_factory
key=key, lnk=self, keep_lock=keep_lock)
File "/anaconda3/envs/dan_env/lib/python2.7/site-packages/theano/gof/cmodule.py", line 1174, in module_from_key
module = lnk.compile_cmodule(location)
File "/anaconda3/envs/dan_env/lib/python2.7/site-packages/theano/gof/cc.py", line 1523, in compile_cmodule
preargs=preargs)
File "/anaconda3/envs/dan_env/lib/python2.7/site-packages/theano/gof/cmodule.py", line 2362, in compile_str
(status, compile_stderr.replace('\n', '. ')))
Exception: ('The following error happened while compiling the node', for{cpu,scan_fn}(Elemwise{maximum,no_inplace}.0, Subtensor{int64:int64:int8}.0, Elemwise{maximum,no_inplace}.0), '\n', 'The following error happened while compiling the node', Elemwise{Composite{((((i0 * i1) + (i2 * i3) + (i4 * i5) + (i6 * i7) + (i8 * i9) + (i10 * i11) + (i12 * i13) + (i14 * i15) + (i16 * i17) + (i18 * i19) + (i20 * i21) + (i22 * i23) + (i24 * i25) + (i26 * i27) + (i28 * i29) + (i30 * i31) + (i32 * i33) + (i34 * i35) + (i36 * i37) + (i38 * i39) + (i40 * i41) + (i42 * i43) + (i44 * i45) + (i46 * i47) + (i48 * i49) + (i50 * i51) + (i52 * i53) + (i54 * i55) + (i56 * i57) + (i58 * i59) + (i60 * i61) + (i62 * i63) + (i64 * i65) + (i66 * i67) + (i68 * i69) + (i70 * i71) + (i72 * i73) + (i74 * i75) + (i76 * i77) + (i78 * i79) + (i80 * i81) + (i82 * i83) + (i84 * i85) + (i86 * i87) + (i88 * i89) + (i90 * i91) + (i92 * i93) + (i94 * i95) + (i96 * i97) + (i98 * i99) + (i100 * i101) + (i102 * i103) + (i104 * i105) + (i106 * i107) + (i108 * i109) + (i110 * i111) + (i112 * i113) + (i114 * i115) + (i116 * i117) + (i118 * i119) + (i120 * i121) + (i122 * i123) + (i124 * i125) + (i126 * i127) + (i128 * i129) + (i130 * i131) + (i132 * i133) + (i134 * i135)) - ((i136 * i137) + (i138 * i139) + (i140 * i141) + (i142 * i143) + (i144 * i145) + (i146 * i147) + (i148 * i149) + (i150 * i151) + (i152 * i153) + (i154 * i155) + (i156 * i157) + (i158 * i159) + (i160 * i161) + (i162 * i163) + (i164 * i165) + (i166 * i167) + (i168 * i169) + (i170 * i171) + (i172 * i173) + (i174 * i175) + (i176 * i177) + (i178 * i179) + (i180 * i181) + (i182 * i183) + (i184 * i185) + (i186 * i187) + (i188 * i189) + (i190 * i191) + (i192 * i193) + (i194 * i195) + (i196 * i197) + (i198 * i199) + (i200 * i201) + (i202 * i203) + (i204 * i205) + (i206 * i207) + (i208 * i209) + (i210 * i211) + (i212 * i213) + (i214 * i215) + (i216 * i217) + (i218 * i219) + (i220 * i221) + (i222 * i223) + (i224 * i225) + (i226 * i227) + (i228 * i229) + (i230 * i231) + (i232 * i233) + (i234 * i235) + (i236 * i237) + (i238 * i239) + (i240 * i241) + (i242 * i243) + (i244 * i245) + (i246 * i247) + (i248 * i249) + (i250 * i251) + (i252 * i253) + (i254 * i255) + (i256 * i257) + (i258 * i259) + (i260 * i261) + (i262 * i263) + (i264 * i265) + (i266 * i267) + (i268 * i269) + (i270 * i271))) / i272)}}[(0, 272)](TensorConstant{-14.4735313462}, Subtensor{int64}.0, TensorConstant{-7.10659422493}, Subtensor{int64}.0, TensorConstant{0.228484430573}, Subtensor{int64}.0, TensorConstant{7.44272422616}, Subtensor{int64}.0, TensorConstant{14.1509726694}, Subtensor{int64}.0, TensorConstant{19.9664712762}, Subtensor{int64}.0, TensorConstant{24.8196493208}, Subtensor{int64}.0, TensorConstant{28.7477676676}, Subtensor{int64}.0, TensorConstant{29.9043180189}, Subtensor{int64}.0, TensorConstant{28.7477677318}, Subtensor{int64}.0, TensorConstant{24.8196494416}, Subtensor{int64}.0, TensorConstant{19.9664714486}, Subtensor{int64}.0, TensorConstant{14.1509728836}, Subtensor{int64}.0, TensorConstant{7.44272446746}, Subtensor{int64}.0, TensorConstant{0.228484686441}, Subtensor{int64}.0, TensorConstant{-7.10659396128}, Subtensor{int64}.0, TensorConstant{-14.4735310808}, Subtensor{int64}.0, TensorConstant{-19.9418652508}, Subtensor{int64}.0, TensorConstant{-23.1123126663}, Subtensor{int64}.0, TensorConstant{-24.0425229144}, Subtensor{int64}.0, TensorConstant{-23.2982320856}, Subtensor{int64}.0, TensorConstant{-21.2989884203}, Subtensor{int64}.0, TensorConstant{-21.298988378}, Subtensor{int64}.0, TensorConstant{-23.298231998}, Subtensor{int64}.0, TensorConstant{-24.0425227785}, Subtensor{int64}.0, TensorConstant{-23.1123124835}, Subtensor{int64}.0, TensorConstant{-19.9418650348}, Subtensor{int64}.0, TensorConstant{-15.4984236189}, Subtensor{int64}.0, TensorConstant{-10.7004146801}, Subtensor{int64}.0, TensorConstant{-5.93813664148}, Subtensor{int64}.0, TensorConstant{-1.02819780716}, Subtensor{int64}.0, TensorConstant{2.21003852805}, Subtensor{int64}.0, TensorConstant{3.19359808726}, Subtensor{int64}.0, TensorConstant{4.0659737295}, Subtensor{int64}.0, TensorConstant{3.19359811495}, Subtensor{int64}.0, TensorConstant{2.21003858145}, Subtensor{int64}.0, TensorConstant{-14.9346879758}, Subtensor{int64}.0, TensorConstant{-16.7076153138}, Subtensor{int64}.0, TensorConstant{-16.6514104005}, Subtensor{int64}.0, TensorConstant{-14.19162775}, Subtensor{int64}.0, TensorConstant{-13.5508757973}, Subtensor{int64}.0, TensorConstant{-13.6061589293}, Subtensor{int64}.0, TensorConstant{-14.1916276811}, Subtensor{int64}.0, TensorConstant{-16.6514103016}, Subtensor{int64}.0, TensorConstant{-16.7076151802}, Subtensor{int64}.0, TensorConstant{-14.9346878137}, Subtensor{int64}.0, TensorConstant{-13.6061587936}, Subtensor{int64}.0, TensorConstant{-13.550875696}, Subtensor{int64}.0, TensorConstant{11.1021196367}, Subtensor{int64}.0, TensorConstant{9.52927584127}, Subtensor{int64}.0, TensorConstant{8.65379278407}, Subtensor{int64}.0, TensorConstant{9.38025141212}, Subtensor{int64}.0, TensorConstant{8.65379281076}, Subtensor{int64}.0, TensorConstant{9.52927590628}, Subtensor{int64}.0, TensorConstant{11.1021197397}, Subtensor{int64}.0, TensorConstant{14.9450825621}, Subtensor{int64}.0, TensorConstant{16.6244728612}, Subtensor{int64}.0, TensorConstant{16.9483126378}, Subtensor{int64}.0, TensorConstant{16.6244728321}, Subtensor{int64}.0, TensorConstant{14.9450824959}, Subtensor{int64}.0, TensorConstant{11.3230145313}, Subtensor{int64}.0, TensorConstant{11.0547952378}, Subtensor{int64}.0, TensorConstant{11.3660532042}, Subtensor{int64}.0, TensorConstant{11.0547952649}, Subtensor{int64}.0, TensorConstant{11.3230146182}, Subtensor{int64}.0, TensorConstant{13.000637367}, Subtensor{int64}.0, TensorConstant{13.3473126106}, Subtensor{int64}.0, TensorConstant{13.0006373394}, Subtensor{int64}.0, TensorConstant{-27.9999989052}, Subtensor{int64}.0, TensorConstant{-27.8152649598}, Subtensor{int64}.0, TensorConstant{-26.9940529743}, Subtensor{int64}.0, TensorConstant{-25.4570565047}, Subtensor{int64}.0, TensorConstant{-22.6031768021}, Subtensor{int64}.0, TensorConstant{-18.18698962}, Subtensor{int64}.0, TensorConstant{-12.7524670283}, Subtensor{int64}.0, TensorConstant{-6.77296267908}, Subtensor{int64}.0, TensorConstant{4.87141548433e-07}, Subtensor{int64}.0, TensorConstant{6.77296366432}, Subtensor{int64}.0, TensorConstant{12.7524632993}, Subtensor{int64}.0, TensorConstant{18.186985937}, Subtensor{int64}.0, TensorConstant{22.6031731742}, Subtensor{int64}.0, TensorConstant{25.4570553161}, Subtensor{int64}.0, TensorConstant{26.9940542298}, Subtensor{int64}.0, TensorConstant{27.8152662849}, Subtensor{int64}.0, TensorConstant{28.0000026758}, Subtensor{int64}.0, TensorConstant{-22.794916633}, Subtensor{int64}.0, TensorConstant{-19.2901096856}, Subtensor{int64}.0, TensorConstant{-14.3418123952}, Subtensor{int64}.0, TensorConstant{-9.239743953}, Subtensor{int64}.0, TensorConstant{-4.46412385365}, Subtensor{int64}.0, TensorConstant{4.46412056182}, Subtensor{int64}.0, TensorConstant{9.23974543157}, Subtensor{int64}.0, TensorConstant{14.3418138808}, Subtensor{int64}.0, TensorConstant{19.2901111624}, Subtensor{int64}.0, TensorConstant{22.7949133283}, Subtensor{int64}.0, TensorConstant{7.0232063365e-07}, Subtensor{int64}.0, TensorConstant{6.79581248164e-07}, Subtensor{int64}.0, TensorConstant{6.57011170802e-07}, Subtensor{int64}.0, TensorConstant{6.33741294109e-07}, Subtensor{int64}.0, TensorConstant{-5.63343192615}, Subtensor{int64}.0, TensorConstant{-2.92130931606}, Subtensor{int64}.0, TensorConstant{6.09598274082e-07}, Subtensor{int64}.0, TensorConstant{2.92131054352}, Subtensor{int64}.0, TensorConstant{5.63343316294}, Subtensor{int64}.0, TensorConstant{-17.0965413006}, Subtensor{int64}.0, TensorConstant{-14.0867688759}, Subtensor{int64}.0, TensorConstant{-10.4397035462}, Subtensor{int64}.0, TensorConstant{-7.26539794878}, Subtensor{int64}.0, TensorConstant{-10.6920055286}, Subtensor{int64}.0, TensorConstant{-14.3180409429}, Subtensor{int64}.0, TensorConstant{7.26539934104}, Subtensor{int64}.0, TensorConstant{10.4397049618}, Subtensor{int64}.0, TensorConstant{14.0867655406}, Subtensor{int64}.0, TensorConstant{17.0965379485}, Subtensor{int64}.0, TensorConstant{14.3180375781}, Subtensor{int64}.0, TensorConstant{10.6920021633}, Subtensor{int64}.0, TensorConstant{-10.8582778687}, Subtensor{int64}.0, TensorConstant{-6.85898282948}, Subtensor{int64}.0, TensorConstant{-2.81555633074}, Subtensor{int64}.0, TensorConstant{5.84412106264e-07}, Subtensor{int64}.0, TensorConstant{2.81555750645}, Subtensor{int64}.0, TensorConstant{6.85898399689}, Subtensor{int64}.0, TensorConstant{10.8582790212}, Subtensor{int64}.0, TensorConstant{6.98245513755}, Subtensor{int64}.0, TensorConstant{3.06514635869}, Subtensor{int64}.0, TensorConstant{5.48544491608e-07}, Subtensor{int64}.0, TensorConstant{-3.06514525853}, Subtensor{int64}.0, TensorConstant{-6.98245402147}, Subtensor{int64}.0, TensorConstant{-9.17517191857}, Subtensor{int64}.0, TensorConstant{-2.85792976863}, Subtensor{int64}.0, TensorConstant{5.75000711933e-07}, Subtensor{int64}.0, TensorConstant{2.85793092158}, Subtensor{int64}.0, TensorConstant{9.17517306898}, Subtensor{int64}.0, TensorConstant{2.911109135}, Subtensor{int64}.0, TensorConstant{5.65610847048e-07}, Subtensor{int64}.0, TensorConstant{-2.91110800049}, Subtensor{int64}.0, Elemwise{Composite{sqr(sqrt(i0))}}[(0, 0)].0), '\n', 'Compilation failed (return status=1): /Users/kitegamesstudioltd/.theano/compiledir_Darwin-17.3.0-x86_64-i386-64bit-i386-2.7.13-64/tmpAG35r_/mod.cpp:24996:32: fatal error: bracket nesting level exceeded maximum of 256. if (!PyErr_Occurred()) {. ^. /Users/kitegamesstudioltd/.theano/compiledir_Darwin-17.3.0-x86_64-i386-64bit-i386-2.7.13-64/tmpAG35r_/mod.cpp:24996:32: note: use -fbracket-depth=N to increase maximum nesting level. 1 error generated.. ', '[Elemwise{Composite{((((i0 * i1) + (i2 * i3) + (i4 * i5) + (i6 * i7) + (i8 * i9) + (i10 * i11) + (i12 * i13) + (i14 * i15) + (i16 * i17) + (i18 * i19) + (i20 * i21) + (i22 * i23) + (i24 * i25) + (i26 * i27) + (i28 * i29) + (i30 * i31) + (i32 * i33) + (i34 * i35) + (i36 * i37) + (i38 * i39) + (i40 * i41) + (i42 * i43) + (i44 * i45) + (i46 * i47) + (i48 * i49) + (i50 * i51) + (i52 * i53) + (i54 * i55) + (i56 * i57) + (i58 * i59) + (i60 * i61) + (i62 * i63) + (i64 * i65) + (i66 * i67) + (i68 * i69) + (i70 * i71) + (i72 * i73) + (i74 * i75) + (i76 * i77) + (i78 * i79) + (i80 * i81) + (i82 * i83) + (i84 * i85) + (i86 * i87) + (i88 * i89) + (i90 * i91) + (i92 * i93) + (i94 * i95) + (i96 * i97) + (i98 * i99) + (i100 * i101) + (i102 * i103) + (i104 * i105) + (i106 * i107) + (i108 * i109) + (i110 * i111) + (i112 * i113) + (i114 * i115) + (i116 * i117) + (i118 * i119) + (i120 * i121) + (i122 * i123) + (i124 * i125) + (i126 * i127) + (i128 * i129) + (i130 * i131) + (i132 * i133) + (i134 * i135)) - ((i136 * i137) + (i138 * i139) + (i140 * i141) + (i142 * i143) + (i144 * i145) + (i146 * i147) + (i148 * i149) + (i150 * i151) + (i152 * i153) + (i154 * i155) + (i156 * i157) + (i158 * i159) + (i160 * i161) + (i162 * i163) + (i164 * i165) + (i166 * i167) + (i168 * i169) + (i170 * i171) + (i172 * i173) + (i174 * i175) + (i176 * i177) + (i178 * i179) + (i180 * i181) + (i182 * i183) + (i184 * i185) + (i186 * i187) + (i188 * i189) + (i190 * i191) + (i192 * i193) + (i194 * i195) + (i196 * i197) + (i198 * i199) + (i200 * i201) + (i202 * i203) + (i204 * i205) + (i206 * i207) + (i208 * i209) + (i210 * i211) + (i212 * i213) + (i214 * i215) + (i216 * i217) + (i218 * i219) + (i220 * i221) + (i222 * i223) + (i224 * i225) + (i226 * i227) + (i228 * i229) + (i230 * i231) + (i232 * i233) + (i234 * i235) + (i236 * i237) + (i238 * i239) + (i240 * i241) + (i242 * i243) + (i244 * i245) + (i246 * i247) + (i248 * i249) + (i250 * i251) + (i252 * i253) + (i254 * i255) + (i256 * i257) + (i258 * i259) + (i260 * i261) + (i262 * i263) + (i264 * i265) + (i266 * i267) + (i268 * i269) + (i270 * i271))) / i272)}}[(0, 272)](TensorConstant{-14.4735313462}, <TensorType(float64, scalar)>, TensorConstant{-7.10659422493}, <TensorType(float64, scalar)>, TensorConstant{0.228484430573}, <TensorType(float64, scalar)>, TensorConstant{7.44272422616}, <TensorType(float64, scalar)>, TensorConstant{14.1509726694}, <TensorType(float64, scalar)>, TensorConstant{19.9664712762}, <TensorType(float64, scalar)>, TensorConstant{24.8196493208}, <TensorType(float64, scalar)>, TensorConstant{28.7477676676}, <TensorType(float64, scalar)>, TensorConstant{29.9043180189}, <TensorType(float64, scalar)>, TensorConstant{28.7477677318}, <TensorType(float64, scalar)>, TensorConstant{24.8196494416}, <TensorType(float64, scalar)>, TensorConstant{19.9664714486}, <TensorType(float64, scalar)>, TensorConstant{14.1509728836}, <TensorType(float64, scalar)>, TensorConstant{7.44272446746}, <TensorType(float64, scalar)>, TensorConstant{0.228484686441}, <TensorType(float64, scalar)>, TensorConstant{-7.10659396128}, <TensorType(float64, scalar)>, TensorConstant{-14.4735310808}, <TensorType(float64, scalar)>, TensorConstant{-19.9418652508}, <TensorType(float64, scalar)>, TensorConstant{-23.1123126663}, <TensorType(float64, scalar)>, TensorConstant{-24.0425229144}, <TensorType(float64, scalar)>, TensorConstant{-23.2982320856}, <TensorType(float64, scalar)>, TensorConstant{-21.2989884203}, <TensorType(float64, scalar)>, TensorConstant{-21.298988378}, <TensorType(float64, scalar)>, TensorConstant{-23.298231998}, <TensorType(float64, scalar)>, TensorConstant{-24.0425227785}, <TensorType(float64, scalar)>, TensorConstant{-23.1123124835}, <TensorType(float64, scalar)>, TensorConstant{-19.9418650348}, <TensorType(float64, scalar)>, TensorConstant{-15.4984236189}, <TensorType(float64, scalar)>, TensorConstant{-10.7004146801}, <TensorType(float64, scalar)>, TensorConstant{-5.93813664148}, <TensorType(float64, scalar)>, TensorConstant{-1.02819780716}, <TensorType(float64, scalar)>, TensorConstant{2.21003852805}, <TensorType(float64, scalar)>, TensorConstant{3.19359808726}, <TensorType(float64, scalar)>, TensorConstant{4.0659737295}, <TensorType(float64, scalar)>, TensorConstant{3.19359811495}, <TensorType(float64, scalar)>, TensorConstant{2.21003858145}, <TensorType(float64, scalar)>, TensorConstant{-14.9346879758}, <TensorType(float64, scalar)>, TensorConstant{-16.7076153138}, <TensorType(float64, scalar)>, TensorConstant{-16.6514104005}, <TensorType(float64, scalar)>, TensorConstant{-14.19162775}, <TensorType(float64, scalar)>, TensorConstant{-13.5508757973}, <TensorType(float64, scalar)>, TensorConstant{-13.6061589293}, <TensorType(float64, scalar)>, TensorConstant{-14.1916276811}, <TensorType(float64, scalar)>, TensorConstant{-16.6514103016}, <TensorType(float64, scalar)>, TensorConstant{-16.7076151802}, <TensorType(float64, scalar)>, TensorConstant{-14.9346878137}, <TensorType(float64, scalar)>, TensorConstant{-13.6061587936}, <TensorType(float64, scalar)>, TensorConstant{-13.550875696}, <TensorType(float64, scalar)>, TensorConstant{11.1021196367}, <TensorType(float64, scalar)>, TensorConstant{9.52927584127}, <TensorType(float64, scalar)>, TensorConstant{8.65379278407}, <TensorType(float64, scalar)>, TensorConstant{9.38025141212}, <TensorType(float64, scalar)>, TensorConstant{8.65379281076}, <TensorType(float64, scalar)>, TensorConstant{9.52927590628}, <TensorType(float64, scalar)>, TensorConstant{11.1021197397}, <TensorType(float64, scalar)>, TensorConstant{14.9450825621}, <TensorType(float64, scalar)>, TensorConstant{16.6244728612}, <TensorType(float64, scalar)>, TensorConstant{16.9483126378}, <TensorType(float64, scalar)>, TensorConstant{16.6244728321}, <TensorType(float64, scalar)>, TensorConstant{14.9450824959}, <TensorType(float64, scalar)>, TensorConstant{11.3230145313}, <TensorType(float64, scalar)>, TensorConstant{11.0547952378}, <TensorType(float64, scalar)>, TensorConstant{11.3660532042}, <TensorType(float64, scalar)>, TensorConstant{11.0547952649}, <TensorType(float64, scalar)>, TensorConstant{11.3230146182}, <TensorType(float64, scalar)>, TensorConstant{13.000637367}, <TensorType(float64, scalar)>, TensorConstant{13.3473126106}, <TensorType(float64, scalar)>, TensorConstant{13.0006373394}, <TensorType(float64, scalar)>, TensorConstant{-27.9999989052}, <TensorType(float64, scalar)>, TensorConstant{-27.8152649598}, <TensorType(float64, scalar)>, TensorConstant{-26.9940529743}, <TensorType(float64, scalar)>, TensorConstant{-25.4570565047}, <TensorType(float64, scalar)>, TensorConstant{-22.6031768021}, <TensorType(float64, scalar)>, TensorConstant{-18.18698962}, <TensorType(float64, scalar)>, TensorConstant{-12.7524670283}, <TensorType(float64, scalar)>, TensorConstant{-6.77296267908}, <TensorType(float64, scalar)>, TensorConstant{4.87141548433e-07}, <TensorType(float64, scalar)>, TensorConstant{6.77296366432}, <TensorType(float64, scalar)>, TensorConstant{12.7524632993}, <TensorType(float64, scalar)>, TensorConstant{18.186985937}, <TensorType(float64, scalar)>, TensorConstant{22.6031731742}, <TensorType(float64, scalar)>, TensorConstant{25.4570553161}, <TensorType(float64, scalar)>, TensorConstant{26.9940542298}, <TensorType(float64, scalar)>, TensorConstant{27.8152662849}, <TensorType(float64, scalar)>, TensorConstant{28.0000026758}, <TensorType(float64, scalar)>, TensorConstant{-22.794916633}, <TensorType(float64, scalar)>, TensorConstant{-19.2901096856}, <TensorType(float64, scalar)>, TensorConstant{-14.3418123952}, <TensorType(float64, scalar)>, TensorConstant{-9.239743953}, <TensorType(float64, scalar)>, TensorConstant{-4.46412385365}, <TensorType(float64, scalar)>, TensorConstant{4.46412056182}, <TensorType(float64, scalar)>, TensorConstant{9.23974543157}, <TensorType(float64, scalar)>, TensorConstant{14.3418138808}, <TensorType(float64, scalar)>, TensorConstant{19.2901111624}, <TensorType(float64, scalar)>, TensorConstant{22.7949133283}, <TensorType(float64, scalar)>, TensorConstant{7.0232063365e-07}, <TensorType(float64, scalar)>, TensorConstant{6.79581248164e-07}, <TensorType(float64, scalar)>, TensorConstant{6.57011170802e-07}, <TensorType(float64, scalar)>, TensorConstant{6.33741294109e-07}, <TensorType(float64, scalar)>, TensorConstant{-5.63343192615}, <TensorType(float64, scalar)>, TensorConstant{-2.92130931606}, <TensorType(float64, scalar)>, TensorConstant{6.09598274082e-07}, <TensorType(float64, scalar)>, TensorConstant{2.92131054352}, <TensorType(float64, scalar)>, TensorConstant{5.63343316294}, <TensorType(float64, scalar)>, TensorConstant{-17.0965413006}, <TensorType(float64, scalar)>, TensorConstant{-14.0867688759}, <TensorType(float64, scalar)>, TensorConstant{-10.4397035462}, <TensorType(float64, scalar)>, TensorConstant{-7.26539794878}, <TensorType(float64, scalar)>, TensorConstant{-10.6920055286}, <TensorType(float64, scalar)>, TensorConstant{-14.3180409429}, <TensorType(float64, scalar)>, TensorConstant{7.26539934104}, <TensorType(float64, scalar)>, TensorConstant{10.4397049618}, <TensorType(float64, scalar)>, TensorConstant{14.0867655406}, <TensorType(float64, scalar)>, TensorConstant{17.0965379485}, <TensorType(float64, scalar)>, TensorConstant{14.3180375781}, <TensorType(float64, scalar)>, TensorConstant{10.6920021633}, <TensorType(float64, scalar)>, TensorConstant{-10.8582778687}, <TensorType(float64, scalar)>, TensorConstant{-6.85898282948}, <TensorType(float64, scalar)>, TensorConstant{-2.81555633074}, <TensorType(float64, scalar)>, TensorConstant{5.84412106264e-07}, <TensorType(float64, scalar)>, TensorConstant{2.81555750645}, <TensorType(float64, scalar)>, TensorConstant{6.85898399689}, <TensorType(float64, scalar)>, TensorConstant{10.8582790212}, <TensorType(float64, scalar)>, TensorConstant{6.98245513755}, <TensorType(float64, scalar)>, TensorConstant{3.06514635869}, <TensorType(float64, scalar)>, TensorConstant{5.48544491608e-07}, <TensorType(float64, scalar)>, TensorConstant{-3.06514525853}, <TensorType(float64, scalar)>, TensorConstant{-6.98245402147}, <TensorType(float64, scalar)>, TensorConstant{-9.17517191857}, <TensorType(float64, scalar)>, TensorConstant{-2.85792976863}, <TensorType(float64, scalar)>, TensorConstant{5.75000711933e-07}, <TensorType(float64, scalar)>, TensorConstant{2.85793092158}, <TensorType(float64, scalar)>, TensorConstant{9.17517306898}, <TensorType(float64, scalar)>, TensorConstant{2.911109135}, <TensorType(float64, scalar)>, TensorConstant{5.65610847048e-07}, <TensorType(float64, scalar)>, TensorConstant{-2.91110800049}, <TensorType(float64, scalar)>, <TensorType(float64, scalar)>)]')

How to train the model with confidence layer

Hi Marek,
Thanks for your excellent work.
I notice that you add a confidence layer in the FaceAlignment script, but it seems that there isn't a confidence layer in the FaceAlignmentTraining script. How do you manage the landmark loss and the confidence loss? How can I train the model with the confidence layer?
Thanks.

some problems about training.

I wonder how many epochs should be set to train the model, and should the learning rate be updated during training?

Error when generate validation dataset

Hi friends,
I got an error:
Traceback (most recent call last):
File "TrainingSetPreparation.py", line 21, in
validationSet.PrepareData(imageDirs, boundingBoxFiles, meanShape, 0, 100, False)
File "/home/diffdeep/Documents/DiffCat/face_landmarks/face_landmarks/ImageServer.py", line 71, in PrepareData
boundingBoxes.append(boundingBoxDict[basename])
KeyError: 'image_0295.png'
Please help me! Thanks

The first stage training average face shape

The paper said The first stage only takes the input image as the initial landmarks are always assumed to be the average face shape S0 located in the middle, but the code in FaceAlignmentTraining.py sames doesn't using the average face shape. In the first stage training the code just choose the first landmark in trainSet.initLandmarks.

DeepAlignmentNetwork/DeepAlignmentNetwork/FaceAlignmentTraining.py

Line 174 in 27ccabc

self.initLandmarks = trainSet.initLandmarks[0]

How to predict without face detection?

I'm kind of confused in the testing stage. The paper declares the DAN can still obtain good results without face detections but I cannot find implementation details about this step. I tried to use a fixed small bbox in the center of picture to initialize the mean shape while this method failed. Anybody could give me suggestions? Thanks a lot.

About input image's size and the fc layer?

Hi,
Do you explore differents input image's size? and can we use convolution layers instead of fc layer at the end of the net work? Or the fc layer has some advantage in alignments?

About the training set preparation

In TrainingSetPreparation.py, I found that
trainSet.PrepareData(imageDirs, None, meanShape, 100, 100000, True),
However, there are only about 3400 images in the 300W dataset, why did you set the nImgs=100000?

dont make sense of TransformParamsLayer

hi ,i have read TransformParamsLayer.py again and again,still cannot make sense what TransformParamsLayer doing.could you explain it ?

thank you very much. and my email is [email protected].

question about the ImageDemo

hello：
when i test the ImageDemo.py, i find that a picture of 600*640 pixes need 17s on cpu, and 10s on gpu; However, this is not the case in the paper. Is the processing time normal? Time is mainly spent on a function of theano, can time continue to be optimized?
thanks，best wishes