trigeorgis / mdm Goto Github PK
View Code? Open in Web Editor NEWA TensorFlow implementation of the Mnemonic Descent Method.
License: BSD 3-Clause "New" or "Revised" License
A TensorFlow implementation of the Mnemonic Descent Method.
License: BSD 3-Clause "New" or "Revised" License
Hi! First of all, congratulations on your work!
Could you please provide the runtime of the algorithm for processing one frame (on CPU and GPU)?
Hi Trigeorgis,
Thanks for your code.
I have few questions about your AUC.
In your code,
Line 130 in fc8d011
(errors < .08).mean()
I think it should be the accuracy of the 0.08 threshold rather than AUC?
Would you mind to let me how to calculate the [email protected]? Thanks a lot.
Dear colleagues,
First of all I would like to congratulate you for your excellent work.
I found a critical difference in the bounding boxes annotations. Why your annotations have been added with 1 pixel extra?
For example:
Image: afw/823016568_2.pts
Your annotations:
Bounding box: [ 397.500 445.500 397.500 709.500 685.500 709.500 685.500 445.500 ]
Original annotations:
bb_detector: 396.5,444.5,684.5,708.5
On the other hand, how did you compute the bounding boxes for the 300w private images? Are you using a public face detector? I have not found it in the website. "As opposed to last year's Challenge (300-W 2013), in this version we will not provide any face detection initializations. Each system should detect the face in the image and then localize the facial landmarks."
I look formard to your response.
Best regards,
Roberto Valle
I'm a newbie in Tensorflow. Currently, I'm trying to run the demo (Demo.ipynb) given in the source code. But I can't install Tensorflow (using the method: 'install from source' given in the Tensorflow official website) inside the Conda environment. Specifically, after I git clone from the repository (https://github.com/trigeorgis/tensorflow), I can't pass the configure operation (./configure), could you give me a detailed manual for the environment configuration?
I've installed menpo package under Conda environment, what should I do next? How can I install Tensforflow from your repository into Conda environment, then I can import both Tensorflow and menpo libraries at the same time.
Hey George,
I study your code for a while, but there is no RNN in the model definition.
In the tensorflow, we always use:
cell = rnn_cell.BasicLSTMCell()
output, state = tf.nn.rnn()
to define a LSTM cell.
But in your code, there is only two fully_connected
layer in the rnn
scope.
Could you pls give me an instruction on that?
Hi, trigeorgis!
Thanks for sharing your code.
In the provided code, the pre-trained model get AUC of 41.04 which is much lower than 45.32(reported in the paper).
And I can't find the right website to download test set of 300w. http://ibug.doc.ic.ac.uk/resources/facial-point-annotations/ only provide the full set.
Thanks.
Hey, Georgis,
I have question about moving extracting patches function in v0.10.0. But now it happen with problem that without shape function.
Do you know how to define the shape function for the extracting patches?
To train the model, I have done as README on the github shows several times. But in all training processes, there are two cases that really confuse me:
In the first case, the total training loss is close to zero when the training step comes to 16,000, and then the training process stops because of error "Model diverged with loss = NaN".
In the second case, the total loss keeps Uunchanged after about 2,000 steps (Does it mean that the model has converged after 2,000 steps?), but the max step is set as 100,000. When the steps come to about 30,000, the training process also stops because of the same error: "Model diverged with loss = NaN"...
I utilize the trained models in both cases to evaluate on the 300W, but the results are a little terrible...For example, the RMSE on lfpw testset is about 18.42%...
The only difference from original code is that the extract_patches.cc is compiled into a op library as extract_patches.so, and it is loaded to extract images patches in the training process.
So, could you please explain the two cases to me and tell me how to train the mdm model exactly? I will really appreciate it!
Hi, thank you for sharing the code.
Coulde you share me with the reference_shape.pkl?
Thank you very much
@trigeorgis Can you provide the mapping performed on dlib output (for frontal face detection) to get bounding box used for facial key point prediction.
Traceback (most recent call last):
File "mdm_train.py", line 257, in
train()
File "mdm_train.py", line 82, in train
data_provider.load_images(train_dirs)
File "/home/hliu/Desktop/code/mdm/data_provider.py", line 150, in load_images
group = group or im.landmarks[group]._group_label
Hi, I get a problem on menpo. My menpo version is 0.8.1.
I change the line 150 to " group = group or im.landmarks.keys()[0] ", it works.
But the trained result is bad. I wonder if this is the reason.
when I run "git clone [email protected]:trigeorgis/tensorflow.git", but with error output:
"Cloning into 'tensorflow'...
Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists."
Does the tenforflow missing? Thank you!
I am a newer of tensorflow. I got a problem on tf.image.extract_patches in mdm_model.py , and I can't found this function in my tensorflow1.10.0. Could you tell me what version of tensorflow you used in this Engineer and what is the function of this function.
I use this code for training and with other dataset (which works very well with SDM method), but with MDM loss is decreasing few steps and then just stops, and weights started to be too small
I have the question,
File "/home/yeomja99/FaceLandmark/mdm-master/data_provider.py", line 38, in
if group.lms.n_points == 68
AttributeError: 'dict' object has no attribute 'lms'
so,how did you solve the problem?
Thank you.
Hi trigeorgis,
Instead of installing your custom TF repo to gain the custom extract image patches operation, I tried to write the operation in python with TF 1.0. This is what I came up with:
`
unset_patches = []
for offset in tf.unstack(inits+dx, axis=1):
batch_patches = tf.image.extract_glimpse(images, patch_shape, offset)
unset_patches.append(batch_patches)
# patches = tf.transpose(unset_patches, (1, 0, 2, 3, 4))
patches = tf.stack(unset_patches, axis=1)
patches = tf.reshape(patches, (batch_size * num_patches, patch_shape[0], patch_shape[1], num_channels))`
From here training is able to run, but no learning takes place and losses hover around 1.0 for thousands of steps.
Is there something I'm missing with this? Perhaps the optimizer can't back propagate losses through the operations taking place in python here, or there's an error in the code above when extracting and stacking these patches?
Let me know if you'd like any more information.
Help is greatly appreciated!
Hi trigeorgis, thanks for your nice code. I have compiled and run the training process but found there is a big gap between the model trained myself and the one you provided. Data augmentation is one factor as you have mentioned in README. Is there any other reason for this gap? And could you provide some details about your data augmentation? Thanks.
The command indicated in manual won't work:
git clone [email protected]:trigeorgis/tensorflow.git
This one does:
git clone https://github.com/trigeorgis/tensorflow.git
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.