FLAME parameters from VOCA model

[jzf2101]

FLAME parameters from VOCA model

[TimoBolkart]

Thanks for pointing this out. This is actually a problem of the L-BFGS-B solver used in the TF_FLAME fitting. Weighting the data term higher, will give you a better result. Alternatively you could change the convergence parameters of the solver. Below, you can see your input mesh (left), a fitting based on the Chumpy code (middle), and a fitting using TF_FLAME with higher weighted data term, and using BFGS instead of L-BFGS-B (right). They look much more similar compared to your fitting before. Just pull the TF_FLAME repository and you will get the changes.

Increasing the parameter weights in for jaw_pose by 8 orders of magnitude (1000) leads to a slightly more open mouth, which is still not close to the VOCA mesh.

The weights for shape, expression, neck_pose, jaw_pose, and eyeball_pose are actually weights of the regularizers. Increasing these weights leads to more regularization and hence it even looks more like the mean face.

[TimoBolkart]

VOCA outputs meshes in FLAME mesh topology. However, it does not directly output FLAME parameters but you can easily compute this using this code TF_FLAME. The demo to fit the 3D model to registered 3D meshes should do the job

[jzf2101]

@tursmanor and I have tried to run TF_FLAME/fit_3D_mesh.py on the mesh output of VOCA. However, we find that there are notable differences in pose, specifically with regards to the mouth position.

The mesh on the left is the output of VOCA and the one on the right is the result of fit_3D_mesh.py

Given the fact that the output of VOCA is already in FLAME topology, is there another way to get the FLAME parameters directly from the VOCA model without rerunning it through FLAME?

[TimoBolkart]

The mesh on the left is the output of VOCA and the one on the right is the result of fit_3D_mesh.py

This looks not right, FLAME should be much better in fitting this mesh. Can you please provide this particular mesh to me?

Given the fact that the output of VOCA is already in FLAME topology, is there another way to get the FLAME parameters directly from the VOCA model without rerunning it through FLAME?

VOCA outputs meshes in FLAME topology, but there is no way to directly go from a FLAME mesh to the model parameters without optimization.

[jzf2101]

Thanks, the file is below:

00056.obj.zip

We have attempted to vary the weight parameters in TF_FLAME/fit_3D_mesh.py to improve performance with little luck. Increasing the parameter weights in for jaw_pose by 8 orders of magnitude (1000) leads to a slightly more open mouth, which is still not close to the VOCA mesh.

[jzf2101]

Thank you !!!!