Comments (5)
Hi, there are two steps you need to do, First, you need to register the scan with the FLAME model (for details you please see the paper). Then you need to undo the effects of pose (i.e. global orientation, head pose, jaw pose) and facial expression. After this process, the mesh is in zero pose (i.e. canonical FLAME pose). Then it can be used together with VOCA.
from voca.
Hi thank you for taking the time to answer.
But it still doesn't clarify how to take a non-FLAME mesh and convert it to FLAME topology.
You mention:
First, you need to register the scan with the FLAME model
My question relates to using a mesh which is not a scan but a 3D mesh modeled by a 3d artist.
In the paper, I can read: the authors provided us with a static mesh, to which we aligned the FLAME topology
.
But how can you align the FLAME topology over a 3D model?
You also mention: Then you need to undo the effects of pose
A handmade 3D model should already by in a neutral pose, from my understanding it just need to have it's transform reset so that the mesh position overlays one of the FLAME ply meshes.
I think I can do it using Blender.
(fyi, my skill set is more about 3d animation then ML.)
from voca.
Has anyone got any hints regarding how to register a 3D mesh, modeled by a 3d artist, to FLAME topology?
IE a mesh purchased on TurboSquid.
I am sorry, I fail to find instructions in the paper.
from voca.
Thank you, please check out Section 4.2 of the FLAME paper on how to align a general scan that is not in FLAME topology. The basic idea is to replace the vertex to vertex distance from the fit_3D_mesh demo with a general point to surface distance and minimize this instead. This is often referred to non-rigid ICP.
Another option could be using the provided FLAME Blender plugin to output a mesh in FLAME topology that then can be animated by VOCA.
from voca.
We also just added some demo code to fit FLAME to a scan of arbitrary topology in the flame-fitting repo.
from voca.
Related Issues (20)
- Windows support and installing MPI-IS / mesh HOT 1
- Should --uv_template_fname be the same as --template_fname in run_voca.py?
- can you give pretrained_models?
- Initialization Of Decoder Layer
- Problem of training voca
- How to control expression in the edit_sequence.py? HOT 2
- Can I train this on custom dataset?
- Unknown mesh file format. HOT 1
- Training with new Tensorflow Version
- Unsolved reference tfbody
- Missing data on subj_seq_to_idx.pkl file HOT 1
- If I want to control the expression of the eyes, how should I set the parameters?
- I haven't found the 'output_graph.pb' file, where can I get it?
- dataset/voca_face_former/templates.pkl not found HOT 1
- Where to download npy and pkl files HOT 1
- Inference other .ply files
- Hot to get blink_exp_betas for another flame version?
- Is the the vertice's coordinates in registered data the same as it in data_verts.npy in training data? HOT 1
- How extract FLAME parameters (expression, jaw) from vertex offset. HOT 1
- After add eyeblink, the face twitches HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from voca.