belinghy/motionvaes-webgl

An interactive webgl demo for real-time character animation and control using Motion VAEs [SIGGRAPH 2020]

motionvaes-webgl's Issues

How to get mocap.npz?

Hi, thanks for your really awesome work. Now, i want to use the mvae model to drive the virtual character. Unfortually, when i use the 6D joint rotation information, the virtual character cannot act correctly. I need more detail of how to generate the mocap.nzp to use the 6D joint rotation information correctly.
Could you please offer more detail of how to generate the mocap.nzp dataset with corresponding python script?

About joint positions, velocity and orientations

Hi, thanks for your really awesome work. I have one simple question regarding to the implementation. In the paper, the input and output of VAE both contains joint positions, velocity and orientations (or angles).

However, if I'm not wrong, the joint positions and orientations are interchangable, which means they could be transformed to each other with some formulas. The same happens to velocities and positions. If so, why should we generate them both? I'm also not sure if the positions, velocity and oritentations are generated in parallel, or the velocities and orientations are extracted from the generated positions.

Thanks you in advance. Look forward to your reply.

possibility to use glb rigged Avatars

Hey,

this is awesome work! do you think its possible to use a rigged SkinnedMesh Avatars Bones?
like the Avatar are created on https://fullbody.readyplayer.me/ the result glb has the rigged bones

How to drive the character with 6D joint orientations?

Sorry to open a new issue. And, i still have two problems.
Firstly, could you please provide more details about how to generate root delta x, delta y, delta facing?

Secondly, could you please show how to drive the character with 6D joint orientations rather than positions? Actually, i have tried to drive the character with the 6D joint orientations for weeks, but cannot get correct results. With the 6D joint orientations, i perform a series of operations, such as denormalization, convert to local space, convert to rotation matrix and Euler angle. Could you please show me how to use the 6D joint orientations correctly?

About 6D joint orientations

Hi, sorry for opening a new issue. But I'm quite interested in your work. In the paper, you mentioned "Joint orientations (𝑗 𝑟 ∈ R 6 ) are represented using their forward and upward vectors, which avoids the problems of angle-based representations."

Would you mind being a little more specific? I checked the referred literature, but there's not many details. Is it the proposed rotation represenation in "On the Continuity of Rotation Representations in Neural Networks", which is exactly the first two columns of the rotation matrix? Or I would appreciate it very much if you could refer some materials or code I could look in.

Thanks in advance.