kosrud/dq-skinning-for-unity

Dual quaternion skinning for Unity

Features

• GPU skinning with compute shaders
• blend shape support (calculations performed in compute shader)
• works with any platform that supports compute shaders
• preserves volume with deformation (look comparison)
• zero GC allocations per frame
• original bulging compensation method

Comparison DQ vs built-in linear

Gif	Difference

Bulging compensation demo

Unity version

The script was tested with following Unity versions:

• 2020.1.0a13.1443 (earlier versions do not support #pragma multi_compile in compute shaders)
• 2020.2.0f1 (briefly tested - seems to work just fine)

How to set up

• Create a skinned character with SkinnedMeshRenderer component
• Add DualQuaternionSkinner.cs component (it will require a MeshFilter component)
• Enable mesh Read/Write in import settings
  

• All materials of the mesh should use a special shader to apply vertex positions. The shader is MadCake/Material/Standard hacked for DQ skinning
• In order for bulging compensation to work properly, all the character's bones must have one local axis (consistent for all bones) aligned with the limb controlled by the corresponding bone.

Bones properly aligned	Bones not aligned

• Whichever local axis of the bones was aligned with the limbs must be selected in the script gui:

If bulging is increased instead of decreased, select same axis with different direction (X => Negative X)

Common problems

The script is programmed to automatically detect common setup problems. Check out the editor:

Warning

You will not see any effect in edit mode.
The scipt only works in play mode.
If you see no effect in play mode verify that you are using the right shader.

Why do i need SkinnedMeshRenderer?

My scripts uses SkinnedMeshRenderer to extract an array of bones from it. Yep, that's it.
The order of bones is unpredictable and does not depend on their hierarchy.
Only SkinnedMeshRenderer knows it    ¯\_(ツ)_/¯

After extracting the bone array in Start() my script disables SkinnedMeshRenderer component as it is no longer needed. All the animations are performed by the script. You can verify it in the editor after hitting play button.

How do I use custom shaders?

Alas it's complicated.
I added comments to "Standard hacked for DQ skinning" marking the alterations i made to the Standard shader.
You can try to do the same with your own shader to make it work with the script.

Feel free to contact me in this thread at unity forum if you need help.

I would also like to hear about your projects that use my script and your experience with it.

Known bugs

Must use cullingMode = AlwaysAnimate in Animator. Otherwise, the mesh is treated as permanently invisible.

You can write a short script that will toggle cullingMode based on visibility to get proper culling.

Performance

During my testing the amount of time spent on actual skinning was negligible compared to the amount of time extracting localToWorldMatrix from every bone in the hierarchy.

As long as you are not creating hundreds of characters with complex rigs (no matter the polycount) there should be no significant performance hit.

If anyone knows how to optimize extracting localToWorldMatrix of the bones please create an issue or message me on unity forum.

Works A LOT faster with IL2CPP, about 30% slower than built-in skinning in worst-case scenario according to my testing.

API

Documentation

Future plans

• Reduce numthreads for compatibility
• Add instancing support
• Allow controlling the strength of bulging compensation with a texture
• Implement simple runtime switching between DualQuaternionSkinned and built-in SkinnedMeshRenderer
• Implement proper animation culling (see known bugs)
• It might make sense to group the data from all instances of the script into one batch and run the compute shaders only once per frame regardless of how many animated characters you have

Discussion

If you have any questions, ideas, bug reports, or just want to discuss the script, you can contact me on unity forum

Bulging compensation method

The bulging compensation method used is described in the paper.

Notes:

• In formula (6) there is a mistake, x should be replaced by w_2 (there is no variable named x)
• Since publishing the article I found a better polynomial instead of (9), the coefficients are as follows: 2.2; -9.6; 10.4
• In formula (12) there are weird brackets around V_bisector. This is a software bug, they should be ignored. This happens multiple times later in the article.
• The stretched region of the model in Fig (7), that looks weird even after applying the fix (though is no longer jagged), is actually caused by using improperly prepared armature. The spine and the shoulder have different local axes aligned along the joint. This contradicts one of the requirements listed at the top of page 17.

dq-skinning-for-unity's Issues

Use RenderTexture.DiscardContents()

https://docs.unity3d.com/ScriptReference/RenderTexture.DiscardContents.html

Extra ping-pong buffer unnecessary?

It looks like the code would work just fine with only 2 ping-pong buffers.

DQ-skinning-for-Unity/Code/DQ skinning/DualQuaternionSkinner.cs

Lines 537 to 539 in 9d096e2

	bufSource = bufTemp_1;
	bufTemp_1 = bufTemp_2;
	bufTemp_2 = bufSource;

Sample vertex position texture only once

Apply all blend shapes in a single run of compute shader

Implement toggling between DQ and SkinnedMeshRenderer in Play mode

is workaround still needed?

DQ-skinning-for-Unity/Code/DQ skinning/DualQuaternionSkinner.cs

Lines 575 to 577 in 9d096e2

	this.shaderComputeBoneDQ = (ComputeShader)Instantiate(this.shaderComputeBoneDQ); // bug workaround
	this.shaderDQBlend = (ComputeShader)Instantiate(this.shaderDQBlend); // bug workaround
	this.shaderApplyMorph = (ComputeShader)Instantiate(this.shaderApplyMorph); // bug workaround

More information should be provided on the bug, investigate if the workaround is still needed.

Misspelled warning

https://github.com/ConstantineRudenko/DQ-skinning-for-Unity/blob/d519c5e7bbbbaad38064cb6fdd3224c30a03d922/Code/DQ%20skinning/DualQuaternionSkinner.cs#L223-L225

"blendshapes ara available" - misspelled

use $@ prefix (verbatim + string interpolation) to remove the need for operator+

Shadows are displayed incorrectly with viewFrustrumCulling = true

https://github.com/ConstantineRudenko/DQ-skinning-for-Unity/blob/d519c5e7bbbbaad38064cb6fdd3224c30a03d922/Code/DQ%20skinning/DualQuaternionSkinner.cs#L192

Null reference

Line 24: smr is null

DQ-skinning-for-Unity/Code/AssetPostProcessorReorderBones.cs

Lines 19 to 24 in 9d096e2

	void Process(GameObject g)
	{
	SkinnedMeshRenderer smr = g.GetComponentInChildren<SkinnedMeshRenderer>();
	if (smr == null)
	{
	Debug.LogWarning("Unable to find Renderer" + smr.name);

Get the mesh of DQ blending skinning

I have implemented the dqbs in my own project. And I want to get the mesh of human model of dqbs in obj format. However, I found i get the obj model of lbs. How can i get the mesh of dqbs? Thank you.

boneOrientationVector should be private

DQ-skinning-for-Unity/Code/DQ skinning/DualQuaternionSkinner.cs

Lines 13 to 17 in 9d096e2

	/// <summary>
	/// Bone orientation is required for bulge-compensation.<br>
	/// Do not set directly, use custom editor instead.
	/// </summary>
	public Vector3 boneOrientationVector = Vector3.up;

Use private field with SerializeField and public getter/setter functions. Do not use property, use functions to emphasize that changing the value requires additional internal logic and can be expensive.

Logic for setter:

DQ-skinning-for-Unity/Code/DQ skinning/Inspector/DualQuaternionSkinnerEditor.cs

Lines 83 to 88 in 9d096e2

	var newOrientation = (BoneOrientation)EditorGUILayout.EnumPopup("Bone orientation: ", currentOrientation);
	if (this.dqs.boneOrientationVector != this.boneOrientationVectors[newOrientation])
	{
	this.dqs.boneOrientationVector = this.boneOrientationVectors[newOrientation];
	this.dqs.UpdatePerVertexCompensationCoef();
	}

Implement per-vertex masking of bulging compensation

Through vertex colors or some other means

Is AlwaysAnimate needed on Animator?

Bounding box is taken here:

DQ-skinning-for-Unity/Code/DQ skinning/DualQuaternionSkinner.cs

Line 190 in 9d096e2

this.mf.sharedMesh.bounds = this.smr.localBounds;

Should be possible to use proper culling?

Missed camel case

https://github.com/ConstantineRudenko/DQ-skinning-for-Unity/blob/d519c5e7bbbbaad38064cb6fdd3224c30a03d922/Code/DQ%20skinning/DualQuaternionSkinner.cs#L167

viewFrustrumculling => viewFrustrumCulling

Character exported from Reallusion's 3DXchange does not animate properly

https://forum.unity.com/threads/dual-quaternion-skinning-for-unity.501245/page-3#post-5906549