keijiro/skinner

Special Effects with Skinned Mesh in Unity

Skinner

Skinner is a collection of special effects that use vertices of an animating skinned mesh as emitting points. It uses a special replacement shader to convert vertex positions into GPU-friendly data, and thus it avoids spending extra memory and CPU time for handling them (uses GPU resources instead).

Skinner Asset Types

Skinner provides some special asset types to preprocess relevant data.

Skinner Model

A Skinner model is a simplified variant of a mesh asset that only has vertices and skin weights. When converted from an original mesh, all the topological data of triangles is removed and overlapped vertices are stripped out.

Skinner Template

A Skinner template is a pre-built mesh asset that provides vertices and topological data to the effect renderers. For instance, a Skinner particle template has thousands of particle instances that are placed at the world origin; A Skinner particle renderer will move them at run time.

Skinner Component Classes

Skinner also provides some component classes to handle these assets during run time.

Skinner Source

The Skinner source component is a subsystem that converts a deformed skinned mesh into baked data. This data will be provided to multiple Skinner renderers.

Skinner Renderers

The Skinner renderer components are special types of mesh renderers that deform a Skinner template based on data provided from a Skinner source. Then it creates some interesting special effects.

How to Set Up

Install the package.

Download one of the unitypackage files from the Releases page and import it into a project.

Convert a skinned mesh into a Skinner model.

A skinned mesh has to be converted into a Skinner model in advance to be used in the Skinner system. This can be done from the context menu; click a mesh asset to select it, then choose Skinner -> Convert Mesh from the right click menu.

Note that the mesh asset is usually located inside an fbx file. A few extra clicks are needed to select it. See the GIF below.

Set up a character as usual.

Drag and drop a character prefab into the scene.

Attach a Skinner source to a skinned mesh renderer.

Add a Skinner source component to the game object that has a skinned mesh renderer component. Then set the Skinner model converted in the previous step to the model property.

This Skinner source will override the skinned mesh renderer and then use it to convert vertex data. Note that this character will disappear from the scene, because it will be exclusively used for vertex conversion. If it has to stay visible, another instance of the same character should be added to the scene as a substitution.

Create a Skinner renderer object.

Create an empty game object and add one of the Skinner renderer components to it. Then set the source property in it to refer to the Skinner source object created in the previous step.

For starters, it's recommended to use the Skinner debug component that simply visualizes the vertex data provided from the source. If it shows nothing, there may be something wrong in the previous steps.

Skinner Renderer Components

Currently, the Skinner package provides four types of renderers.

Skinner Debug

The Skinner debug renderer simply visualizes vertex data provided from a source.

This component doesn't need a template asset.

Skinner Glitch

The Skinner glitch renderer draws triangles between randomly choosing vertices in a source. Although the number of triangles is fixed (21,845 triangles), triangles with long edges or a large area will be pulled out to maintain the silhouette. This behavior can be controlled by the threshold properties of the component.

This component doesn't need a template asset.

Skinner Particle

The Skinner particle is a particle system that emits particles from vertices in a source. Several parameters (duration, rotation, etc) of each particle can be changed according to the speed of vertices, and thus it can be used to give some emphasis to character movement and trajectory.

The shapes of particles are defined in a Skinner particle template asset. Any shape can be used in a template, but it's recommended to use meshes with the very low poly count because the number of particle instances is determined from the number of vertices in the shapes (low poly == more particles!).

Skinner Trail

The Skinner trail renderer draws trail lines from vertices in a source. The width of the lines can be changed according to the speed of vertices, and thus it can be used to give emphasis to movement too.

The length of the trail lines is pre-determined in a Skinner trail template asset. The number of lines is also pre-determined by its length. The longer the lines are, the fewer lines are drawn.

Compatibility

At the moment Skinner is only tested on Windows, macOS and iOS (metal). Possibly it runs on PS4 and Xboxone, but not sure about GLES3 and WebGL.

License

MIT

skinner's Issues

How to use Skinner with URP pipeline？

Hi

First of all, I must say that the effects you have achieved so far are amazing. I sincerely thank you for your contribution.

With the increasing requirements for image rendering, more projects use the URP pipeline. How can Skinner be used under the URP pipeline? Do you have any suggestions or methods?

Looking forward to your reply.

Running on iPhone 5s

I am getting the following issue on building for iPhone 5s:

This set of render targets requires 24 bytes of pixel storage. This device supports 16 bytes.

It does work on iPhone 6, so I am a bit puzzled. Could it be that the GPU is too weak on the 5s?

Components

are you working on any shaders that can take a singular point to draw a trail? since trails from the LineRenderer are probably one of the most common things to have in a Unity Game. (this way you can have Trails on any object. and not just things that are skinned)

Cloth component support

Although I'm not sure if it's possible, I note it for future reference.

Glitchy Shaders when instancing at run-time

if you create a prefab that holds your trail object and the actual object itself into aa container. then spawn those two containers it seems as though the shader is running both Synchronously as I see it jump back and forth between the moving objects. https://youtu.be/Lvn_YEMjVCo

LWRP Support?

Im wondering if its working on LWRP, or any plans?

Shader sometimes doesnt draw

Hi, thanks for your great assets.

I am having issues with shader drawing: sometimes, randomly, it disapears for no apparent reason and after appears again. It keeps drawing in scene window and shadows are also rendering in game window.

Do you know why it happens?

If I create in a new scene or in demo scene it works... Maybe it is some config that I haven't seen.

Thank you in advance

Can't convert

I'm trying to test Skinner with Robot Lab https://www.assetstore.unity3d.com/en/#!/content/7006 but can't convert meshes:

The given mesh (prop_floorBot_female) is not skinned. Skinner only can handle skinned meshes.
UnityEngine.Debug:LogError(Object)
Skinner.SkinnerModelEditor:CheckSkinned(Mesh) (at Assets/Skinner/Editor/SkinnerModelEditor.cs:34)
Skinner.SkinnerModelEditor:ConvertAssets() (at Assets/Skinner/Editor/SkinnerModelEditor.cs:54)

What is a skinned mesh?

PS: it works with for example the robot legs in https://www.assetstore.unity3d.com/en/#!/content/75207 - In case somebody else is just trying to just see the results for themselves first. :)

Constant updated mesh created programatically (ER)

Hello kei,

Awesome work as usual 👍 , i wonder if i can apply this to a mesh created programatically from a pointcloud that its updated and reacreated on every frame.

The plug in requires to point an already creatd mesh but this is created at runtime.

Changing simulation space to world

We are trying to achieve glitch effect with gpu particles system in built-in and you have already made that effect but the issue it has that system is always in local simulation space.
Is there a way to change the simulation space to world?

URP support

Hey there,

Congrats for your work! I love this shader, but unfortunatelly it's not supported with the URP.
Do you have any plans to port it soon to Shader Graph?

Thanks!

render target error on tvOS

I tried to build for tvOS, and get this error:

[MTLRenderPipelineDescriptorInternal validateWithDevice:], line 1659: error 'This set of render targets requires 48 bytes of pixel storage. This device supports 32 bytes.'

I was wondering if there is a way to get this to compile on tvOS, or if tvOS is unsupported.

HDRP conversion

Hi,
First, this is an amazing effect, @keijiro . I've tried to upgrade the project to HDRP but the materials doesn't convert properly. Is there something I can make to fix it?
Thx

Compatibility with HD Render Pipeline?

Seem that the shaders are not working in the new HD Render Pipeline. Is there any way of making it works?

skinner targeting using FinalIK

Hi Keijiro, can you think of any reason why the Skinner mesh would target the particles correctly when animating using the animation controller, but not when using a bone retargeting plugin like Final IK? Basically what happens is the particles will only spawn from a static T pose of the model when using Final IK ( and VR, so VRIK) The root postition does translate, just not the full bone transformaitons and rotations. I've been working on an application to utilise your effects on a first person model in VR but this has got me stumped. I'm really not very good at programming though so feel free to tell me you cant help :)

thanks!
Josh

Other Avatars support

I tried to integrate this with other avatas but i was not succesful.
Do we have any reason as 2 why you use 2 avatars to render this?

Accessing vertices positions from specific body regions?

Hi @ keijiro

I was wondering how could I access the skinned mesh vertices using your project.
I would like to know how can I access a list of vertices so then I can select random vertices from that list. Then I would like to grow a plant or flowers at random locations.

Is this possible with your setup or would it be overkill?

Thank you for your time and for sharing all your beautiful work?
Guillermo

Not working in Unity 2017 or 5.6 w/ VR enabled

Can't get the skinner renderers - Glitch, Particle or Trail - to work in 2017 or 5.6 with VR enabled in player settings.

When VR is not enabled, these renderers work fine in 2017.

Skinner also works in 5.5 and 5.4.3 with VR enabled.

Could you upload a test scene?

This package isn't working in my project, and I'm not sure if it has to do with how Unity changed their shader pipeline, because I'm not able to get the render to show up at all.

Could you upload a test scene so I could compare how it's working to my issues?

Thanks,
Caroline

Skinner doesn't work after building when used with SteamVR.

We are having problems where a unity build crashes when we use Skinner in conjunction with SteamVR. It works fine when we run in the editor - and Skinner renders well in VR.

45% memory in use. 8134 MB physical memory [4416 MB free]. 9414 MB paging file [4150 MB free]. 134217728 MB user address space [134216841 MB free]. Read from location 000043c0 caused an access violation.

We have also noticed when in editor performance starts out good then after a period of time drops drastically, this happens sooner when you have more Skinner objects, suspected memory leak.

Custom Particle Systems

Hello,

Is it possible to use a custom particle system for the skinner particle template? If so, how would I do it?I would like to create an effect with a character made out of water.

Convert Mesh shows IndexOutOfRangeException error

Converting mesh failed at a voxel mesh object.
error:

IndexOutOfRangeException: Array index is out of range.
Skinner.SkinnerModel.Initialize (UnityEngine.Mesh source) (at Assets/Plugins/Skinner/SkinnerModel.cs:57)
Skinner.SkinnerModelEditor.ConvertAssets () (at Assets/Plugins/Skinner/Editor/SkinnerModelEditor.cs:62)

target mesh:
TanukiAdult_mesh_skinner.asset.zip

Skinner Not working with Unity 5.6 and VR.

Running the Repo project and enabling VR breaks Skinner. Nothing Renders?

I'm sure it has worked before?

Skinner Renderers had to be at 0;0;0

...or they will not work as expected.

Not sure if this is to be considered bug (and I can close issue if not), but if renderers are not at global 0, they will start count from their point of origin and rendering far off from skinned mesh. Spent 10 hours trying to figure what's going on, because I'm not very good at programming.

(moving guy's on the left trail is so far away I thought it's not drawing)

Ofc, it's all kinda OK but some warning or instruction could save a lot of nerve cells for some idiot like me. =^_^=
We need them xD

Also, thank you very much for this VFX marvel. Domo arigato =)

iPhoneX test

I did build test on iPhoneX but no matter how I change the parameters it only show trail on it's hand and it even stop emitting middle of play.
do you know what happen? or where to go fix it?

Not running in Unity 5.6 with SteamVR

When I run the Test scene with Virtual Reality supported and SteamVR launched, I get absolutely nothing rendering from the SkinnerRenderers (not even the Debug one). It seems that there's some kind of basic incompatibility with your project and SteamVR.

There's no crashing, no error -- just absolutely nothing renders.

Would it be possible to modify this to work with a mesh using blend shapes?

Only debug working in 2019.2. Looking for workarounds, can I connect this to Shader Graph?

I was happy to see that at least SkinnerDebug.cs is working in 2019.2, probably because its the only effect script that runs on the CPU?

Can this data be fed into Shader Graph?

How could I get some kind of effect working using only SkinnerDebug, since it sounds like this will not be updated any time soon?

Not working in Universal Render Pipeline

I can only get the shaders to work in standard 3D projects, but not the universal render pipeline. Is it possible to fix this?

Possible to map a mocap suit controller to your avatar?

I woukd like to try implement this project for a live performance using the perception neuron mocap suit, do you know if that would be possible with the project in its current state? I managed to get it to a. Point where it somewhat works, but the mapping of the particles and effects in respect to the animation is not correct.
P. S. I love your work and have been playing around with your projects for a while, modifying them and using them in VR on the HTC vive - so so beautiful!

Effect jumping between Skinner sources

Hey @keijiro, hoping you can shed some light on this issue. The rendered effect seems to be jumping between Skinner sources. Here's a simple project that demonstrates the issue:
SkinnerBugDemo.zip