CrowdMorph - GPU animation package

[fmcdeoliveira]

I remember when UniteAustinTechnicalPresentation was released, there wasn’t any official animation package at the time, I was really interested in DOTS and the ideia of a pure dots project was in my mind, so I decided to create my on animation package, I’ve working on this project for a long time and I’m pretty excited to share my progress.

The package received a lot of design changes, in a certain point the official animation package released and I thought the wasn’t fair to develop a package for replacing the official one, so I’d redirected the focus of the package to cover something more specific where my package could fit better than the unity package in some situantions. Well, what could I develop that the official package wasn’t providing as a feature? GPU Animation! the Hybrid Rendering supports GPU Skinning but the animation package do everthing on the CPU then uploads to GPU, so what if it animate the skeletons in the GPU? this ideia came with a lot of limitations but at the same time there’s room for heavy optimization.

So the purpose of the package is to be used for large amount of animated entities where there’s no requirement to read the bones position on CPU (as all the animating is done on GPU, the skeleton matrices are never read back to CPU for performance reasons). I tried to convert most of the features from the built-in Animator to the package, so developers that wanna convert their project from authoring to pure dots can do it as simple as possible keeping the visual fidelity.

Performance

spicyspicycomet

1000 GameObjects (w/ AnimatorUtility.OptimizeTransformHierarchy):
~30fps @ i7-10700, 16GB, RTX 2060

20000 Entities:
~90fps @ i7-10700, 16GB, RTX 2060

Features

I think it’s easy to explains the features by mentioning what it’s not supported:

• Humanoid animations are not supported, they all should be Generic;
• Transition Interruption doesn’t produce the same visual result as the built-in animator;
• Only bones can be animated, there’s no support for generic float/int values;
• As animating happens on GPU, it’s not possible to read bones position on CPU; (it’s possible to access bone transformation from shader, visually it’s possible to attach objects/effects to bones)
• BlendTree aren’t supported yet, I’m trying to find a way to produce the same result as the authoring version;
• AvatarMask only works using a specific avatar (that you should import in the editor, and select the actived bones);
• Layer “IK Pass”, “Sync” and “Timing” have no effect on the converted version;
• AnimatorControllerOverride isn’t supported yet, but everything is ready to implement it, probably the next feature I’m gonna add;
• As humanoid avatar are not supported, mirroring animations aren’t gonna work;
• “Solo” transitions are ignored;
• Live Link isn’t supported, but I really think that it’s possible to create a proxy gameobject to replicate the animator state of the selected entity;
• SkinnedMeshRenderer bounds doesn’t work as expected, they should be update to work properly. Unity compute the world bounds by using the position of the SMR root bone and this is not available on CPU for entities using my package, so they will not be updated, it’s necesssary to resize they to overlap the area that the SMR can reach when being animated;
• RootMotion not available;
• Animation Event are basically a NativeMultiHashMap that stores all the events produced in the frame, group by event name hash, an event stores the entity that raised it and a Integer or Float value that was assigned in the AnimationClip inspector.

Conversion Settings

To convert a GameObject to an animated entity, it’s required to replace the UnityEngine.Animator component to the CrowdMorph.Animator, notice in the image below that CrowdMorph.Animator has a property to a “Parameters Component Type”, this is a ComponentData type that holds the parameters used by the animator, the parameter names should match field names from the parameters component type.

As a component type holds the parameters, it’s possible to set the parameters using a ComponentSystem. For example, an equivalent system from a MonoBehaviour that set parameters would look like this:

Finally, a Skeleton component should be added, same thing as a Rig component from Unity.Animation, except it only have skeleton settings.

[DreamingImLatios]

Did you intend to share a link?

@fmcdeoliveira

SkinnedMeshRenderer bounds doesn’t work as expected, they should be update to work properly. Unity compute the world bounds by using the position of the SMR root bone and this is not available on CPU for entities using my package, so they will not be updated, it’s necesssary to resize they to overlap the area that the SMR can reach when being animated;

This is something I can help you make automatic if you are interested. The mesh and pose are separable in computing a bounds and each can be generalized across a domain of transformations.

I’m also curious what kind of compatibility you allow for regarding shaders and skinning gpu buffers?

[fmcdeoliveira]

@DreamingImLatios

This is something I can help you make automatic if you are interested. The mesh and pose are separable in computing a bounds and each can be generalized across a domain of transformations.

This is actually impossible, the skinned mesh from my package are rendered by Hybrid Rendering and the occlusion occurs in the CPU, so it’s not possible access the root bone transformations.

@DreamingImLatios

I’m also curious what kind of compatibility you allow for regarding shaders and skinning gpu buffers?

To support GPU Skinning I had to do a workaround because shader graph doesn’t have a node to access BLENDWEIGHTS and BLENDINDICES, I’m using the Linear Blend Skinning node and a custom function node that define some macros to override “Linear Blend Skinning” function and in the new function I access my SkinMatrices buffer (named _CrowdMorphSkinMatrices) and I use another property to store the SkinMatrixBuffer. The package provides a SubGraph for “Linear Blend Skinning”

[DreamingImLatios]

@fmcdeoliveira

This is actually impossible, the skinned mesh from my package are rendered by Hybrid Rendering and the occlusion occurs in the CPU, so it’s not possible access the root bone transformations.

All you need is the bounding box of all bones across all animations for a given GPU character and the max distance of any vertex to any bound bone (both of these can be computed during conversion) and this becomes your local bounds for your mesh.

@fmcdeoliveira

To support GPU Skinning I had to do a workaround because shader graph doesn’t have a node to access BLENDWEIGHTS and BLENDINDICES, I’m using the Linear Blend Skinning node and a custom function node that define some macros to override “Linear Blend Skinning” function and in the new function I access my SkinMatrices buffer (named _CrowdMorphSkinMatrices) and I use another property to store the SkinMatrixBuffer. The package provides a SubGraph for “Linear Blend Skinning”

That solution sounds like it will be compatible used alongside other animation solutions. Nice!

[fmcdeoliveira]

@DreamingImLatios

All you need is the bounding box of all bones across all animations for a given GPU character and the max distance of any vertex to any bound bone (both of these can be computed during conversion) and this becomes your local bounds for your mesh.

I had read you post and started to think about a solution to do this in GameObjectConversionSystem and I came with the code below:

            var skinnedMeshToRootBoneMatrix = skeleton.RootBone.worldToLocalMatrix * smr.rootBone.localToWorldMatrix;
            var bounds = (MinMaxAABB) new AABB
            {
               Center = skinnedMeshToRootBoneMatrix.MultiplyPoint3x4(smr.localBounds.center),
               Extents = skinnedMeshToRootBoneMatrix.MultiplyVector(smr.localBounds.size) / 2,
            };

            var clips = new List<AnimationClip>();
            foreach (var clipSources in smr.GetComponentsInParent<IAnimationClipSource>()) // CrowdMorph.Animator implements this interface
               clipSources.GetAnimationClips(clips);

            var skeletonRootWorldToLocal = skeleton.RootBone.worldToLocalMatrix;

            AnimationMode.StartAnimationMode();
            AnimationMode.BeginSampling();

            var skeletonGameObject = skeleton.RootBone.gameObject;
            foreach (var clip in clips)
            {
               int frameCount = (int)math.ceil(clip.frameRate * clip.length);
               for (int i = 0; i <= frameCount; i++)
               {
                  AnimationMode.SampleAnimationClip(skeletonGameObject, clip, i * clip.frameRate);
                  var rootToBone = skeletonRootWorldToLocal * smr.rootBone.localToWorldMatrix;

                  bounds.Encapsulate(new AABB
                  {
                     Center = rootToBone.MultiplyPoint3x4(smr.localBounds.center),
                     Extents = rootToBone.MultiplyVector(smr.localBounds.size) / 2,
                  });
               }
            }

            AnimationMode.EndSampling();
            AnimationMode.StopAnimationMode();

I’ve never used AnimationMode API so I don’t know why but it’s not worknig, I’ll try to do something else later, this was the fastest thing I could make today. Now I got you, I really like this idea of precomputing the bounds in conversion, so probably I gonna implement it. thx for the idea

[mochenghh]

Please put a preview version, Its not neccessary to finish everything and offer it once totally

[fmcdeoliveira]

@mochenghh

Please put a preview version, Its not neccessary to finish everything and offer it once totally

I’m gonna try to launch a preview in an week

[mochenghh]

@fmcdeoliveira

I’m gonna try to launch a preview in an week

really appreciate it, Humanoid animations in dots would be the most important part, I hope IK would be developer friendly, just like dots physics, can be modified both in editer and code

[mochenghh]

@fmcdeoliveira

I’m gonna try to launch a preview in an week

Unity Dots Team said would put a new version ecs 0.50 in the first quarter,and said 0.50 would be attach to production level, I guess Dots Animation of Humanoid animations would be a part of it. But there is no any message or update or reply about 0.50 version.

[Antypodish]

@mochenghh

Unity Dots Team said would put a new version ecs 0.50 in the first quarter,and said 0.50 would be attach to production level, I guess Dots Animation of Humanoid animations would be a part of it. But there is no any message or update or reply about 0.50 version.

Personally I am not holding a breath for 0.5 in first quarter. I won’t be surprised if will be pushed back.

There is no official confirmation, so you should not guess, nor expect. Most likely Unity Animation will be released in earliest, after 1.0 is released. Not with it. And that may be equally 1 to even 2 years or more. So I suggest, look for alternative DOTS animation solution until then, so you won’t be disappointed.

[mochenghh]

Im now using build-in Animator, But its really inconvient to conbine with monobehavior，make code looking so complex.
entities 0.50 looks 99% push back,you are right,It really grow my negative emotion. I really wonna do something rather than wait

[mochenghh]

@fmcdeoliveira

I’m gonna try to launch a preview in an week

where I can download preview version

[fmcdeoliveira]

@mochenghh

where I can download preview version

https://github.com/felipemcoliveira/com.felipemcoliveira.crowdmorph

sorry, but it probably is very buggy and I only test it in unity 2021.1. tomorrow I’m gonna record a video explaning how to use it

[mochenghh]

@fmcdeoliveira

https://github.com/felipemcoliveira/com.felipemcoliveira.crowdmorph

sorry, but it probably is very buggy and I only test it in unity 2021.1. tomorrow I’m gonna record a video explaning how to use it

really appreciate it，waitting your doc

[fmcdeoliveira]

@ I’ve just notice that I’d included anything about the skinning support for shader graph in the package, fixing it (some other stuff) and creating a sample for the package. i’ll try to commit it asap, probably tonight and maybe a video (w/ very poor english )

[ThynkTekStudio]

hi, is this thread still alive?

[Guedez]

@fmcdeoliveira

Only bones can be animated, there’s no support for generic float/int values;

If your stuff is based off https://blog.unity.com/technology/animation-instancing-instancing-for-skinnedmeshrenderer like mine is, you can add 16 whole float values to be accessible by the shader by adding an extra bone to every animation.
This lets me do stuff like tell the shader ‘how far into the total animation’ the plant is, and with it I can do stuff like progressively change the alpha cutout of a texture to make vines appear even though the total animation had 6 different animation clips.

Also, not the place to ask this, but it’s about time this forum get on with the times and enables webm uploads.
https://imgur.com/a/Vx55jp5

[fmcdeoliveira]

Hi guys, long time since I created the thread.

I’m a little busy and currently not working on the documentation. But now, there’s a sample that can be imported using the package manager window.

@Guedez

If your stuff is based off https://blog.unity.com/technology/animation-instancing-instancing-for-skinnedmeshrenderer like mine is, you can add 16 whole float values to be accessible by the shader by adding an extra bone to every animation.
This lets me do stuff like tell the shader ‘how far into the total animation’ the plant is, and with it I can do stuff like progressively change the alpha cutout of a texture to make vines appear even though the total animation had 6 different animation clips.

Also, not the place to ask this, but it’s about time this forum get on with the times and enables webm uploads.
https://imgur.com/a/Vx55jp5

It’s not, but you can write your own node to access _CrowdMorphSkinMatrices buffer and use matrix components of a bone as your doing. To know the bone index within the buffer, during the conversion get the skeleton bones (Skeleton.GetBones) find the index of the desired bone and store it in a component. At runtime, add the found index to SkeletonMatrixBufferIndex value to access the bone matrix in the buffer.
I guess doing so, you can achieve the same result you’re doing.

[ThynkTekStudio]

@fmcdeoliveira

Hi guys, long time since I created the thread.

I’m a little busy and currently not working on the documentation. But now, there’s a sample that can be imported using the package manager window.

It’s not, but you can write your own node to access _CrowdMorphSkinMatrices buffer and use matrix components of a bone as your doing. To know the bone index within the buffer, during the conversion get the skeleton bones (Skeleton.GetBones) find the index of the desired bone and store it in a component. At runtime, add the found index to SkeletonMatrixBufferIndex value to access the bone matrix in the buffer.
I guess doing so, you can achieve the same result you’re doing.

hi, is the sample using the new entities 0.51.0 version?

[fmcdeoliveira]

@ThynkTekStudio It didn’t, but I’ll try to update the package in 2-3 weeks, so it works with Entities 0.5x. And also, I’ve replied your issue on GitHub.