Well, this post will make a nice departure (and arrival) from my previous ones!
I think the correct term is WOO HOO!
So excited to be able to share this announcement with all of you, as we have all waited long enough.
While this post coincides with my regular quarterly updates, it really just serves as a follow-up to our first official announcement about the new animation system, which was shown at the Unite 2024 roadmap session on September 19th 2024.
Here is the link to the entire Roadmap presentation on Youtube. However I have started it at the animation section.
The roadmap session is a look at what we are currently working on, which we hope/expect/should appear in a future version of Unity. During this session, we took a look at the new animation system in more detail.
As the session is not a deep dive but more of a glimpse, I will attempt to put a little more colour into what was shown, as it will no doubt generate a ton of questions. I will also try to follow the presentation order as much as possible.
Note, the content used during the animation section of the roadmap presentation started out as the FPSSample, but was rebuilt, simply to test the new system internally. So it is not supposed to represent the end result of where it can go, but rather to simply illustrate some of the new capabilities; and in a way which might give a reasonable comparison to a known Mecanim project.
Three big things before we begin:
- I want to remind everyone that I can’t always respond to every question, especially if they are overly technical, or due to my not always having time, maybe double-secret probation stuff, or other constraints.
- These are highlights only. Expect more to come in the future.
- Even though we have shown some highlights, STUFF CAN CHANGE!
Note. Below I have included some direct quotes from the presentation in italics.
Right then, here we go.
Intro
In my previous post, I mentioned we needed to think far ahead. So a big focus for the new system is being prepared to handle massive animation demands for many years to come, no matter the team size. Smaller teams will want need to punch above their weight, and larger teams will demand a high degree of flexibility and customization.
Typically there is always at least one person who wears the hat of a Gameplay Programmer. They represent the design vision and write the gameplay code related to characters, cameras and controllers. They also support Technical Animators and Animators. The Technical Animator is most frequently the link between animators and gameplay programmers. Amongst other things, they oversee character creation, and create custom tooling to control the animation pipeline between the DCC and the engine.
Digging into these roles and what they need to be successful has been the basis for our initial design. If we try and please everyone for everything in the first iteration, we actually will end up pleasing no one. Sure, there will always be exceptions, and nuance in these roles for every production, but during our many interviews (yes, we actually did loads) we understood these roles to be the cornerstone of the animation pipeline.
Previewing
We will start with a really big change. You will see everywhere in the new system, that in just one-click, you can preview everything in a project, without having to load it into the scene first. This will reduce frustration and human error, as it only shows the compatible clips that the character can use.
This really is a big deal, but it sounds so innocuous. I am sure many of you at some point have struggled to know exactly what could be used where, and sometimes went bonkers as you probably ended up loading everything into one scene, as it was the only way to preview everything.
We added a kind of “scene within a scene” to enable a better previewing workflow. There will probably end up being some limitations, but man it opens up a world of experimentation. You will no longer bang your head on your desk as you try to navigate hundreds of clips to find compatible ones, especially when they’re all named “idle 001”.
For another preview example, check out the sockets section as well.
We are really excited about this new previewing workflow, and I am pretty sure you all will be too.
Skeleton Editing
In a similar vein as above, knowing exactly what is in your imported file (like an FBX) is not only a bit of a black art, but often rather fragile, as any changes you need to make requires breaking the link, since the file is read-only. This workflow really needed to be changed.
When bringing in your FBX file, I am pretty sure, the first thing you want to do is look at your skeleton hierarchy, then wanting or needing to make just some small edits, like renaming to your convention, disabling the face, or making larger edits like changing the default pose from T to A etc. Then if you make a change in your DCC, all that hard work was a major pain to redo or manage. Well, these changes will not only be simple to achieve, but are stored locally, making updates from the DCC far less painful.
Sockets
There is a new socket object which is used as a virtual transform. It no longer has to be fragile bones made in the DCC.
While we see this as a requirement for pretty much every 3D character, you can use sockets whenever you want to attach one object to another.
Because we have made it a dedicated object in Unity, we can make a bunch of assumptions and workflows around it. It does not get destroyed when updating the original character hierarchy, and so all of its position and orientation are kept, as well as any graph rigging setup (see below).
For anyone who has set up more than one character with more than one weapon, I am pretty sure you have at one time said “OH FFS, I just want to preview all my weapons with all my characters and animation in one place PLEEEEASE!” Instead, you end up loading everything all into one scene, yet again.
Well, that is a statement we never want you to have to utter again, as you will be able to preview AND edit the socket, ensuring the position and orientation are correct for all animations.
Clip Markup
The presentation video had to start somewhere, so maybe clip markup is slightly out of order, given you might add that before sockets, but anyway here it is.
In Mecanim, we currently force you to make permanent decisions when importing, while offering limited customization, basic previewing, all restricted to inspectors only.
It is slow with large data sets, limited to read-only clips, and adding animation events is via an inspector only. This is a major source of frustration for pretty much everyone, and well, that sucks.
So there will no longer be commitments up front when importing, as you will be able to preview everything, edit your clip, and add custom properties. You will be able to create and edit animation events using multiple tracks and range markup.
The multiple tracks and range markup is going to give you sooo much more control, especially when using it in the state machine. But it will also address some events not firing, as before they all used to be on the same frame, and if a frame was dropped…boom! Everything got screwed up.
Clip Markup should give you a truck-load more freedom in how you manage events, making gameplay much slicker for one.
Remapping
You will be able to remap anything to anything, independent of naming, hierarchy or proportions. It will even work offline and at runtime.
It will be a universal solution, fully customizable, supporting things like non-bipedal, fingers, tails, faces, backward legs, etc. You will be able to remap 1:1, 1:Many, Many:1, and it should all work with any custom constraint. All of which can be saved as a template.
Remapping covers the vast majority of use cases, though does not try to “retarget” things like IK constraints, which means you will have to manage the constraints yourself (see custom nodes later.)
Graphs and Tooling
Oh boy, this is probably the biggest shift for us, resulting in a serious investment in graphs in general and their tooling. Graphs are at the center of everything in the new animation system.
They are designed to give Technical Animators more creative freedom for all their gameplay implementation, without having to code. Graphs are approachable yet extensible, and will reduce production bottlenecks, as they ease the burden on programmers. They are built on a new graph stack, and can be packaged and reused.
A key point to understand is it will be a choice. You will be able to build what you need through code, or take advantage of the graph tooling, or a mix.
You will be able to set up, rig, debug (read more about visual debugging below) and reuse pretty much everything via a graph. Controlled by local and reusable blackboard variables, they can even feed into the State Machine, all without having to write a line of code.
Sub-graphs will be a great way to manage or package parts of a graph, and will for sure be exciting for our Asset Store creators. Sub-graphs really will be a powerful tool for creating modular and reusable animation logic. They promote organization and flexibility in the animation workflow. Much easier to copy and reuse sub-graphs, than it is a bunch of code. Oh, and you will be able to use these graphs on different characters, independent of the skeleton, but, more about this at another time.
These new graphs and tooling will come with a comprehensive node library, but most importantly, you will be able to create your own custom nodes. A get-out-of-jail-free-card if you will! These custom nodes will also use the same backend as the shipped animation nodes.
They are built on a new graph stack, which is an evolution of Graph Tools Foundation (GTF), with an optional runtime which is an evolution of Data Flow Graph (DFG).
This new graph stack will consist of an authoring and runtime layer. It will be multithreaded and should support horizontal/vertical layouts, debugging and live editing, flow graphs and state machines.
Okay now to veer off course for a bit. A follow-up question might be “What about GraphView?” Well, we are not touching it. It is still experimental and will be until the day it retires. We know it is still being used as there is no decent alternative from us, but hopefully, one day everyone will have migrated to the new system (including all of Unity), so there will no longer be a need for anyone to keep using GraphView.
I was going to do my Q3 update on just the graph stuff, but when it became clear we would be able to announce something at Unite, I thought to just wait, so I could include it here.
Hierarchical Layered State Machine
I went into some detail in my previous post about this, so it is nice to be able to show you a peek behind the curtains now.
The all-new State Machine is truly hierarchical, designed to scale even if you have thousands of animation sources. It’s highly organized, and much easier to read, refactor and reuse, with layers, blend spaces, subgraphs and integrated rig graphs, as well as constraints and math ops
You will be able to easily swap animation while keeping your orchestration logic intact.
It not only offers a serious level of control over your production but incredible performance at scale
Just so you don’t have to completely review the old post, I’ll do a bit of a summary here.
Designed for performance at scale. it’s truly hierarchical, easy to refactor and reuse orchestration logic.
When it comes to transitions, we will support multiple per frame. Transitions can be time/ratio/metadata based, and can be overridden.
There will be per-bone weight masking, instead of per layer. Inside the State Machine you will have math ops and integrated rig graphs, AND access to constraints in states.
This sounds like a small point, but will have a big impact. Constraints in states will allow creating custom rig setups per state. Meaning, in one state you could have a graph controlling the IK on the lower body, and have a completely different IK graph in another state controlling the upper body. These rig graphs can then be activated or deactivated using the state’s transition system.
The new State Machine has inverted exit rules which will make life so much easier, because every state no longer needs to know all the exit conditions for all other things, so you will be able to exit from the parent state. The new system will also support multiple animation sources that can contribute to the frame’s final rendered pose.
A few other bits and pieces. Easy re-connectability of nodes, and copy/paste with connections. There are no doubt a few other things I have missed, but we will cover them in time.
The big takeaway is, as we examined the Mecanim architecture over the years, there was simply no way we could do half this stuff without pretty much rewriting it. Which is why we made a clean break, and started preparing the new State Machine for a long future.
Visual Debugging
Okay, while it is difficult to choose from everything we have gone through so far, I guess you could say I saved the best until last. Eh, okay penultimate, but that really doesn’t roll off the tongue.
Everyone knows that detecting erroneous setups or defective gameplay behaviors in an animation system are a huge time-consuming aspect of game development. So quickly identifying the source of these problems offers tremendous value for anyone in the animation pipeline.
Pretty much everything in the new animation system will be debuggable.
It starts with the animation “Recorder”. Anyone on your team can record then rewind animation game data and step through frame by frame. Not only step through, but step into. Whether a graph or the State Machine. You’ll be able to debug events, transitions, blackboard, inspectors, layers, and masks. Plus select/isolate in real-time as you navigate complex systems.
Sure it will help you accurately pinpoint the issues, but it will also mean you can iterate faster, identify bottlenecks early, resulting in higher fidelity gameplay.
We are working on live visual graph debugging, with in-game recording and replay.
Making it easy to navigate the source of any animation problem, no matter how complex the setup.
You can rewind game data and step through frame by frame.
Not only will it ensure you ship fewer In-game bugs, but also means you can afford to make large changes, even late into your production.
That last thing right there…is going to be massive! By making graphs and the state machine far more robust and easy to refactor, when you identify the problem you will have the confidence to make changes, even big changes, instead of avoiding the issue, praying it will not fall apart.
Pipeline and Performance
Knight by Dmitriy Poskrebyshev Castle by Astrofish Games
The new animation system is designed with performance in mind.
It is built on ECS, and equally works with GameObjects.
Written in C# and HPC#, all animation runtime code is burst compatible, meaning high performance native execution. You will have control over execution order, and customizable phases, and all callbacks are deferred.
The ECS/GameObject flexibility is not to be overlooked. Even when you are using GameObjects we are still using ECS under the hood. So you can take advantage of the best of both worlds as you see fit, without having to make any decisions up front.
For instance Unity Entities/Baking animation runtime code is backed by Entities that offers optimized runtime memory layout for memory caching and data streaming, authored assets and runtime data. The new system can take decisions for optimization at runtime without impacting the authoring data, offering a more lean data representation at runtime, by optimizing transform marshaling inside a frame.
Some other stuff which might fly under the radar:
We have removed some old limitations such as sync point bottlenecks, not having deferred callbacks, and have removed the rebinding overheads when changing animations at runtime. This last one was simply not possible to workaround, and was such a major pain for so many productions. Now it will be gone.
So, what sort of performance can you expect?
Remember performance can be measured in different ways, such as by the number of transforms, or the cost of swapping animation, or whether code is Jobs/Burst compatible.
Well, to give you some idea regarding transforms, one early internal test has revealed we are far ahead of what we have today with Mecanim. This test has 300 characters, with 78 bones each, 10 clips and a 3 axial mixer, executing in under 2 milliseconds. It is still not finished, so this is pretty cool to see already. It will be even faster if you prefer a pure ECS implementation.
Another fun test was a couple of devs and a tech artist playing with some asset store content (in the above gif) to see how hard we could push the system. It was not optimized at all, just a test. But everyone was really happy with the result, so we decided to show it off at Unite.
We call it the 1000 knights scene. Here is the breakdown:
- It uses HDRP
- The new world building system
- SpeedTree
- Cinemachine
- 40,000 DOTS physics rigid body arrows.
- Each character was 100k triangles, had 24 bones, and used their own ECS StateMachine, and they were desynchronized.
- Each had dynamic control of bones with constraint nodes, and used 2 sockets for their weapons.
- Oh yeah, and it all ran at 30fps inside the Unity Editor.
Question Time
Some other questions I imagine you have.
I am guessing most of you are curious when you can get your hands on the new animation system?
- Honestly I don’t know. The new animation system will be part of a broader Unity release, and not just a standalone package, so you will most likely be able to test it when it goes into open Beta. That will not be this year though.
When will there be more to see?
- I suspect we won’t be showing much more until GDC.
What is happening with Mecanim?
- As I have stated before, it will continue to be supported but not extended.
Where can I get more information on the new system?
- As this announcement was only part of a roadmap session, getting into the nitty-gritty of what’s in, how it all works etc, will come at a future time. Again most likely at GDC.
Thank you!
This has been quite the journey already. Yes, we have made a couple of false starts along the way, but now that we can finally show you a proper peek into the future, I hope you are as excited to see it, as we are to show it to you.
I would also like to thank everyone in the Unity team who has helped us get to this point. For the past few years, they have been totally committed to delivering a solution that will make you proud you are a Unity user.
However, as with most teams who work behind-the-scenes, they rarely get the credit they deserve, but truly are the unsung heroes.
Chinny
Product Manager - Animation
