Unite Now - Kinematica – Democratizing Motion Matching for Character Animation

[AskCarol]

Unite Now Session

In this session you’ll learn how to get started with Kinematica and implement it within your animation pipeline. Based on Motion Matching, Kinematica uses raw animation clips or motion capture data – not animation graphs – to produce character poses automatically, based on your description of the intended movement.

Update (May 29th)
The demo is now available! We’ve made the GitHub repository public as well.
The repo address is - GitHub - Unity-Technologies/Kinematica_Demo: Kinematica Demo Project

The release can be downloaded here.

When
Thu, May 7, 9AM PT

Where
You can watch the session here.
Make sure to sign up to get access to all Unite Now content.

---

Together with this Unite Now session we are also hosting a Q&A here in this thread to answer as many of your questions as possible.

A team of product experts from across Unity will be available starting after the session. Our Community team will continue to field questions to foster an ongoing discussion with the community. Please feel free to ask your questions in this thread.

Some basic rules

• Don’t bundle multiple unrelated questions in one reply. One question/topic per reply.

• Only questions related to the topics of the session are permitted.

• All questions will be fielded by our Community Managers (@LeonhardP and @AskCarol)

• Replies will have to be approved by the moderators to show up in the thread.

• Once approved, the questions will be forwarded to the relevant experts.

We really look forward to hearing from the community.
Thank you!

[laurentlavigne]

How can I use Kinematica with a small set of keyframed clips?
Example of small set: walk and run forward/backward/strafe right/strafe left, vertical jump up, fall down, land

[merpheus]

Should we motion match with Kinematica for Upper body animations such as aiming and shooting a pistol or swinging a sword etc or use an animator mask with a new animator layer that driven by traditional animations/transitions?

Second question, if we want to blend Kinematica with IK or traditional animation, how hard it is to make?

[hippocoder]

I’m interested. I’d like this system to work out the best anims / pose given a head and 2 hand transforms (a typical VR setup). This will allow a full body representation a lot easier, and work on anything from a Quest to a Console. Will you do that incredibly challenging and inspiring work for me?

[DavidGeoffroy]

Edit: I misread your question. I’ll leave the answer here because it is tangentially related, and answer your actual question at the end

@laurentlavigne

How can I use Kinematica with a small set of keyframed clips?
Example of small set: walk and run forward/backward/strafe right/strafe left, vertical jump up, fall down, land

At the moment, we have limited support for clips that are shorter than what Kinematica calls the “Time Horizon”. The default for this setting is 1 second. We want to add implicit support for short, looping clips, but this hasn’t made the cut for the first preview. For any clips that are shorter than the Time Horizon, you will need to stitch clips together to have at least 1 second of continuous motion.

For clips that are at least as long as the Time Horizon, you can provide “boundary clips”, which will provide trajectory information outside of the range of your clip. This allows you to have valid frame and trajectory for the whole duration of your clip. Otherwise, your trajectory abruptly stops for last 1 second of your clip.

For the moment, we support only a single boundary clip for pre and post trajectory extrapolation, so you should use your most likely transition, or the same clip if it’s meant to loop. While testing and building the demo content, this hasn’t been a problem, but we may adjust if this becomes problematic to trigger correct transitions in specific cases.

Regarding your actual question: using Kinematica with a small set of clips:
Yes, you can use it with restricted set of clips, either like a classic state machine with motion matched transitions: “Find me a good pose inside ‘Locomotion’ to transition to”, or by using trajectory matching. How smooth the result will depend on how well connected your animations are.

You can also use Kinematica as a "state machine if you want to. It’s a bit overkill, but it’s helpful if you want to force certain clips to play and you don’t care about matching. There are samples in the package that showcase this.
If your set of clips for locomotion is limited to, for example a strict 8-direction motion set, and you want to use path-based motion matching, then you will get better results if you limit your desired trajectory input to strict, 8-directions

[DavidGeoffroy]

For your information, the package is already publicly available on the Package Manager , so you can go try it out as you wait for the talk to start.

Also worth mentioning, there is a Demo project mentioned in the video. Sadly, while the package is available, the demo is not yet available for public release, we need a bit more time to polish it.

We will do our best to have it available as soon as possible, as well as give targeted samples that detail the more advanced use cases like parkour.

[DavidGeoffroy]

@merpheus

Should we motion match with Kinematica for Upper body animations such as aiming and shooting a pistol or swinging a sword etc or use an animator mask with a new animator layer that driven by traditional animations/transitions?

Second question, if we want to blend Kinematica with IK or traditional animation, how hard it is to make?

Q1: It’s up to you. Kinematica itself doesn’t support partial animations at the moment, so doing this through motion matching would have a memory cost. But, Kinematica goes through the Animator and an PlayableGraph to write to GameObjects, so you can add other AnimationOutputs to overwrite parts of the hierarchy if you want to.

Q2: It depends. The workflow we have focused on for the first preview is the “Full Kinematica” workflow, where you only use Kinematica, and go through the Kinematica Component. This should interact nicely with things that postprocess the Animation, like the Animation Rigging package or FinalIK.

If you want to combine it with an AnimatorController, your own AnimationJobs, a Timeline, etc, it is definitely possible, but will require a bit of fiddling with PlayableGraphs. This is also an area where we have spent much less time, so feedback and bug reports are appreciated.

[DavidGeoffroy]

@hippocoder

I’m interested. I’d like this system to work out the best anims / pose given a head and 2 hand transforms (a typical VR setup). This will allow a full body representation a lot easier, and work on anything from a Quest to a Console. Will you do that incredibly challenging and inspiring work for me?

Puppeteering is an interesting use case. I don’t think Kinematica will perform well (or at all) in your use case. It is clearly optimized towards finding the next best pose given your current pose (which is usually known ahead of time). In the case of puppeteering, I imagine your possible poses are endless.

Try it out, tell us if it works!

[Kaen_SG]

I’ve watched all of Michael Buttner’s videos from GDC, Unite, Siggraph etc and I’ve seen the talks about Dance cards and continuous mocap clips.

My question is about support for “Cut-Clips”. Do you support them and if so, can they be also be combined with dance cards?

(Cut-Clips are small mocap clips that have been prepared to support the old state machine workflow, for those poor indie devs like me who don’t have access to a mocap studio/suit. Currently all Mocap packs on UAS are sold as cut clips)

[Kaen_SG]

If access to a mocap studio is somehow available to small indies like myself, what is the recommended option:

1. Provide as much data as possible to kinematica to aid in matching. eg. Overlapping motions & multiple redundancy in motions.

OR.

2. Provide only the best, with no redundancy or overlaps (perhaps it confuses kinematica or consumes too much RAM?)

[Kaen_SG]

Regarding the link in the video. "

Github says…

Us plebians need permissions. Thanks.

[DavidGeoffroy]

@Kaen_SG

I’ve watched all of Michael Buttner’s videos from GDC, Unite, Siggraph etc and I’ve seen the talks about Dance cards and continuous mocap clips.

My question is about support for “Cut-Clips”. Do you support them and if so, can they be also be combined with dance cards?

(Cut-Clips are small mocap clips that have been prepared to support the old state machine workflow, for those poor indie devs like me who don’t have access to a mocap studio/suit. Currently all Mocap packs on UAS are sold as cut clips)

This is partially answered here , and in the video. You can definitely use Kinematica without a full dance card. All the samples in the project use a limited move set, for size reasons, and to showcase how to use Kinematica without a mocap studio

But Kinematica doesn’t generate motion, so your results will be as smooth as your input motion. Kinematica will find the best match for you. It may happen that the best match is not a great match. You can still tell kinematica “go to my JumpForward animation, I don’t care if it pops”.

@Kaen_SG

If access to a mocap studio is somehow available to small indies like myself, what is the recommended option:

1. Provide as much data as possible to kinematica to aid in matching. eg. Overlapping motions & multiple redundancy in motions.

OR.

2. Provide only the best, with no redundancy or overlaps (perhaps it confuses kinematica or consumes too much RAM?)

The more data you give Kinematica, for a given Tag (e.g.: Locomotion), the harder you will have to work if you want to get this specific animation, and the more specific you will have to be on what joints you use to match these poses.
This means more processing and more memory.

But Kinematica lets you force a specific sequence of poses without requiring you to always use the pose matching, or the trajectory matching. If you tag a section of a clip or a clip as “JumpForward3M”, then you can tell Kinematica to give you JumpForward3M, and it will happily do so.

Regarding the demo content:
This is also mentioned here . I’m sorry, the demo content is not ready for prime time yet.
The video was produced a few months ago, and while we were hoping to have the content ready in time for today, a global pandemic happened, and our plans were a bit shuffled.

We’re working to make it available as soon as possible

[Kaen_SG]

@DavidGeoffroy

This is partially answered here , and in the video.

We’re working to make it available as soon as possible

Thanks. Does kinematica have a preference performance-wise?
ie. “1000 Cut-Clips” vs a “45min Single Mocap Clip”

[sqallpl]

Hey,

Is it possible for Kinematica to blend more than one ‘clip’ at the same time? I’m not asking about blending between poses/animations on start and end but general blending, layering and layering concept.

For example it could be a melee locomotion for lower body blended with melee actions for upper body for compound actions like walk forward + weapon swing, step backward + weapon swing, stand still + weapon block, walk forward + hit stagger etc. Or maybe it could somehow use ‘melee locomotion’ for the base motion while playing upperbody actions at the same time? I’m just wondering here how it can work with things like that, based on some concepts from the presentation.

In classic animation graph like Mecanim it’s usually done with layering, masking and playing more than one clip. For example we play ‘stepforward’ clip on the legs and root layer, ‘weapon swing’ on the upper body layer that overrides the base layer and ‘head stagger’ on the additive layer for head and all of them are controlled independently by script and state machine logic.

How it could be achieved with Kinematica? Would it require a dedicated ‘fullbody’ clips that contain fullbody compound motions? Or maybe it can mix separate clips so it could use sets of clips (upperbody, lower body, fullbody, hands only etc.) like one-handed locomotion, one-handed upperbody attacks, one-handed upperbody blocks, upperbody staggers etc. ?

[DavidGeoffroy]

@Kaen_SG

Thanks. Does kinematica have a preference performance-wise?
ie. “1000 Cut-Clips” vs a “45min Single Mocap Clip”

This is partially covered here . There are limitations on how short your clips can be before it becomes tricky. But besides that:

Runtime-wise, there is no impact. Kinematica only knows about each discrete segment you have decided to tag. There is no functional difference between tagging all your “clips” out of a long mocap clip, or having all your clips separate and the whole clip is tagged.

Workflow-wise, it’s a balance. Longer clips are a bit easier to work with, but once your clips get long enough, it becomes harder to reason about them. It’s easy to reason about “RunTurn45Deg: frame 45”, but it becomes harder to figure out what animation is actually playing if the current frame is “RunTurnSpirals15To105Deg_mocap: frame 259”.

[DavidGeoffroy]

@sqallpl

Is it possible for Kinematica to blend more than one ‘clip’ at the same time? I’m not asking about blending between poses/animations on start and end but general blending, layering and layering concept.

Kinematica currently only plays one segment at a time. On any transition between poses or segments, it uses a version of inertial blending (see this talk for the general idea). So you will get some amount of blending, but you will not get multiple clips running at the same time. If you want layering, you can setup Kinematica to run in a PlayableGraph, and there you can layer Kinematica with other Animation sources, with IK, or with another Kinematica Synthesizer if you want to.*

This is partially also covered here

*As mentioned in the comment above, we have validated that it’s possible but this is not where we have invested the brunt of our efforts, so the user experience is not as straightforward, and will require more programming.

[chusmaverde]

Hello, the Kinematica feature looks amazing, I am eager to try it out but for some reason I am unable to see the preview package in the package manager. I tried to find it in versions 2019.2.6f1, 2019.3.13f1 and 2020.2.0a10.

Current screenshot from 2019.2.6f1

1. I would appreciate some hint on what I am doing wrong.

2. What is the expected date to release the full demo from GitHub (taking into account the current situation)?

Thanks

[brunocoimbra]

@chusmaverde

Hello, the Kinematica feature looks amazing, I am eager to try it out but for some reason I am unable to see the preview package in the package manager. I tried to find it in versions 2019.2.6f1, 2019.3.13f1 and 2020.2.0a10.

Current screenshot from 2019.2.6f1

1. I would appreciate some hint on what I am doing wrong.

2. What is the expected date to release the full demo from GitHub (taking into account the current situation)?

Thanks

Taken from 2019.3.9f1

[Kaen_SG]

@chusmaverde

Hello, the Kinematica feature looks amazing, I am eager to try it out but for some reason I am unable to see the preview package in the package manager. I tried to find it in versions 2019.2.6f1, 2019.3.13f1 and 2020.2.0a10.

Current screenshot from 2019.2.6f1

1. I would appreciate some hint on what I am doing wrong.

2. What is the expected date to release the full demo from GitHub (taking into account the current situation)?

Thanks

Shows up for me in 2019.3.12f1 as well.

If you somehow do not see it in your package manager. You can add it yourself by editing the file “manifest.json” in your project folder /packages/

and add this line

“com.unity.kinematica”: “0.5.0-preview”,

[chusmaverde]

@brunocoimbra

Taken from 2019.3.9f1

Thanks, I do not know what happened, but now I can see the package.