Simple suggestion to prevent performance issues caused by Timeline / PlayableDirector

I wrote a massive long post about this then realised that it all boiled down to one thing:

PlayableDirector needs an option to reuse its PlayableGraph rather than destroy / recreate it constantly.

Currently the behaviour of PlayableDirector with a TimelineAsset is this:

The cost of TimelineAsset.CreatePlayable() (called by PlayableDirector.Play()) can be significant - as much as 20ms on the least powerful platform we are targeting with our game!

I guess since timeline was primarily created for cinematics this seemed reasonable, but it has serious implications for using the system for anything other than that purpose - because reasons, the game I am currently working on makes super heavy use of Timelines during active gameplay for everything from character actions to environmental object effects; so as you can imagine this has caused a bit of a headache for us.

I’ll reply below with our workaround(s) for this but TL;DR:

Please, please, please (please) add an option for PlayableDirector to reuse its PlayableGraph rather than destroy it when one of Stop() / playable ends / OnDisabled() happens.

As promised, here’s the context & workarounds (because I found almost nothing when I searched for this stuff).

Context

Many things in a game involve co-ordinating the animation of properties across arbitrary sets of objects - from characters opening doors, to attack animations, to environmental object effects - controlling / syncing animation, Vfx, audio, collider enabling etc etc. all of which is very set-up heavy & often involve custom editor tooling etc.

The Timeline package does all of the above, and has a mature editor integration so seemed a logical choice for the team early in pre-production, and the testing the pre-production team did didn’t show horrendous performance hits (on PC…) so the game went on to makes very heavy use of Timeline for core gameplay interactions throughout its development.

I’m currently working on a multiplatform title targeting PC / Mac / Steam Deck and all current gen consoles.

The game runs at a very solid 60fps on PC, but when we hit beta and started the serious process of profiling and optimising on some of the less computationally powerful hardware platforms we found that Timeline had some very unfortunate characteristics which were seriously affecting framerate stability.

PlayableDirector.Play() was costing as much as 20ms (on the first call, subsequent calls were cheaper but still typically 1-2ms. This meant that:

  • PlayableDirector.Play() (the 1st time each PlayableDirector is used) is a guaranteed frame drop (1 frame @60fps : 16.66…ms @30fps : 33.33…ms)
  • even the lower 1-2ms cost of re-Play()ing is a pretty big problem - especially since we often have 2 or 3 calls on the same frame in active gameplay

Attempted Workaround: Manage Our Own Playable
Initially I assumed it would be easy to just use TimelineAsset.CreatePlayable() and manage our own PlayableGraph.
It turns out that this is actually non-trivial:

  • partly because the game uses custom timeline tracks, some of which rely on there being a PlayableDirector…
  • …but also because the playables API is :exploding_head:

I spent a morning trying to work out how to use the binding data on a PlayableDirector instance to set up the outputs of a PlayableGraph (created using playableDirector.PlayableAsset.CreatePlayable()) and gave up.

I’m sure it’s completely doable, but I couldn’t find any sample code which explained how to do it, or find anything in the documentation which shed much light on it either, and because the objects in a PlayableGraph arre all handles I found that my usual approach of using the debugger to try to make sense of the data at runtime didn’t work, so because time pressure I decided to try something else.

Actual Workaround: Abuse the PlayableDirector API to prevent it destroying its PlayableGraph
On a hunch, I tried setting the PlayableDirector wrap mode to hold and discovered that this prevents it ever properly stopping and therefore it doesn’t destroy or recreate its PlayableGraph.

There was another fly in the ointment though, even with the wrap mode set to hold, if the parent GameObject of a PlayableDirector is disabled then this also causes the PlayableGraph to be destroyed; the way the game manages much of the behaviour of the objects using Timeline is to (en/dis)able them so this needed an additional layer of workaround…

The final workaround for was to clone all PlayableDirector instances in the game, and cache them in an area of the hierarchy which is never disabled.

For each PlayableDirector used in active gameplay:

  • create a new PlayableDirector…
  • …copy all the generic bindings from the original…
  • …for tracks which use other binding approaches (e.g. ControlTrack) manually iterate the original’s asset bindings and copy them over
  • prevent the cost on initial Play() by using RebuildGraph()
  • avoid the automatic destruction a PlayableDirector.playableGraph by
  • setting playableDirector.extrapolationMode = DirectorWrapMode.Hold (which prevents the timeline stopping automatically)
  • check for the Timeline ending with Time vs. Duration of the timeline Playable and use this to send an event equivalent to PlayableDirector.stopped
  • instead of Stop() call Pause() then reset PlayableDirector.time

Summary
Several days and a lot of stress would all have been saved if PlayableDirector had an option to reuse its PlayableGraph rather than destroy / recreate it constantly.

also: re binding the outputs on a manually created playable graph (i.e. not owned by a PlayableDirector) from the generic bindings of a director

This bit of info filled in (most of) the blanks: Runtime modification of bindings

Can’t believe I missed it before :sob:

Sadly, there’s still the issue of how to bind ControlTrack clips in a PlayableGraph which isn’t owned by a PlayableDirector (see this post: When TimelineAsset.CreatePlayable() is used to make a PlayableGraph, why is PlayableGraph.GetResolver() always null?)

I cooked up some code which removes the need to use the PlayableDirector at runtime by manually creating a PlayableGraph and then cloning the scene bindings from the PlayableDirector.

This includes a mechanism for copying the bindings which use PlayableGraph.GetResolver() When using TimelineAsset.CreatePlayable() to make a PlayableGraph PlayableGraph.GetResolver() is always null; how do we create a resolver for it? - #4 by darbotron

ohohohooooo.

It actually gets worse.

Not only is there no way to stop PlayableDirector from destroying its PlayableGraph on Stop()

but several / many of the PlayableBehaviour classes in the Timeline UPM package actually rely on this behaviour in order to work correctly.

:man_facepalming:

To support my goal of manually creating and managing PlaybleGraph instances from TimelineAsset instances (to avoid poor performance caused by horrendous memory churn) I have had to modify the UPM asset code to work around this.

The fix is relatively straightforward: code in classes deriving from PlayableBehaviour which is in OnPlayableDestroy may also need to trigger from OnGraphStop, if this is the case it may also need additional (re)init steps.

hey @darbotron
We’re glad that we found someone with the similar issue we have. Did you get a working solution for this? Like modifying the Timeline Package?

yeah, that was the fix - if you copy the timeline package out of <ProjectRoot>\Library\PackageCache\ into <ProjectRoot>\Packages and modify it there that does the trick :smiley:

It is ludicrous that these kind of workarounds are necessary though - why on earth the original implementers thought it would be a good idea to force the graph to be destroyed on Stop / Disable in the first place is beyond me!

I think i’m starting to understand the fix :slight_smile: So first it’s using the demo code you put into the unity package where basically you copy the needed bindings and put them into objects that never get deactivated, right?
And the second part is modifying the timeline package where you make sure that code that’s being executed in the “OnGraphDestroy” methods gets handled in “onGraphStop”.

As a starting point that’s the whole trick, right?

And yes i totally agree - it’s ludicrous that timelines are that wasteful. But i guess then maybe we’re both using timelines in a different fashion that it was intended for :slight_smile: We use it a lot for gameplay elements and end up having multiple hundred timelines running at the same time. Which brings down performance A LOT when running on the steamdeck :smiley:

Thanks a lot for your research into this and even more for sharing your knowledge.

btw. have you reported this as a bug? ( = better changes of it getting improved…)

yeah, not a bad idea tbh.

will get around to it at some point :smiley:

Any idea on how to make this work with director control tracks?

ooooooh. that is going to be “fun”.

I’ve extended our in-house version of this code to do correctly initialise a bunch of TimelineAsset types and the corresponding runtime PlayableBehaviourl; and I’ve managed to get it to initialise ControlTrack instance (though we don’t use it for controlling Directors).

Your best bet is to look at the source code for the director control track in the Timeline package.

If it were me, I’d be inclined to move the timeline package out of your /Library directory and into /Packages so you can edit it - the easiest way to fix this is probably to extend ControlPlayableAsset so that it can handle these cloned PlayableGraph instances rather than having to use a PlayableDirector..

Good luck!