How to read the Track Binding from a Timeline Behaviour


Specifically, I wrote a monobehaviour, which I named BlendShapeController, that scans an attached SkinnedMeshRenderer and stores info about any BlendShapes it finds into an array of ScriptableObjects.

So far, so good. I would like to be able to control the blend shape values from a Timeline. Where I am running into problems is getting a reference to the PlayableBehaviour (named BlendShapeControlBehaviour) for the Timeline track in such a way that the BlendShapeControlBehaviour can generate its own array of ScriptableObjects, which the user can employ to animate the blendshape values on the BlendShapeController bound to the track.


Where I am running into a snag is that, although I am able to obtain references to the BlendShapeController to the PlayableBehaviour, it seems to be at a stage where it is too late to useful. Although Debug.Log() can reference the BlendShapeController in the BlendShapeControlBehaviour or BlendShapeControlMixerBehaviour’s ProcessFrame (and a few other methods) , it doesn’t stick, and the fields in BlendShapeControlBehaviour are always empty. I suspect I am dealing with a clone of the PlayableBehaviour in ProcessFrame and not the original instance that actually stores the data longer term.
I suspect that I need to be getting and processing the reference in the CreatePlayable method of the BlendShapeControlClip but I haven’t been able to figure out a way to do this. I tried making the BlendShapeController property in the useful. Although Debug.Log() can reference the BlendShapeController in the BlendShapeControlBehaviour an exposed reference (usually used for inputs, I guess) and then assigned it using Resolve(graph.GetResolver()) after Creating the ScriptPlayable holding the Behaviour, but that does not show in the inspector. Neither does any Blendshape data I attempted to extract and store.


Is there any way to accomplish this without either hardcoding the number of blendshapes in the BlendShapeControlBehaviour or assigning the BlendShapeController manually for each clip?
Attached below is a screen of what the BlendShapeController inspector looks like. I would like the clip inspectors to mirror this layout, but without sharing the actual object references.
I will be happy to append code, if anyone feels it might be useful, but since none of it seems to be even close to being a viable approach, I didn’t think it would. Thanks for any suggestions.
[edit]Perhaps I should have posted this in Default?[/edit]

Figured it out. As far as I can tell, none of the different bits can accomplish that except the TrackAsset.

I put
director.SetReferenceValue(blendShapeControlBehaviour.controller.exposedName, _mTrackBinding);

into the GatherProperties Method of the BlendShapeControlTrack

If you are writing a custom track with the binding to your BlendShapeController, it will be passed as the playerData argument in ProcessFrame, you just need to cast it to the appropriate type.

I’d recommend doing all the work in BlendShapeControlMixerBehaviour.ProcessFrame. i.e. walk the clips to find which ones are active, do any blending (if required) and write that back to the binding.

GatherProperties is not an appropriate place to store references, as it won’t be called in the player.

Thanks for making this code available. I don’t really understand what you are talking about but I can see that you’ve implemented a way to animate Blendshapes on timeline which I do understand (and was looking for).

I’m hoping you can help me to get this code working. I have installed Collections and Maths, changed the project to .net 4, added a track with a skinnedmesh with shapekeys, added the Controller component and added a clip. I can move the sliders in the Clip and the Blendshape movement is visible so I think everything is working to this part. But I cannot add keys and there is no record button.

I can see a Type Mismatch in the M Track Binding field. I’m trying to get my head around this code (and Playables in general) but it is mighty confusing. Would you be able to offer me some advice on how to record keys?

I find a trick from the forum of unity, they assign the binding to behaviour when create the playable of the track.

    public override Playable CreateTrackMixer(PlayableGraph graph, GameObject go, int inputCount)
        foreach (var c in GetClips())
            (c.asset as FacialExpressionClip).parentTrack = this; // this field would be serializable in assets
            (c.asset as FacialExpressionClip).binding = go.GetComponent<PlayableDirector>().GetGenericBinding(this); // not serializable in assets
        return base.CreateTrackMixer(graph, go, inputCount);