How can you import lip synched animation from Blender?

Unity Engine
1

I have made some lip synced animation in Blender with blend-shapes but I can’t import it into Unity. Any tips? Or is it possible?

2

@Baste seems to know a process for this.

3

Right, so our process for this is pretty straight-forward, but it takes a bunch of work and workflow changes.

  • For each blend shape, add a bone to the rig. We call these the “driver bones”
  • Set up constraints on the blend shapes, so the driver bones’s position sets the blend shape values. You can also use scale, we’ve used both successfully.
  • Make sure these “driver bones” as we call them deform! Otherwise they won’t export.
  • Animate using the driver bones! This means that instead of keyframing the values for your blend shapes, you keyframe the driver bone positions, and have that cause the blend shape to change through the constraints.
  • Export to Unity. Your model will now contain two things; the blend shapes, and the driver bones. The constraints will not export from Blender (sadly)
  • Recreate the constraints through a script in Unity. This is pretty simple to write; set up a thing that takes one bone and one blend-shape, and sets the value of the bone based on the value you used in Blender.
    NOTE! It’s important to use localPosition or localScale here, not global position or scale. You don’t want walking forward to cause your character to smile.
    NOTE2! Unity and Blender (well, fbx) uses different coordinate systems, so you’ll have to compensate. For example, if your constraints are set to +z in Blender, your constraint script has to use -y.
    NOTE3! ExecuteInEditMode is your friend when debugging the Unity driver script.

We put a lot of work into making the process easy - the biggest thing is to be consistent about naming, so each of the bone’s names are the blend shape’s name + “driver”. Then we can write an editor script in Unity that automatically assigns the driver bones to blend shapes based on names.

One thing else to note is that if you’re optimizing your rig through the import settings, it’ll be a real pain, because you have to set the driver bones to not be optimized away. And, BOY does the interface for selecting which bones to keep suck majorly. Anything else than one level deep is a nightmare. So I recommend keeping all of your driver bones just under the root.

Ask if there’s anything unclear about the process - I’ll help as best I can. It’s taken so much R&D to get this working that I’m not really willing to share code for free. The code’s pretty simple now, but it took a while to get here. It might be that we’ll polish this up enough to put it on the asset store, or I might change my mind and open source it at one point. We’ll see.

It might also be that there’s a better way to do this in the future. Unity’s creating a constraint system, so that’ll simplify things quite a bit, but from the threads on that it seems like Blender’s constraints don’t export to FBX, so you can’t skip the recreate-in-Unity step.

I would also love to find some way to read the animation curve data without having to have the bones in the scene, as that would mean big performance gains (the Animator is_so_friggin_slow at animating Transforms). You can apparently have “animated custom properties from FBX” since 2017.2, though we haven’t tried using that yet. That sounds like something that might work to replace the bone-based solution.

Final note: I’m not the animator, but the animation programmer, so my terminlog might be a bit off on some terms. I don’t remember how to split a triangle in Blender! I have never rigged a single thing, nor created any animations in Blender!

I’m pretty killer at Blender scripting, though. Ask me about it sometime, if you want to hear me whine about how absymal the API is.

1 Like