Animation interpolation of a quaternion property

From Unity’s manual:
The Animation View lets you choose which form of interpolation to use when animating Transform rotations.

That works. If I animate the rotation of a transform then the animation window let me choose the interpolation type and the animation interpolate the rotation between frames properly.

But if I try to animate a quaternion property of one of my own script, then the interpolation doesn’t work. The animation window doesn’t let me choose the interpolation type and the animation interpolates the quaternion’s values XYZ and W independently from each others instead of interpolating the rotation represented by the quaternion, which produce garbage result.

Is there a way to make the animation interpolate my quaternion properly, like it does with transform’s rotation?

No, there is no direct way.

As far as I understand it, the Animation Editor is just a layer over the actual animation data (curves and keyframes). Changing the interpolation mode triggers a recalculation of keyframe (values and tangents).

In my experience the linear interpolation of a quaternion’s components (xyzw) is not so far off, as long as the rotation is along the shortest path. Quaternion.Slerp for example ensures that, but not AnimationClip!

To solve this, you need to flip subsequent keyframe value quaternions that whose angel is too large. Here’s some working, but slow code:

static void FixRotations(AnimationCurve rotX, AnimationCurve rotY, AnimationCurve rotZ, AnimationCurve rotW) {
    Profiler.BeginSample("AnimationUtils.FixRotations");
    var prev = new quaternion(
        rotX.keys[0].value,
        rotY.keys[0].value,
        rotZ.keys[0].value,
        rotW.keys[0].value
    );
    for (var i = 1; i < rotX.keys.Length; i++) {
        var keyX = rotX.keys[i];
        var keyY = rotY.keys[i];
        var keyZ = rotZ.keys[i];
        var keyW = rotW.keys[i];
        var value = new quaternion(
            keyX.value,
            keyY.value,
            keyZ.value,
            keyW.value
        );

        if (math.dot(prev, value) < 0) {
            value.value = -value.value;

            keyX.value = -keyX.value;
            rotX.MoveKey(i, keyX);
            
            keyY.value = -keyY.value;
            rotY.MoveKey(i, keyY);
            
            keyZ.value = -keyZ.value;
            rotZ.MoveKey(i, keyZ);
            
            keyW.value = -keyW.value;
            rotW.MoveKey(i, keyW);
        }
        prev = value;
    }
    Profiler.EndSample();
}

I recommend doing this in a C# Job!

hth