Rigidbody interpolation does not work with custom simulation timestep (custom interpolation example added)

I am running my game via a custom game loop. To allow for multiple time scales for different gameplay systems I am managing time/delta time myself. This means the only time-related engine variable being used is Time.unscaledDeltaTime, and I increment my own time variables accordingly. I don’t touch Time.timeScale because I am managing my own time scales.

Because I am shouldering the burden of time management, I am also manually ticking the Physics simulation from within a regular Update method.

// void Update()
while (_gameTime.AccumulatedTimeSincePhysicsUpdate >= _gameTime.PhysicsDeltaTime)
{
    _gameTime.AccumulatedTimeSincePhysicsUpdate -= _gameTime.PhysicsDeltaTime;
    _gameTime.PhysicsTime += _gameTime.PhysicsDeltaTime;
    Physics.Simulate(_gameTime.PhysicsDeltaTime);
    ...
}

This appears to work at first, but interpolation is broken because it is driven by Time.time This is very obvious when I lower my own time scale.

I get that I’m doing something a bit unorthodox by managing my own game loop and timing, but I figure if you can manually tick the Physics simulation with a custom time-step, the interpolation phase shouldn’t assume Time.time is correct.

Is there anything I can do to manage interpolation timing myself?

I happen to be pre-allocating/instantiating everything that exists in the game, so I can FindObjectsOfType<Rigidbody> after all my systems are initialized to get all rigidbodies being simulated. With an array of all the rigidbodies I was able to take a stab at interpolation myself, tho I know it isn’t going to be nearly as fast as the native implementation.

Here’s a simplified example of what the integration looks like for a custom game loop:

GameTime _gameTime;
RigidbodyInterpolationState _rigidbodyInterpolationState;

void Start()
{
    // Initialize systems.
    // ...
    // Instantiated pooled objects.
    // ...

    // Finally, allocate arrays for rb interpolation.
    RigidbodyInterpolation.Initialize
    (
        ref _rigidbodyInterpolationState,
        FindObjectsOfType<Rigidbody>(true)
    );
}

void Update()
{
    _gameTime.DeltaTime = Time.unscaledDeltaTime * _gameTime.TimeScale;
    _gameTime.AccumulatedTimeSincePhysicsUpdate += _gameTime.DeltaTime;
    _gameTime.ScaledTime += _gameTime.DeltaTime;

    // Physics tick.
    while (_gameTime.AccumulatedTimeSincePhysicsUpdate >= _gameTime.PhysicsDeltaTime)
    {
        _gameTime.AccumulatedTimeSincePhysicsUpdate -= _gameTime.PhysicsDeltaTime;
        _gameTime.PhysicsTime += _gameTime.PhysicsDeltaTime;

        RigidbodyInterpolation.PreSimulationInterpolationUpdate(ref _rigidbodyInterpolationState, _gameTime.PhysicsTime, _gameTime.PhysicsDeltaTime);
        Physics.Simulate(_gameTime.PhysicsDeltaTime);
        RigidbodyInterpolation.PostSimulationInterpolationUpdate(ref _rigidbodyInterpolationState);
    }

    RigidbodyInterpolation.Interpolate(ref _rigidbodyInterpolationState, _gameTime.ScaledTime);
    // This call was necessary for me because I am doing sphere-cast based projectiles. amongst other physics queries.
    // If the colliders aren't synced with the interpolated transforms,
    // the desync between colliders and their transforms is large enough for hit detection to be noticeably inconsistent.
    Physics.SyncTransforms();

    // Gameplay tick
    // ...
}

And this is the RigidbodyInterpolation code. Definitely not optimal, but it’s a start. Should be pretty straightforward to multithread this, but I don’t have many rigidbodies at the moment so I’m not sure it’s necessary.

public struct RigidbodyInterpolationState
{
    public Rigidbody[] Rigidbodies;
    public Transform[] Transforms;
    public Vector3[] PreSimPositions;
    public Quaternion[] PreSimRotations;
    public Vector3[] PostSimPositions;
    public Quaternion[] PostSimRotations;
    public double InterpolationStartTime;
    public double InterpolationDeltaTime;
}
public static class RigidbodyInterpolation
{
    public static void Initialize(ref RigidbodyInterpolationState state, Rigidbody[] rigidbodies)
    {
        state.Rigidbodies = rigidbodies;
        state.Transforms = new Transform[rigidbodies.Length];
        state.PreSimPositions = new Vector3[rigidbodies.Length];
        state.PreSimRotations = new Quaternion[rigidbodies.Length];
        state.PostSimPositions = new Vector3[rigidbodies.Length];
        state.PostSimRotations = new Quaternion[rigidbodies.Length];
        for (int i = 0; i < state.Transforms.Length; i++)
            state.Transforms[i] = rigidbodies[i].transform;
    }
    public static void PreSimulationInterpolationUpdate(ref RigidbodyInterpolationState state, double time, float deltaTime)
    {
        state.InterpolationStartTime = time;
        state.InterpolationDeltaTime = deltaTime;
        for (int i = 0; i < state.PreSimPositions.Length; i++)
            state.PreSimPositions[i] = state.Rigidbodies[i].position;
        for (int i = 0; i < state.PreSimRotations.Length; i++)
            state.PreSimRotations[i] = state.Rigidbodies[i].rotation;
    }
    public static void PostSimulationInterpolationUpdate(ref RigidbodyInterpolationState state)
    {
        for (int i = 0; i < state.PostSimPositions.Length; i++)
            state.PostSimPositions[i] = state.Rigidbodies[i].position;
        for (int i = 0; i < state.PostSimRotations.Length; i++)
            state.PostSimRotations[i] = state.Rigidbodies[i].rotation;
    }
    public static void Interpolate(ref RigidbodyInterpolationState state, double currentTime)
    {
        var t = (float)((currentTime - state.InterpolationStartTime) / state.InterpolationDeltaTime);
        for (int i = 0; i < state.Rigidbodies.Length; i++)
        {
            state.Transforms[i].SetPositionAndRotation
            (
                Vector3.Lerp(state.PreSimPositions[i], state.PostSimPositions[i], t),
                Quaternion.Lerp(state.PreSimRotations[i], state.PostSimRotations[i], t)
            );
        }
    }
}