UnscaledDeltaTime value when there is lag, or when pausing playMode

Hello, I'm using deltaTime to animate UI, and I want to use unscaledDeltaTime when animating the Pause menu's UI, because I set the timeScale to 0 there.

My issue is the following :

If I "pause" playMode and use the "next frame" button (over and over), it still works fine, I can see my pause UI animating frame by frame properly, and the timers used, incremented properly.
But when I "unpause" playMode, I get a sudden bump of how long I was "playMode-paused".

So for example, if I was "playMode-paused" for 20 secs, and the timer variable value was for example at 0.1456975, it will become on "unpause" 20.1456975.

Also, if there is lag, the same issue happen.

The issue is not there when I use deltaTime.
Why is that happening ? How can I prevent it ?

I thought unscaledDeltaTime was supposed to work exactly like deltaTime, but just not take into consideration the timeScale.

Thank you.

This seems to be the exact issue I'm having :
https://discussions.unity.com/t/675468/4

Does this issue have any workaround ?