Flaky raycast with timescale at .001

I have implemented pause in my game by setting the timescale to 0.001. I have a GUI that works based on raycasts against colliders around sprites. When I pause the game, the raycast returns null 9 times out of ten when I tap a button. However, if I back the timescale to .01, it works fine. Unfortunately this isn’t slow enough to work for pausing the game.

Does anyone know why timescale affects raycasts, or how I might work around this?

I have the same problem.

Could it be a physics rate issue? Have you tried slowing fixedDeltaTime by the same ratio?