Recorder frame Accumulation breaks physics

Hi!

Using Unity 6000.0.27f1, Recorder 5.1.1 and HDRP 17.0.3

To my suprise, recording a video of the game using the recorder’s accumulation feature completely changes the behavior of physics in the game. Using Debug.Log(Time.fixedDeltaTime), I discovered that the physics timestep is divided by the amount of samples used - which does not make the slightest sense.

Eg. the default timestep of 0.02 is 0.02 when using 1 sample, but 0.0033 when using 6 samples. As a result, joints are stiffer, friction forces are stronger, piles of objects behave in a completely different way, etc. This makes frame accumulation completely unusable for most projects using any kind of physics.

I’m assuming this is a bug. Has anyone else encountered this problem?

Traced this back to HDRP’s SubFrameManager.cs class, line 224:

//This is required for physics simulations
Time.fixedDeltaTime = m_OriginalFixedDeltaTime / m_AccumulationSamples;

Beg to differ with whoever wrote this, no it’s not required, in fact it breaks pretty much everything :/. Remove this line and the recorder will behave correctly - and a lot faster than before as it does not force physics to be updated every single accumulated frame.

Do make sure to use rigidbody interpolation so that physically simulated objects are correctly rendered when the framerate is much higher than the physics timestep. Otherwise object transforms will only be updated on frames that happen to perform a physics step.

As a workaround, I’d add a checkbox on the recorder that allows to globally override physics interpolation settings for all rigidbodies in the scene so that you get proper motion blur on them out of the box. However having the recorder change the accuracy of physics and hence the behavior of the entire game is not acceptable - after all the recorder needs to capture what happens in the game without modifying it.

kind regards,

Thanks for bringing this to our attention. I wonder if the bug here is in the accumulation code or in the physics simulation.

I would expect that when reducing the timestep, the physics simulation, and in particular the underlying ODE solver, will be more numerically stable and precise. But physics is not my area of expertise, so I’m not sure.

We will investigate this internally, but yes, perhaps one solution is to leave the timestep unchanged and enable the interpolation instead, although there are some downsides as well (we’ll have to keep track of more internal state).

Meanwhile, it will help us fix this faster if you can submit a bug report with a simple repro case. Thanks!

1 Like

When reducing the timestep, numerical integration is more accurate and that’s precisely the problem: altering the accuracy of a simulation also significantly changes its outcome, specially since we are not dealing with a small change (0.02 to 0.018 for instance) but with often an order of magnitude smaller timestep since good accumulation results require at least 8-12 samples.

I sent a bug report that consists in a stack of just 5 boxes. The stack collapses during normal gameplay, but doesn’t when recording using only 6 samples.

In a game you typically choose a timestep and stick to it for the entire project. Then you adjust things like friction coefficients, joint stiffnesses and mass ratios so that behavior is just right, assuming of course that timestep length won’t change. The recorder however throws this assumption out the window, and results in game recordings that are not even in the same ballpark of what happens during normal gameplay.

Now I don’t expect results to be exactly the same every single time a simulation is run since the underlying engine is not fully deterministic, but they should at least be qualitatively similar. As it is now, accumulation is essentially unusable. It’s also weird how recording at different FPSs doesn’t change the timestep - you can record at 12, 24, 60 or 120 FPS and physics behave the exact same - but using accumulation does, when it is basically a very high-FPS recording where frames are grouped and blended together to simulate shutter speed.

Years ago I wrote a runtime recorder that allowed to export .mov and .mp4 files using h264, and allowed for accumulation too (this was a soccer game and we wanted users to be able to export goal playbacks with high-quality motion blur). I faced the same issue. We ended up storing the interpolation settings of the ball, switch it to “interpolate” during recording, and switching back to “none” once all frames had been sent to a C++ plugin which then used ffmpeg to build the final video. If it helps to know, this worked really really well, but then again we only had one rigidbody to deal with.

For those facing the same problem, I reported this as a bug and it was reproduced an accepted as such by Unity: