Calling the physics simulation from C#

Hello everyone,

One of the long standing and rather frequent requests to Unity’s physics team is to expose the explicit control over FixedUpdate.

Let me quickly show a few examples as a motivation for this.

An obvious usage could be networking physics: turns out that in order to do convincing multiplayer interaction in FPS-like games one needs to be able to rewind back time, apply the correction that arrived from the server and tweak it with the locally applied input. The latter was pretty much impossible really since in Unity you can’t call the physics simulation manually.

Another example is a simulation game. Think golf, pool or bowling. It’s desirable to be able to preview the trajectory of the controlled ball given this particular direction, strength and type of the strike. In order to do that seamlessly one needs to be able to simulate the scene for a few frames in advance which isn’t directly possible in any of the public builds right now.

One additional bonus is that with this control exposed, it’s easy to have some logic post-update. Currently, the FixedUpdate script callbacks are called right before the physics simulation is run. Thus, if you wish to have a script that gets transforms right after the physics update was called, you have to jump through hoops. A possibly smart workaround being used these days is to create two overlapping triggers somewhere far away from the game area, subscribe to OnTriggerStay from them, and get the post-simulation callback that way.

There were multiple talks over the years where many theoretical concerns were pointed out with regards to exposing Physics.FixedUpdate to scripts. Moving the physics objects may invalidate the PhysX’s caches and thus cost performance spikes when rewinding the time back and forth. Calling simulation with the variable delta will get you unpredictable solver results. Possibly, there are lots of ways to break some assumptions when FixedUpdate is called at unexpected times from scripts.

However, having many theoretical ideas, there wasn’t much actually tried out. So this thread is targeted at correcting that. Let’s try it out together, collect ideas and have a conclusion whether it’s worthy having the control exposed this way or not.

I’ve prepared a custom build off of 5.4.3p3. Can do any Unity version if makes sense, my change is minimal. Let me know. Here it is: http://beta.unity3d.com/download/482a73abaf24/public_download.html

That’s only editor that doesn’t seem to be capable of building players. Probably, an inconvenience for the networking tests, but I believe you could still run multiple editors on the same machine. Let me know if that’s a blocker for you, I’d figure something out.

As to what’s added. The build has only two little things added: Physics.autoSimulation and Physics.SimulateStep(float delta).

  • Physics.autoSimulation (also, available in the project settings): toggles whether the simulation should be run automatically, or the user should be calling Physics.SimulateStep manually
  • Physics.SimulateStep: a new function that updates physics by ‘delta’ seconds forward.

Here is an example of how you get pretty much the old logic of updates, but from C#:

using UnityEngine;

public class BasicSimulation : MonoBehaviour {

    private float timer;

    void Update () {
        if (Physics.autoSimulation)
            return;

        timer += Time.deltaTime;

        if (timer >= Time.fixedDeltaTime)
        {
            timer -= Time.fixedDeltaTime;
            Physics.SimulateStep(Time.fixedDeltaTime);
        }
    }
}

Note that in this build Physics.autoSimulation controls only the actual simulation calls to the physics engine. It doesn’t affect the FixedUpdate calls to your scripts. Those will still be called based on the fixed frequency set in the properties.

Please let me know what you think. Is it a step in the right direction? Is it a step in the wrong one? What needs to be done in a different way? What do you think?

Anthony

24 Likes

Great stuff Anthony, I know many people have had this on their wishlist for a long time, looks like a great first step to figuring out if this is a viable feature. I’ll build a benchmark to see what the performance implications are like for rewinding time :slight_smile:

1 Like

This looks fantastic! The lack of a SimulateStep function has precluded the use of a lot of PhysX features on our current project.

This is very exciting to think about, and could possibly lift some weight of our shoulders where our only current alternatives are switching to Bullet or writing our own physics system. Both are very exciting prospects, but also very time consuming.

Being able to tick a physics world non-realtime would give us a lot of needed flexibility, but I’ll have to think hard about whether it’s enough for our current projects. The way I’ve set up rewind/correct logic in my experimental networking code is by creating small sets of physics objects and rewinding/correcting them separately. Here’s how we’re currently thinking about networked physics for Volo Airsport:

Operating on Subsets of Physical State

Different physics objects (be they a single rigidbody, or a set of strongly interacting rigidbodies like a ragdoll) often do not need to be corrected in unison. Sometimes it makes more sense to roll back and correct, say, only the physics state for a single object (a ragdoll for each player, in our case). This would avoid performing correction work on physics state that doesn’t actually need to be touched.

Say I’m using the physics engine for some networked state, but also for some client-side-only effects (like explosion debris or whatever). I do not want to resimulate the entire game to correct my player’s state, because that also means having to resimulate that irrelevant physics-based effect.

Having granular control over which subsets of physical state in your game are interacting with each other can greatly help you keep computational complexity under control in networking.

I suppose this can be achieved by managing which objects are reacting to this new FixedUpdate call at invocation time. If I want to simulate my avatar’s physical state for 1 second, non-realtime, while not resimulating anything else, I could temporarily freeze all non-player state, unhook its associated FixedUpdate implementations, and then tick the physics knowing that only the state of my player’s physics will run. How efficient this is I’ll have to test.

With a more data-oriented or purely functional approach this would look something like:

Physics.Simulate(PhysicsWorld world, float deltaTime)

Considering Bullet for moment, we think using several PhysicsWorlds could achieve this. One world for global, real-time, on-screen state. Another world to do rewind/correct for specific networked objects that need it. It’s not the only way to get it done, but it’s one we’ve tested and we’re quite happy with.

Blending Dual Representations For Perceptual Continuity

Calculating corrected physical state non-realtime from some received server state is one thing; applying it smoothly to client state (which is visible on screen) is another. If you receive a server state that says your predicted avatar state was off by 5 meters, and instantly apply it, you get jarring visual discontinuity of motion.

Our particular solution is to always have our clients run a full physics simulation for the entire game world that just goes on in real-time, with a separate world containing duplicate avatar physical state also running in realtime, not directly visible on screen. This second representation of the player has non-realtime correction (derived from the latest server state) applied instantly, which we can trivially do because it’s not visible.

Then, every frame we interpolate the player object’s global state to this corrected state such that the 5 meter error is applied smoothly over multiple visible frames. You still technically have perceivable on-screen discontinuity this way, but now it’s hidden in further up the derivatives of the player’s position, not position itself.

In Bullet this can be implemented as having a PhysicsWorld for real-time simulation on the client, with a separate PhysicsWorld in which only a single physics object for your player exists, which can be ticked in non-realtime.

Without this multi-world implementation it becomes harder to properly separate collisions between the two representations and such. But I guess this too would be possible with extra scaffolding built on top of Unity (with smart managing of collision layers or something). Another thing I’ll test the performance of.


I’ll give this more thought throughout the week, I’d love to see a version of Unity that ships with a physics API flexible enough to do rigorous networked physics. Even with just this change I’d guess there’s a lot of new things possible that were not before.

(Oh by the way: I see ConfigurableJoint.CurrentForce and CurrentTorque have made their way into 5.5. Thanks a bunch, Anthony! I’ll put that to good use.)

2 Likes

Great to see features like this being considered!

For this feature to be useful it’s absolutely crucial to have control over which objects are interacting with one another.
In a multiplayer context you often simulate a partial copy of the world before using the result of this calculation to drive gameplay logic and rendering. It makes no sense to resimulate the entire world since the state of the objects that are currently rendered on screen are just as relevant as the resimulated state. In other words: what you see is often not directly what you simulate.

So in the end what you do in multiplayer is to make a copy of a player object that is invisible and use that together with all the static colliders and invisible copies of other players to perform simulation. The result of that calculation is smoothly applied over time onto the actual visible objects in the world to prevent glitches/stuttering etc.

I don’t see this feature being as useful if we don’t get to decide which objects are being simulated since we would still be very limited in what we can resimulate. In multiplayer you always hold multiple views on what is ‘true’ and we need to be able to make that distinction in code.

A pseudo-code API could look like:

IList<GameObject> simulatedObjects = ...
float deltaTime = ... // The total time passed
float timeStep = ... // The actual simulated time per tick
Physics.Simulate(simulatedObjects, deltaTime, timeStep);

Actually Bullet does this pretty well, giving you control over many aspects of the simulation even allowing for interpolation between physics ticks. It allows you to render objects very smoothly even with a low timeStep.

1 Like

I did a bit of investigation in to the suspicion that rewinding object state would break physx’s caches. From what I can see now there is no such effect. Every frame I rewind and resimulate 20 steps of 10 milliseconds, plus and additional step to actually move time forward. It appears we spend about 20% less then 21 times the cpu time measured in a single step scenario, not sure why. Below is the test setup, I attached the script that runs the benchmark.

Object state is position, rotation, velocity, angular velocity

Initialization

  • Spawn 1000 cubes at random locations withing an 50x50x50 volume
  • Set their velocity towards the centre of the volume with a magnitude of 100

For each frame, first two steps are skipped in the single step scenario

  • Set all cubes state to what they were 20 steps ago
  • Step 20 times, recording state after each step
  • Step once to advance time

There is a cost to setting and storing object state, between 4.2 and 5 ms for a 1000 objects resetting once and recording 20 times on my machine. I guess this could be much reduced if rewinding and recording state happened behind an API that allows you to rewind up to a predefined nr of steps. I agree that for this to be truly useful you would need control over what gets rewound and stepped, potentially layers can be used for this?

Physics.Rewind(int steps, LayerMask mask)
Physics.Step(float stepLength, LayerMask mask)

I think spawning and destroying should be the responsibility of the user, if you destroy a gameobject and rewind, it will not be recreated for you. Similarly for (de)activating, if you deactivate a gameobject it does not get rewound or stepped. I dont think unity can mitigate any negative consequences of stepping a subset of objects, so it would be up to the user to avoid or live with these.

2877436–211054–PhysicsBenchmark.cs (2.73 KB)

2 Likes

Very interesting feature but this topic should be exposed better if you want some feedback.
I saw this by accident, there should be a blog post at least or an announcement.
I agree with the above, ideally we should be able to simulate only certain objects.

Wondering if we can tick at editor time, for example the artist places 500 objects, simulates where they fall and so on, it opens up various editor-time physics.

The problem I see with this is you would want to simulate only specific layers. So an editor script would set some objects to layer 31, and these would get simulated. Or otherwise specify something. I’m not really going to use this much, but it’s just general feedback.

3 Likes

This is an amazing first step. We’ve been waiting for this for a long time and like some of the others here have considered moving physics out of the Unity PhysX instance into something external.

I will have a look over the weekend, but first impression is that we need a way to simulate collections of objects in isolation to do any sort of rewind / replay without overloading the CPU. I understand the issues around simulating part of a solver where objects can interact are not trivial, however something like being able to say “these 5 layers are part of physics space A and all other layers are part of physics space B, and A / B don’t interact with eachother at all” would be all we need to get started.

Required interactions between non interacting spaces could be done with non kinematic clones. Though complex, it would still save a heap of CPU time vs rewinding every object. This would also let us do things like partition the world geographically and only rewind/replay small chunks.

Edit: To give some insight into our concrete use case. We would never run the player through PhysX, we think this should always stay as a custom character controller. However a large part our current title Hurtworld is vehicles, which are server authoritative and run on PhysX so we currently can’t reliably client side predict. We currently run them purely on the server and stream positional updates to the clients, adding heaps of delay for anyone > 100 ping.

We don’t require stable stacking, or any sort of predictable realistic results when rewinding and replaying collisions between 2 dynamic objects, 99% of our simulation time will be a single kinematic rigid body simulating against static geometry in isolation. If we get crappy results when correcting a crash, we can hide that with misdirection tricks, our main goal is to allow 0 latency terrain driving.

This is probably better suited to an asset, but it would be nice to be able to cache/pool/disable GameObjects but keep their colliders active in a separate layer/pooled space. Maybe it would be a priority queue indexed by the delta time range/period it was visible. Then we can run the PhysX timesteps against that separate pool to cull and use the (simulation delta time - last/next visible delta time), searching by a range forward/backward, and return a list of GameObjects/periods involved?

Similar to how you might use Vector3 distance in an LOD calculation. I think we would want a similar culling/LOD capability but with time instead of distance. If it’s n seconds in the future, load from a cache, n0.5, prepare a low LOD for viewing, n0.25, switch to a higher LOD … something like this could add utility and avoid stuttering?

It sounds great! This way we could distribute CPU power by using lower fixedStep time for complex mechanical systems, while other parts of the game run with high fixedStep values. I’ll definitely check it out.

If the desired outcome is a callback after simulation and after all collision callbacks, the workaround we use is yielding WaitForFixedUpdate. For years it had been misdocumented as occurring before the physics simulation but that has not been the case on any version I have tried for either physics system and has been corrected on the latest version of the docs.

But yes, this opens up a lot of possibilities. Do your changes currently extend to the Physics2D system?

1 Like

@yant
Any updates on this? After reading the comments in this thread, it seems like all we really need is a way to call the physics simulation on subsets of objects (whether it’ by PhysicsWorlds, layers, per-rigidbody, or whatever plays nicer with PhysX) and we’d be good to go

My personal preference would be the per-rigidbody approach, because that’s what gives you the most control, but I don’t know about the performance implications of this. The next best thing would be to put rigidbodies into different “PhysicsWorlds” in some kind of initialization phase. And finally, working with layers would be my least favorite approach, because it would impose a lot of limitations

1 Like

Wow why is this not posted elsewhere with more visibility? I never knew this was posted until I randomly dropped into this part of the forum (which I normally never visit).

This is very exciting. I’ve been waiting for Unity to have some type of public message about at least looking into this, so I’m glad we have some real evidence now.

Based on what I’ve read on how Rocket League’s networked physics works, they run the full simulation on the server that runs in realtime. They don’t do any type of prediction or anything, they just receive inputs and send states back to each of the clients. The clients, on the other hand, run their own simulation but ahead of the server using client-side prediction. This is pretty typical, except they do client-side prediction even for the other players’ vehicles assuming that their states will remain constant (ie. if one is accelerating, they will continue to accelerate).

AFAIK, on each received state update from the server, a client does a rewind to verify realtime positions/states and what the client had predicted for each vehicle and if any is off then a smoothed correction is made and a fast-forward happens with the client-prediction again.

At least in Rocket League’s case it doesn’t seem like there is a requirement to have physics rewind/fast-forward only on certain rigidbodies, so perhaps this architecture is a good one to test the custom editor given above to at least see if, performance-wise, this can work for networked physics.

2 Likes

I’ve been following the replies here for a bit, but I just wanted to chime in and also say thanks for finally working towards this. It would be a big step towards being able to do proper client-side prediction for physics objects.

Signing in just to say THANK YOU THANK YOU THANK YOU!!!

THANK YOU!

Is there any update on this? Just finding this thread now after wrangling with Bullet Physics and not being satisfied for networked physics. This kind of control with PhysX would be incredible!

Yes please. This is absolutely necessary to do any decent networked physics.

Hi guys,

I’m closely watching what you guys post here, thanks for all the ideas.

Seems clear there is no way around exposing the control over subsets of the scene by allowing to do partial simulations, especially for the advanced users. I think there will be a way figured out on how to achieve that with the current architecture limitations we have.

That said, my current options are quite limited I’m afraid, and I would say it’s pretty much about either shipping what was described in the initial suggestion (amended by fixes for any problems you point out and/or arise in the testing) or, based on the input from this thread, not shipping it at all if it’s concluded as not useful, and waiting for 2017Q4+ until we have a proper time slot to work on that.

Anthony

1 Like

@yant Would it be possible to upload a version with build targets to Windows? Very hard to test for networking with just the editor and no builds.