Netcode for GameObjects Custom Client Prediction Tips

Unity Engine
, , , ,
1

Hey there,

My goal has been to develop a co-op multiplayer game in Unity using the Netcode for GameObjects package. I discovered along the way that there are latency problems when using the NetworkTransform and NetworkRigidbody. When it comes to real time physics interactions, particularly when bullet impacts are supposed to affect the knockback of enemies, these components do not offer a real time influence on such objects. So, this led me to research client prediction and server reconciliation. If i’m missing something about these components, then please educate me, however I do not believe they provide a solution to the real-time physics problem.

Since I do not care much for cheaters, my network reconciliation approach became a simple problem. All I have to do is spawn my enemy once on each client, check for differences in position and velocity between client and sever, then correct the differences on all client instances when variance becomes too large. I do not do any prediction at this moment, so when I re-adjust a clients’ position to match the servers’, enemies will sometimes lerp “backwards” in time to a previous place, causing a little jitter.

My code is not very clean, but I have provided my class, ReconTransform, which does the “reconciliation” for enemies. In short, ReconTransform achieves sub-optimal synchronization by:

  • CheckForMovementReconciliation(…) is called on an interval and takes a function. This function determines the rules which dictate the need to resynchronize. For example, this function can tell the ReconTransform that a resync needs to occur when client and server representation are X distance apart. The function also provides a float that determines the ideal distance the the objects should be from each other when recon is complete.
  • AddForce(…) is a function that mimmics Rigidbody.AddForce(…) but will cache physics interactions during the lerping phase. When resynchronization occurs, all physics interactions are paused. During this time, physics interactions are cached and re-applied when the lerping/recon is complete. Lerping just means smoothly repositioning the client representation to match the servers representation. I use smoothdamp now instead of lerp.

Please help me make this package better. I don’t know what I’m doing wrong. The recon action is too jittery. In a real time situation where I am shooting a bunch a bugs for example, as a client I want to be able to aim at the enemy without it suddenly lerping to the correct position breaking emersion and frustrating players by ruining their expectations of enemy behavior. When I am up against an enemy that can kill me in two hits, I do not want to abruptly lerp the enemy away from a potential player projectile that could kill it. Anyway, here’s the repo:

2

No, it’s latency that prevents anything networked to be real-time. It’s always going to be (real-time + latency).

Correct.

You’re trying to predict and reconcile physics. That’s about as hard as it gets. It’s the quantum mechanics of software engineering, sort of. :wink:

The approach is backwards. Reconciliation is for competitive games (mostly). Competitive games don’t actually use networked physics for gameplay, only for visualization (few exceptions like some party games with intentionally funky physics).

It will not work correctly with physics precisely due to latency because the body that you add a force to, will be in a different world / place / pose when the addforce is applied to the body in the remote side. So the force applied will be incorrect, by a small degree but this will add up. This means you’re always playing catch-up and jitter is unavoidable.

It’s similar to having an object walk towards a position in discrete steps and trying to make it match the target position. It overshoots, turns around, and overshoots again. Solution: allow a “close-enough” range. Same with physics - there’s a limit to how accurate you can (and have to) represent anything on the remote side.

For coop games, the plain and simple solution is to allow each client to run its own physics simulation. Ideally it should behave predictably (deterministic) but more commonly you can simply allow for some deviation and only correct things that are far out of place, eg a ragdoll that’s more than 1 unit away from where it ought to be.

3

Client prediction and reconciliation is one of the hardest problems in networking, there is no universal solution and most games work around it rather than trying to solve it.

The real problem here is authority, determining which player has authority over a given action and how the other players should see that. Take your example of a bullet impact providing knockback to an enemy, most games won’t do this with actual physics and they definitely wouldn’t have each client do it independently and then reconcile the results with the server.

Instead, they would have the knockback calculated by one person and then transmit the results to all clients. The knockback might be the server’s responsibility, it might detect the bullet hit and then in a script calculate the knockback and provide a physics impulse to the enemy’s rigidbody. The results would then be automatically transmitted to everyone if the object is using NetworkRigidbody and NetworkTransform. NetworkTransform synchronises the position, and NetworkRigidbody helps synchronise slightly better if the object has a rigidbody. This is also why rigidbodies default to kinematic on clients when using NetworkTransform etc, the server is supposed to be in charge of it.

The tricky bit is dealing with lag, there are actions that you want to visually happen on the client instantly for that sharp visual feedback but then you transmit them to everyone else. You may notice that the knockback is slightly delayed if it’s processed by the server for example, and might want to make it a client-authoritative process where the knockback happens instantly for the shooter and is then transmitted to everyone else.

Ultimately, very very few online games use real synchronised physics. I’m working on a heavily physics-based online game and even we don’t synchronise our physics 99% of the time, we work around it and fake it. We have most of the physics performed locally and only transmit the locations and vectors of large forces, collisions, impacts, and destruction.

4

You mentioned it twice already. I’m curious, is there anything online to watch or follow?

5

Sure! There was a first look video posted here: x.com

The level contains thousands of physics-based objects, and the trick is honestly that we don’t bother synchronising the objects themselves. When a player performs a physics action like bumping into a table, they apply a physics bump in a location and then that gets transmitted to everyone else so they perform the same bump in the same location. The results are close enough to make no functional difference between players, and every player sees their own interactions happening locally with no delay.

6

We solved this problem with distributed physics. So the client calculates all its owned physical objects. And if its objects touch items in the world ownership will be transferred from server to client.

This makes it smooth on the client doing the interaction but it can be a lag for third party clients before the new clients start to send sync data for object in question. A trade off that works pretty well

You can see it in action here in our trailer when a team throws a mag to the player. This is an old video and we have redone the entire netcode since then but the result is more or less the same

Here is another old vidoe showing how recursive ownership is working (the door is owned by player the items on the floor by the server).

7

Hello again CodeSmile.

Firstly, thank you for the detailed reply.

A few months ago I posted about starting this system (which you replied to - and thank you for that guidance). This system has been on the shelf for some time now. I want to use it, but I feel as though I am simply wielding a wimpy rapier against a deadly dragon.

I did see that unity 6 had some important simulation updates. I saw that unity 6 was going to include an AnticipatedNetworkTransform which may help with what I’m trying to do. Any thoughts on this?

8

Hey there AndersMalmgren,

I was thinking about this solution and how much simpler it would be to implement. Do you have any tips to avoid the lag when switching ownership?

Also, when it comes to fighting off a hoard of physically-based enemies, do you think that the ownership switch would be frustrating to deal with? I’m hesitant to move forward with this approach because my enemy hoards will sometimes require real-time reactions and precise aim to avoid perma-death.

My other thought is to look into Entities, but that would mean refactoring tens of thousands of lines of code.

I’m stuck…

9

How would I give authority to the client network rigidbody? By changing ownership? I know there is a way to override the Network Transforms client authority so allow for responsive movements, but I’m not sure if this is the same matter for the rigidbody. I’m searching for guidance.

10

You can only minimize the ownership switch delay. For example directly when a client touches a item he starts sending his sync data for that item. But server havent received the ownership request yet. He might loose that ownership if another player got to the item first. Classic client side prediciton. Without it it wouild take new owenerrs pingx2 until third party players start to get the new owneship synced data. Now it takes owner ping x 1.

There are some sideefffects we havent bothered with. For example if a player throws object at a player and a dynamic item is behind the receiving player like a door and the receiving player catches the item, if his ping is so low that it will hit the item on the other side before taking ownership all players will see the other item move because we havent implemented recursive rewind.

Also interpolation introduced even more lag so I recommend having minimmum ping implemnted.

11

Typically by changing ownership as AndersMalmgren is suggesting, that’s the typical way to do it in a distributed authority model. NGO will automatically make the rigidbody non-kinematic on the new owner so he can affect it as usual through physics or applying forces in code and it will automatically transmit all location changes to all other clients.

This has the benefit that client 1 sees instant results, and while other players will see some lag, they’re seeing that lag in client 1’s actions anyway so it looks fine. The main down-side is that there is a large delay when changing ownership and only the server can do it, to keep the delays down you have to speculatively change ownership as early as possible. It’s also a huge mess if you have multiple clients fighting over ownership (e.g. two people shooting the same enemy).

There are two alternatives I’ve used that can work depending on your game design too:

Deterministic with Server Authority and lag compensation:
Use a NetworkTransform and NetworkRigidbody to sync the object’s position, and have all collisions and physics done server-side, but deterministically. E.g. if the client is shooting, the same exact shots with the same exact timings and pattern also happen on the server, and that’s where the knockback is executed.

The player who initiates the action will perceive a lag in the results of their actions, but depending on the game design it’s possible to compensate for this. For example, you may be able to give the weapon a short wind-up animation or make it so that physics effects like knockback happen every 2-3 hits or you could give the weapon some visual travel time.

Anything you can do to add a short delay before the physics impact is expected would give you the room to do lag compensation. When the server receives the instruction to start firing, it can skip ahead in the firing process by a duration equal to the player’s ping and do the physics bump early. The player would then see the physics bump timed perfectly with the shots as if it happened locally.

Local physics and sync:
Don’t use a NetworkTransform or NetworkRigidbody to sync the object’s position at all. All clients perform their own physics on the object and you just try to keep that physics and any random elements deterministic. e.g. if player 1 is shooting at enemy 1, that info is sent to all players and they all simulate the same exact hits from player 1 to enemy 1 locally and apply the same knockbacks.

This has the advantage that all players visually see instant results regardless of their pings and perspectives and everyone will see roughly the same thing. It has the major disadvantage that things can get out of sync depending on lag and the complexity of physics interactions. This method is actually very effective for games where you need lots of instant-feeling local physics simulation but don’t actually care that much about the positions of objects being perfectly in sync between clients.

You might want to implement your own sync system for this where clients periodically check in their positions of the objects and then all re-sync if it’s too out of step with each other. Honestly it depends on the game design though, I was mid-way through implementing a background/offscreen sync system when we realised the desync was minimal, didn’t affect the gameplay at all, and none of the testers actually noticed it.