Netcode entities non-trivial client prediction use case

Multiplayer & Networking Netcode for Entities
, , ,
1

Hi, I spent a long time trying to get client prediction working for a RTS game I’m making but the result is not very promising. I checked out many sample projects but all application of client prediction is very trivial (eg. player presses a button and the character moves). This is different from RTS. For example, I have an (client-only) input gathering system such that after I issue a (discrete, as opposed to in the samples a continuous arrow key press) move command to one or more units using IInputComponentData with a mouse click, a path finding system would run, and afterwards a movement system would run according to the path finding results. The input system is run in GhostInputSystemGroup and the pathfinding and movement system are in PredictedSimulationSystemGroup. All relevant systems and jobs are marked with simulate tag, and all relevant entities/units are predicted ghosts.

Expectation: Even when average ping is very high, say for example 500ms, client is still able to have instantaneous feedback. That is, after the mouse click, path finding system on the client runs and immediately generates a path according to client’s local data, and the movement system on the client also runs immediately after and directs units to move using the path generated. Of course the server would also run the exact same systems, and rollback if necessary. Assuming the player is not cheating, rollback should also not happen even with the insane latency since server state would match client state (eventual consistency).

Reality:
Problem 1: Experiments under such high ping showed that units did not respond immediately. It’s still behaving as if there is no client prediction at all. That is, they only move after a very long noticeable delay.
Problem 2: Movement is very jagged/jittery. Unit are constantly teleporting back and forth, and after reaching the destination they always glitch once. It looks like client and server are constantly fighting (this is even noticeable with 0 ping in testing, and of course a lot worse under high ping).

I don’t know what I am doing wrong, or if this is even a valid use case that netcode entities support. Would appreciate advice/deep dive of non-trivial use case of client prediction, especially in the context of RTS games.

2

Hey @InSpirationhh. Netcode for Entities isn’t well suited to RTS games, and client prediction makes this even less viable. However, for a small number of total ghosts, this does sound feasible.

From the sounds of it, you’re likely getting misprediction errors due to code incorrectness. Recommendations:

  1. The IInputComponentData must be added to each predicted ghost, and the local player must have ownership over them, to ensure snapshots received causes the correct ghosts to rollback.
  2. The pathfinding waypoint data must be marked with [GhostField] to ensure rollback & resimulation correctness. You can use SendData=false if you’re sure you can deterministically reproduce said path on the client, reducing bandwidth consumption significantly.
  3. I’d recommend disabling the pathfinding temporarily - and simply have the ghost move straight towards the input destination, to start from a place of correctness.
    • I’d also recommend temporarily disabling static-optimization on the unit prefab, if it’s enabled.
  4. Mispredictions should only occur if:
    • A desync occurs leading to the server to not receive your input.
    • Another players units movement blocks or interrupts your own units movement - as you wouldn’t be able to predict that.
3

Hi @NikiWalker,

  1. Currently I have a singleton ghost entity that is used to issue command to units the player controls in the game. It has an IInputComponentData attached and is used to inform subsequent systems (eg. pathfinding and movement etc., all run in PredictedSimulationSystemGroup) which entities to operate on. The unit entities themselves do not each have an IInputComponentData.
    (Q1) Just want to confirm this is not the intended design and each unit should have their own IInputComponentData attached, and every player input must be updated on all of the player controlled entities?
    (Q2) What if each unit entity is logically linked to other entities, do those linked entities also must have IInputComponentData as well? (To elaborate, by linked I mean either the unit entity has an IComponentData that references another entity, or via LinkedEntityGroup).
  2. Path finding way point data (an IBufferElementData) is already marked with [GhostField].
    (Q3) If SendData=false which I assume just means to not replicate this data to the server (computed locally) how does server prevents cheating or perform correction on client predicted actions?
    Please feel free to elaborate on any misconceptions I may have or how Netcode entitieis is designed to work. Thank you very much for your response!
4

Some other quick follow ups:
(Q4) In an owner-predicted ghost setup (e.g., using a pathfinding buffer or similar owner-specific component data), which interpretation of the SendToOwnerType field is correct?

  1. SendToOwner: The owner is the only authoritative or interested party for this data; non-owner clients should not need this intermediate information.
  2. SendToNonOwner: The owner already possesses the data and only non-owner clients require synchronization of this component to replicate the owner’s state.

(Q5) Also what’s the point of the None type? If we don’t want to replicate anything it should not be a ghost field in the first place, as opposed to a ghost field with type being None? Could you provide a valid use case/example of each of the four available types?

(Q6) According to the doc SendToOwnerType is only used with IInputComponentData, or it can also be used in IComponentData? Any difference?

(Q7) Regarding IComponentData replication requiring a ghost field, does the same apply to IInputComponentData? My hunch is that IInputComponentData already manages continuous client-to-server input streaming by default, so making IInputComponentData ghost field is only for when other clients need to predict this player’s input?

Thank you!

5

Niki is on vacation but I’ll try to reply.

So, 500ms delay will result in a lot of rollbacks as it needs to re-simulate so many ticks with that kind of delay (depending on tick rate). Also the rollbacks will always happen on all the ghosts received in the snapshot, it doesn’t just run on mis-predictions. Maybe ok for small amount of ghosts.

(Q1) Just want to confirm this is not the intended design and each unit should have their own IInputComponentData attached, and every player input must be updated on all of the player controlled entities?

It should be on each ghost locally controlled. But it also depends on what’s inside the input data (does each ghost/unit have control data to place in the inputs?). I’m not sure if it’s required for ensuring correct ghost rollback for this particular usecase, but Niki will need to follow up on that one. I’d say this isn’t really suited to your usecase (RTS style) but could work with a small number of ghosts like Niki mentioned. The inputs are not really designed for this and also the client side prediction rollback will be very costly with RTS style amount of units.

(Q2) What if each unit entity is logically linked to other entities, do those linked entities also must have IInputComponentData as well? (To elaborate, by linked I mean either the unit entity has an IComponentData that references another entity, or via LinkedEntityGroup ).

Normally linked entities like this should go into a ghost group together.

(Q3) If SendData=false which I assume just means to not replicate this data to the server (computed locally) how does server prevents cheating or perform correction on client predicted actions?

It means not replicate the ghost field data from server to clients. So affects the server authoritative data and is not meant for inputs. Also, normally client predicted simulation will only be visible on the client itself, they do not affect the server simulation at all, so they won’t be able to cheat or nobody will see what they did at least (will just result in mis-predictions).

(Q4) In an owner-predicted ghost setup (e.g., using a pathfinding buffer or similar owner-specific component data), which interpretation of the SendToOwnerType field is correct?

Those are both correct, SendToOwner = server will send this to only the connection which owns this ghost and SendToNonOwner = server will send this to everyone except the connection which owns this ghost. Maybe I misunderstand what you mean.

(Q5) Also what’s the point of the None type? If we don’t want to replicate anything it should not be a ghost field in the first place, as opposed to a ghost field with type being None ? Could you provide a valid use case/example of each of the four available types?

  • None is a special case, it’s used internally for the DontSerializeVariant case for example. Like when there are child entities which have ghost data which should not be sent to clients.
  • All is the default, synch ghost field data to all clients which probably applies to most data.
  • SendToNonOwner can be used for example with predicted remote player inputs. When the public fields in an input struct are marked with the GhostField attribute it will treat the input command buffer like any other dynamic buffer and synchronize it to all clients. Marking it with SendToNonOwner makes sure only other players get the data to use for input prediction as there is no need to make the client which owns and sent the inputs to the server to receive them back from him.
  • SendToOwner can be used for example for data you want the server to control or at least have authority over but only send it to the owning client as it’s sensitive/secret and other clients should not see it.

(Q6) According to the doc SendToOwnerType is only used with IInputComponentData , or it can also be used in IComponentData ? Any difference?

It’s not just for inputs, I think the remark in the docs Typically used by IInputComponentData structs to replicate each clients inputs ONLY to other players. is actually just describing the SendToNonOwner scenario I described above. For example SendToOwner doesn’t work on inputs (send the client back the inputs it already sent to the server).

(Q7) Regarding IComponentData replication requiring a ghost field, does the same apply to IInputComponentData ? My hunch is that IInputComponentData already manages continuous client-to-server input streaming by default, so making IInputComponentData ghost field is only for when other clients need to predict this player’s input?

Yes, correct. IInputComponentData will replicate the input struct from client to server only by default. If you mark the fields with GhostField it also replicates to other clients for input prediction.

6

Thank you for your reply. I would like to follow up on the comment regarding whether “each ghost/unit has control data to place in the inputs.” Specifically, what should the per-entity IInputComponentData include?

Suppose I only place the destination coordinates into the IInputComponentData, which is set within a GhostInputSystemGroup system in response to a player’s mouse click. Consequently, each entity has destination data in the command stream, and this data can be used as input for downstream predicted systems. However, this data requires processing to be useful, and the resulting output is not an IInputComponentData. For example, a pathfinding system would take the destination as input and generate a path, but the path itself is a dynamic buffer, not IInputComponentData.

Q1: I understand that NetCode for Entities is not typically designed for RTS games, though I hope this is primarily due to scaling concerns. To use an extreme example: if there were only one unit in the entire game session, would client prediction work for the pathfinder? (To clarify, in NetCode samples, the IInputComponentData is used directly to control character movement direction without further processing; in this case, however, it does need to be further processed).

Q2: Suppose the pathfinder is a predicted system that runs on both the server and the client. Should the generated path be a dynamic buffer?
If so, should it be a ghost field? But snapshot update, assuming it happens in this frame, is still not instataneous, which defeats the whole purpose of client prediction?
If not, then even though client and server can compute individually, how does roll back work?
What is the recommended approach to make this work?

Appreciate your support and looking forward to your input!

7

Well, in that case, which Netcode solution is suitable for an RTS in which 70-100 synchronized objects are present at the same time? (there may be about 5-15 for each of the clients)

8

Specifically, what should the per-entity IInputComponentData include?
Suppose I only place the destination coordinates into the IInputComponentData , which is set within a GhostInputSystemGroup system in response to a player’s mouse click. Consequently, each entity has destination data in the command stream, and this data can be used as input for downstream predicted systems. However, this data requires processing to be useful, and the resulting output is not an IInputComponentData . For example, a pathfinding system would take the destination as input and generate a path, but the path itself is a dynamic buffer, not IInputComponentData .

Right, you could put the direction or target coordinate of each unit. Normally this would be controller input like forward, lef, shoot and this is the equivalent, as in this is what the unit is doing atm until the user changes it. This makes the most sense where the server is just running the navigation, then the client is just sending his coordinate or direction and it’s processed as normal, it’s just coming over the network. The result of the navigation is then replicated to all clients (including the owner).

But with client side prediction added it gets trickier. We don’t really support this or have any solution made for this. You’d need to be able reset the navigation system in the prediction loop and then re-navigate the path it gives you up to latest tick. Maybe it’s enough to only reset at the first prediction tick.

Q1: I understand that NetCode for Entities is not typically designed for RTS games, though I hope this is primarily due to scaling concerns. To use an extreme example: if there were only one unit in the entire game session, would client prediction work for the pathfinder? (To clarify, in NetCode samples, the IInputComponentData is used directly to control character movement direction without further processing; in this case, however, it does need to be further processed).

It actually scales easily to RTS number of units, you can try out thousands of asteroids in the NetcodeSamples project. However, normally RTS games do not synchronize thousands of units, they depend on determinism and just needing the clients and server to process the same inputs, then they simulate exactly the same output for those inputs, then you don’t need to synchronize actual unit position/etc. No need for any rollback as it’s always identical outputs.

Client prediction needs something close to determinism to work (to minimize mis-predictions) but doesn’t require 100% accurate determinism. How it works with a pathfinder depends on the implementation.

Q2: Suppose the pathfinder is a predicted system that runs on both the server and the client. Should the generated path be a dynamic buffer?
If so, should it be a ghost field? But snapshot update, assuming it happens in this frame, is still not instataneous, which defeats the whole purpose of client prediction?
If not, then even though client and server can compute individually, how does roll back work?
What is the recommended approach to make this work?

I’d think you would synchronize the result of the simulation, so where the ghosts end up after the navigation is done and has moved it already. Rollback will be the tricky part, like I mentioned above, depends on what kinds of implementation you have. To actually rollback you’d need to be able to reset the navigation or pathfinding to the current state of the ghosts (the prediction system will roll the transforms/ghostfields back automatically), then redo the navigation according to the input/direction for each predicted tick.

I know this isn’t very clear and unfortunately we don’t have a recommendation for this at the moment. The easiest is to not do client prediction and only run the navigation/simulation on the server. Only use interpolation, then it can scale to a lot of units and will look perfect as they play back on the clients. The usual way of hiding the latency is to have some delay where some animation plays or something happens to let you know the command was registered before the actual movement arrives from the server.

Well, in that case, which Netcode solution is suitable for an RTS in which 70-100 synchronized objects are present at the same time? (there may be about 5-15 for each of the clients)

That scale is probably ok with the methods discussed so far, with just interpolation it would be fine, with some movement logic which is rollback compatible those number of predicted units (5-15) is probbaly ok too. With RTS I was thinking thousands of units, and the traditional way of doing that is with lockstep determinism. That’s not just the netcode but more like the engine needs to support that, at least every feature which affects the simulation. There is also a delay there (like with interpolation), while inputs are gathered, but then the simulation is 100% deterministic everywhere. You could do that with Photons Quantum solution afaik, but maybe others can suggest other solutions like that.