Multiplayer: Large-scale physics Synchronisation

[Zaphodgame]

Hello,

we have recently tried to implement multiplayer for our game.
We’re using Unity.Entities and Unity.Physics for large-scale battles of 10000+ PhysicsBodies.

After reading a bit through the Internet we figured out how to create a connection using Unity.NetCode and how to instantiate the Entities as NetworkGhosts, unfortunately it does not scale well.
At about 2000 units we are experiencing asynchronous behaviour and the movement sometimes turns into teleportation.
At about 10000 units most of the units are just sliding along somewhere and collisions feel very unresponsive.

We tried prediction and interpolation with both static and dynamic optimisation and had the best results with dynamic interpolation.

Looking at the NetDbg stats we are experiencing very old snapshot ages of 300+, so maybe that’s the problem.

Question 1: Is it realistic to expect a large-scale physics battle to “just work” in multiplayer, or is this idea unreasonable?

Question 2: Why is the snapshot age this high? Is it due to the limited bandwidth of the server and packages fill up too quickly to support this many units?

Thanks in advance for advice, feedback or help!

[timjohansson]

10000 objects is a lot assuming they are actually changing every frame. I would not expect that tot “just work”, you need to optimize and make trade offs to get close to that many entities ghosting correctly.

Server snapshots will only fit a very small subset of the entities in each snapshot, an age of 300 would mean each entity is sent every 5 seconds, and it would mean you fit around 30 entities per packet assuming 10k entities and 60Hz sim/send rate.

Prediction of physics objects is not supported in netcode 0.6 so they need to be interpolated.

Some of the things I would look at to increase the number of entities which can be smoothly updated (in the order I would try them) are:

• If most of the objects are only moving when someone interacts with them and normally are not moving, enable “static optimization” in the GhostAuthoringComponent. This will mean the entities are not sent at all when they are not moving. The quantization factor will impact what is considered “not moving” so you might need to tweak that. If the objects all spawn in mid air and start falling it will take a bit before this has an effect.
• If many of the objects are not visible, setup relevancy to make sure only the visible entities are sent (if the view is top-down 2d this is easy and works really well). Example: https://github.com/Unity-Technologies/multiplayer/blob/master/sampleproject/Assets/Samples/Asteroids/Server/Systems/ShipRelevancySphereSystem.cs
• If the entities are spread out over a large area, setup distance based importance to make sure the things far away are sent less often than the things close to the player (this usually works best in a 3d world). Example: the setup of GhostDistanceImportance in https://github.com/Unity-Technologies/multiplayer/blob/master/sampleproject/Assets/Samples/Asteroids/GameMain.cs and https://github.com/Unity-Technologies/multiplayer/blob/master/sampleproject/Assets/Samples/Asteroids/Server/Systems/UpdateConnectionPositionSystem.cs. You need to tweak the grid size parameters for this to work well.
• If you can reduce the data you synchronize per entity that will help, so look at what you synchronize and see if there is something you can remove or change the quantization of.
• If all else fails, increase how much data you can send per frame by adding a NetworkStreamSnapshotTargetSize component to the connection entity and set the value of it to something high (IIRC max supported is 16384). This will introduce other problems like high bandwidth usage, worse handling of packet loss and higher CPU cost on the server.

[Zaphodgame]

Thank you very much for the info.
We will look into it some more and try the things you recommended!