I’m working on a 2D game with potentially many units on screen (200-300 at maximum). I’m encountering a problem with the synchronisation of my units positions. I’ll try giving as much info as I can, let me know if you need more to help me diagnose this problem.
I’m using NGO, Unity Lobby Services and Unity Relay.
I tried 2 methods:
NetworkVariables directly on the Unit object, only updated on the server when a certain treshold is exceeded. Packet size is 9B for a single unit synchronization, updated 3-5 times/s.
The units all have a “troop” parent gameObject with a NetworkObject too so I tried sending their updated positions through the troop object by RPC. In this scenario a message contains a Vector2 with 5 unit positions. Packet size is 53B for 5 unit positions, updated 4 times/s.
Here is a screenshot of the profiler when the game is running fluently on the client.
Even though I could optimize further this system, what bugs me is that these values don’t seem excessive and shouldn’t lead to this problem ? When I test in local without using Lobby Services, I face no issues even with 300 units.
How do you actually synchronize them? Your code would be helpful. It could be a simple bug in your code after all.
Generally if you write a system that only periodically sends updates it’s best to subscribe to NetworkTickSystem events so that you can check and send as ticks are processed. This may alleviate the buffering of the send queue over several updates.
You may also want to try raising the UnityTransport values. Although if you’d hit those limits I’m sure you’d be seeing console messages.
Hey CodeSmile, thanks for your answer ! Here’s for the Position Sync code in the “troop” parent object :
private void HandleAllUnitsPositionSync() {
Vector2[] allUnitPositions = new Vector2[allUnitsInTroop.Count];
for (int i = 0; i < allUnitsInTroop.Count; i++) {
allUnitPositions[i] = allUnitsInTroop[i].transform.position;
}
HandleAllPositionsSyncServerRpc(allUnitPositions);
}
[ServerRpc(RequireOwnership = false, Delivery = RpcDelivery.Unreliable)]
private void HandleAllPositionsSyncServerRpc(Vector2[] allUnitPositions) {
HandleAllUnitsPositionSyncClientRpc(allUnitPositions);
}
[ClientRpc]
private void HandleAllUnitsPositionSyncClientRpc(Vector2[] allUnitPositions) {
if (IsServer) return;
for (int i = 0; i < allUnitPositions.Length; i++) {
allUnitsInTroop[i].SetPosition(allUnitPositions[i]);
}
}
I tried raising the UnityTransport values, that didn’t solve the problem. I’ll try subscribing to NetworkTickSystem events then and let you know ! Thanks again
That makes 8 * 150 = 1,200 bytes in a single RPC. Although this should fit in a packet or two. But a NetworkList may be more efficient for that.
How do you control how often these updates are sent?
If possible, try interleaving these updates. So if you send this 5 times per second but the tick rate is actually 20 times per second you could send the updates of 25% enemies each in subsequent ticks.
The golden rule for netcode is to send as little data as physically possible to achieve the desired result, even if you have to cheat. There are ways to achieve sync between clients without actually sending the raw positions of hundreds of units, all you have to do is ensure that every client moves the units locally in the exact same way deterministically. What methods will work depend entirely on your game design, but a few suggestions include:
Define behaviours or actions for enemies, and then just send an RPC when an enemy changes behaviour or performs an action. The behaviours would have to be deterministic. If there are random elements to their movement, give each unit a different random seed and send that to all clients to synchronise that randomness across all clients.
If it’s an RTS, have the server calculate the pathing for a unit when the move order is issued and transmit that to everyone, then everyone moves the unit along that path themselves.
Use group movement behaviours, where an entire group of units acts as a single unit, then just send the group’s position and have all the units move relative to it. In your example it would be like moving the Troop, but instead of the Troop sending the 5 positions of its children it sends only its own position and those children are always moved relative to the parent.
If this is a massively online game where you have 200-300 actual players and so need to sync their actual positions, consider trimming the list to only those whose positions have changed since the last update by more than a certain threshold. The goal is to reduce the amount of redundant position data you’re sending.
For example, I’m working on a game with 10,000 physics objects that are synced between clients, and the trick is that they aren’t synced at all. Everyone does their own physics locally, and only player interactions with objects are synchronised. For example, if a player throws an object we don’t need to synchronise its position constantly but instead just transmit the start position and force vector of the throw to all clients and they all execute the exact same throw locally and see the exact same results.
Which would be far beyond the scope of NGO. It’s advertised as up to 8 players. I’d say 100 is achievable out of the box (that’s Relay’s limit so I’m just using that number) but going beyond, though imaginable through excellent engineering, you may not be using much of NGO anymore but rather custom messages all the way with custom optimizations.
Absolutely! I was more getting at the fact that if the game design really does call for realtime synchronisation of hundreds of independently moving and unpredictable units then there’s still some saving to be made by removing any redundant data and only sending positions that have changed.
But I’d wager that the game doesn’t require that much data to be sent. It sounds like the units may not be completely independent as they’re organised in groups, and their movements are probably predictable. They almost certainly don’t need to be sending the actual positions of all the units 5 times per second.
Sorry if I didn’t make myself clear enough. The code snippet I shared is actually located in the “troop” parent object, which handles 1-20 units. So the packets weighs 40B for a 5 unit troop, and is sent, in my latest attempt, 4x per second.
I also tried making an independent component which would handle the unit’s positions synchronisation in bigger batches, having each batch reach 500B (I read in the forums it was the optimal packet size).
None of these optimizations solutions worked. I should however try spreading out the updates on different frames as you mentioned. I tried randomizing the intital value of the timer that controls the “troop” units synchronisation in order to have them synchronised on different frames, but that didn’t work eighter.
Despite the remaining optimizations I could implement (network list, subscribing to network ticks…), I feel like for 150 units and with my current systems the network should be processing fine… right ? Here’s a screenshot of the Runtime Network Stats, right at the moment the network breaks, client side.
I did think about having each client run it’s own simulation. The problem is : I’m using the Unity 2D physics system. As they are not deterministic, attempting to make the game deterministic is too risky for the efforts that implies.
So I’m stuck with syncrhonizing positions, even though at some specific moments of the game I can predict unit’s movement client-side and stop synchronizing their positions.
And yeah there is room for optimization like synchronizing units only when a treshold is reached, but for testing purposes I need to make it work in the words-case scenario, which is when 300 units positions must be syncrhonized.
If you’re not able to alter the game design (e.g. changing how grouped units behave) and you absolutely must have full sever-side physics simulation on each unit and synchronise 300 units’ raw positions, then the only things you have to work with are data structure, data precision, and frame timing.
Your approach of an independent component that handles synchronisation in bigger batches to match the optimal packet size should have worked better, it’s a solid approach. I agree that for 150 units there shouldn’t be an issue, have you tested it in a build to rule out it just being Unity editor issues?
I also wonder if the main issue is just that Unity’s relay servers are not that reliable and your system is extremely visibly sensitive to delayed packets. In a system where you’re manually setting the position several times per second, it’s going to be very noticeable when a delay happens. Can the visual issue be covered up with client-side movement prediction between received frames?
One more suggestion: have you tried plain and simple NetworkTransform? This has several internal optimizations, and you can further optimize by unchecking what gets synchronized, and synchronizing only half-float delta values (two checkboxes combined, not obvious, see tooltips).
If anything, you’d have a way to compare performance and traffic against the regular workflow.
Unfortunately I really must synchronise the raw unit’s positions. I did test in in a build, I also tested it on two different computers.
My guess is on Units relay servers, I didn’t find any other explanation as of today. I’m already using client-side movement prediction. The intial issue is not really a delay issue, more of a litteral network breakdown on the client side. The client will receive the sent packets once every 3 seconds, in the order they were sent by the server. If I didn’t have movement prediction, visually the units would just stay at the same position the whole time.
I did try the NetworkTransform before manually synchronising the positions, but I dropped that option before checking the network stress because I didn’t manage to get smooth transitions between positions client-side when I set the treshold to high values like .5 (which results in 2 synchronisations per second).
I just gave it another try, the network breakdowns even earlier than by synchronising manually (at 100 units).
If you have any other suggestions, I’m all ears. It’s really nice to have external feedback and advice on that problem
Well, since you haven’t mentioned versions yet (or I didn’t see it), try upgrading NGO and the Transport. Perhaps you’re simply running into a buffer overrun bug of some kind.
