Hello, I’m using Unity 2022.3 to run a dedicated server with Netcode for GameObjects and WebGL on the client side. I have configured the certificates correctly for both the client and server sides, and bellow you can see the configuration of the transport. I started logging the packet sizes, and they never exceed 19 KB.
However, I’m encountering a few issues. I randomly see this warning:
“A lot of pipeline updates have been queued, possibly too many being scheduled in pipeline logic, queue count: 264.”
After a few games, I then start seeing the following error frequently, and it seems like the server freezes:
“CompleteSend failed with the following error code: -5”
“Error sending message: Unable to queue packet in the transport. Likely caused by send queue size (‘Max Send Queue Size’) being too small.”
This is very frustrating. Any help would be greatly appreciated.
Add the Network(Runtime)StatsMonitor to your scene, and preferably also the NetworkSimulator because using an artificially congested preset may help to trigger the issue more quickly.
But mainly you want to observe the StatsMonitor to see how traffic behaves. And not just traffic but also RPC calls, packets sent and so on (you can configure the monitor to show you nearly everything).
You should look out for continuously increasing stats for no obvious reason (eg the number of synced game objects remain similar). This may indicate a coding issue. You may also be able to find “hot spots”, eg something happens and there’s a huge jump in traffic/packets/RPCs and such.
For example, it’s rather easy to set up what would be an infinite loop on a single machine. But on the network, you can create two RPCs in such a way:
void SendToServerRpc()
{
BoundeBackToClientRpc();
}
void BoundeBackToClientRpc()
{
SendToServerRpc();
}
This will cycle endlessly, happily, because it just keeps on sending back and forth without stackoverflow due to the network latency involved.
Something else to consider is how often you send updates. By default, the server ticks at 30 Hz. So what happens if you send an RPC in every Update() on a 120 Hz monitor? Honestly, I don’t know exactly but I don’t think NGO is going to discard three out of four RPCs. Instead it will likely queue up all four and send them all every fourth frame - or it happens on the receiving side, ie sending is immediate but the incoming RPCs pile up on the receiver side until the next network tick. Now if you have this going for many objects your send queue will be filling up rather quickly.
You can subscribe to network tick events to send only at the network tick rate, or you could set the FixedUpdate rate to match the network update rate and use only FixedUpdate for sending RPCs. Though the latter will also affect physics behaviour.
1 Like
What a great answer. I will add the RuntimeNetStatsMonitor and troubleshoot.
So the problem has to be with the network traffic? Or it can also be a memory leak? asking because I don’t see any error for it to be bad memory management and must be related to network because the packets sent are huge and frequent. (19kb per packet)
Thank you very much for your time, I will get back with the result!
That’s part of the analysis, and I would just start with looking at the network stats. Perhaps these don’t indicate anything wrong, then you need to form a new hypothesis and test that. 
You could also increase the queue size but you already have it at 1024 which I believe isn’t the default. You could purposefully decrease it though to provoke the error faster.
1 Like
Update: The error came back. I also noticed that occasionally I get the following error: “failed to decrypt packet (error: 1048580). likely internal tls failure. closing connection.”, not sure if this contributes to the issue.
Recap:
-Other scripts that run on the same machine run without a problem
-Other functions on the Unity game server run without any issue
-Memory/CPU usage is fine
-Seems like the unity server can’t send or receive any packets, after the warnings on my first post. It’s like the transport send queue doesn’t get freed up.
So I believe this isn’t an infinite loop or a memory leak but something else. Not sure if this is Unity’s Netcode problem but that’s my last resort. Is there a way that I can log the queue size or all packets sent/received on the server?
Ok another update, I recommend anyone who has such an issue to read Pipeline use | Unity Multiplayer
Specifically the sections: “The reliable pipeline stage” and “Maximum in-flight packets”
FYI, I used NetworkDelivery.ReliableFragmentedSequenced option when sending these big packets.
Seems like I am sending too many and huge reliable packets which cause the Head-of-line blocking (HOL blocking) phenomenon.
Or I am reaching the Maximum in-flight packets size which is 32.
So I believe it’s a matter of network optimization now and using message reliability carefully.
Update 2: We found another possible cause and created another post, please check here: Unity NGO BUG
