Netcode - ghost destruction causing pack size exceeded error

Unity Engine
1

Hi guys, I have run into a critical issue with netcode 4. I haven’t tested on 5 because I have never managed to get that to run smoothly with the netcode sample.

Issue 1 is that ALL destroyed entities are sent in an entire frame which can exceed the max packet size and result in the error “It is not possible to send packets larger than {0} bytes with this pipeline and driver. Trying to send {1} bytes.” This happens to me quite often as I have a large amount of projectiles that despawn frequently. If too many despawn at once the error happens. I can mitigate this by buffering out my entity.destroy events, only registering a certain amount per tick (which is not great).

Issue 2 is that entities hidden by SetGhostRelevancySystem seem to run Destroy on the client entity and once again this can exceed the packet size if the camera moves to quickly and makes too many entities irrelevant at once. I guess I could buffer irrelevancies out as well by rewriting SetGhostRelevancySystem.

Issue 3 is the most critical. Even if EVERY entity is marked as irrelevant, and the client should have nothing, the client still gets notified about Destroy events. So if I move my camera away from every projectile, and no projectiles are being sent to the client, and the client has never ever had a single projectile - it is still getting 5k destroy events in a single frame - about entities it never knew about. And this of course raises the same error about packet size being exceeded. You can see this issue illustrated in the attached screenshot where (1) the clientworld only has only 2 ghosts, and you can see in netdbg (2) the client never had any entities sent yet (3) there are 5k destruction events sent and (4) the error occurs.

Issue 4 is how destruction hogs bandwidth when there is lag. It appears as though destruction messages are resent each frame until they are acked. If the lag is large (say 500ms) then the destruction messages end up building up and consume all bandwidth, leaving no room for anything else.

In my project issue 2 is so bad that it crashes 100% of the time when SetGhostRelevancy is enabled for projectiles. If I just send all projectiles to all clients all of the time, it runs well, but I need that bandwidth for other things.

@timjohansson do you guys know anything that can help?

Thanks for your help!

6631009--756133--netcodeissue.png
6631009--756133--netcodeissue.png1919×1080 468 KB

2

To summarize:
It sends all manual destroys at once, exceeding badwidth
Relevancy system auto-destroys all at once, exceeding badwidth
Destroy signals are sent about entities that the client doesnt know about, exceeding badwidth
Lag causes destroy signals messages to be sent too often, hogging bandwidth

3

The behavior for this is changed in 0.5, there it should switch to sending a fragmented packets instead of failing to send, which will increase your bandwidth requirement instead of completely breaking.
We do have a task to split up the despawns to reduce the bandwidth cost in this case - but it has not reached our top prio yet. This would also solve issue 4, but would add more latency for ghost being destroyed on the client. It would be good if you can report a bug about to make sure we take that into account when prioritizing.

The logic is not based on what the client has received but what the server has ever sent. Looking at the code it does look like we are missing a special case for entities which have never been sent though, would be great if you can file a bug for this.

4

Thanks for the quick response. Can you please confirm exactly which issue numbers you want bug reports for?

Did you want me to make a stripped down project and submit it with the Help->Report a Bug tool? I can do it but just checking because it will take some time.

5

3 and 4, the errors are fixed in 0.5 but the bandwidth can still be high if you despawn a lot.

The Report a Bug tool is the right place to file it.
Thanks!

6

I reported issue 1297556 a couple weeks back for 3 :slight_smile: We use 0.5, but it still exceeds the 16K limit of the fragmented driver.

Also, in general we can see that ghosts tend to stutter a lot while the relevancy set is changed, but I haven’t looked too closely at that, but that could be 4 we are seeing.

1 Like