NetworkPacketOverflow with 525 bytes using Fragmented Pipeline

Unity Engine
1

I’m working on a data exchange system to pass live player data and multiplayer game invites between multiple concurrently running game servers. So far I’ve been unable to send decent size packet through fragmented reliable pipeline.

EDIT: tested with a short message of just a few bytes, same issue. Sending any data from DataExchangeClient to DataExchangeServer doesn’t work but sending opposite direction works.

Here is the code that initializes NetworkDriver and the pipeline:

        var settings = new NetworkSettings(Allocator.Persistent);
        settings.WithFragmentationStageParameters(payloadCapacity: 64000);
        settings.WithReliableStageParameters(windowSize: 64);
        m_Driver = NetworkDriver.Create(settings);
        reliableSequencedPipeline = m_Driver.CreatePipeline(typeof(FragmentationPipelineStage), typeof(ReliableSequencedPipelineStage));

Here is the code that tries to send serialized invite.

    public void SendInvite(OnlineInvite invite) {
        var logId = "SendInvite";
        var connectionState = m_Connection.IsCreated?m_Driver.GetConnectionState(m_Connection):NetworkConnection.State.Disconnected;
        if(!m_Driver.IsCreated || !m_Connection.IsCreated || connectionState!=NetworkConnection.State.Connected) {
            loge(logId, "DriverCreated="+m_Driver.IsCreated+" ConnectionCreated="+m_Connection.IsCreated+" ConnectionState="+connectionState+" => no-op.");
            return;
        }
        var sendData = MiniJSON.Json.Serialize(invite.Data);
        var messageLength = sizeof(ushort)+Encoding.UTF8.GetByteCount(sendData);
        m_Driver.BeginSend(reliableSequencedPipeline, m_Connection, out var writer, messageLength);
        writer.WriteUShort((ushort)DexMessageType.OnlineInvite);
        writer.WriteVarString(sendData);
        var sentBytes = m_Driver.EndSend(writer);
        if(sentBytes<0) {
            var statusCode = (Unity.Networking.Transport.Error.StatusCode)sentBytes;
            loge(logId, "Failed to send! WriterLength="+writer.Length+" MessageLength="+messageLength+" Conn="+m_Connection+" Driver="+m_Driver+"; SentVal="+sentBytes+" StatusCode="+statusCode+" DriverSettings="+Strings.Dump(m_Driver.CurrentSettings)+".");
        } else {
            logd(logId, "Sent "+sentBytes+"/"+messageLength+" bytes of OnlineInvite message.");
        }
    }

Here is the log output / result that I get:

[SLog] Err@F6624@T20240508 15:11:22.6275: DataExchangeClient-54.227.106.178:25150::SendInvite - Failed to send! WriterLength=525 MessageLength=525 Conn=NetworkConnection[id0,v1] Driver=Unity.Networking.Transport.NetworkDriver; SentVal=-4 StatusCode=NetworkPacketOverflow DriverSettings=Unity.Networking.Transport.NetworkSettings.

I think I’m probably missing something very basic and I’ve already spent one day trying to get this working so I thought to ask help from here. Anyone, please?! Unity Transport 2.2.1, Unity 2022.3.7f1, Target Platform Linux Dedicated Server.

Added DataExchangeClient.cs for completeness. The design is such that each game server has as many DataExchangeClients running as it has peer servers. For example on 3 server setup, each server has 2 DataExchangeClient instances running and exchanging data with the 2 peers.

9822576–1411968–DataExchangeClient.cs (13.8 KB)

2

I was able to find out that I can send from DexClient to DexServer using NetworkPipeline.Null. In my test setup I have 3 servers, and therefore each server has one DataExchangeServer instance and 2 DataExchangeClient instances. All 3 create and use identical fragmented reliable NetworkPipelines.

It seems that something goes wrong with fragmented reliable pipelines of clients in my setup and they always report -5 from the EndSend call. The DataExchangeServer and its pipeline is created first and it works.