HLAPI - Why does ChannelBuffer have a maxDelay and combines packets? LLAPI does this already?

Unity Engine
1

I see in ChannelBuffer CheckInternalBuffer it is only sending out packets based on the maxDelay set, and
SendBytes seems to try and combine messages into a single packet.

Why is ChannelBuffer doing this when I thought the NetworkTransport.Send already handled combining packets and sending out packets based on ConnectionConfig.SendDelay.
ConnectionConfig.SendDelay states -
“Gets or sets the delay in milliseconds after a call to Send() before packets are sent. During this time, new messages may be combined in queued packets.”

Is ChannelBuffer having its own send delay just so we have more control over when each channel sends out its packets, and it combines the packets just to help out the NetworkTransport.Send?

I ask because I am not using the HLAPI, so I am creating my own code over the LLAPI and using the HLAPI as a guide. Do I need to combine packets and handle the send delay, or will the LLAPI handle it for me?

Edit-
One good reason to combine your own messages is because I think the way unity combines the messages is by adding a length before each message so that on Receive it sends each message separately, whereas if we combine it ourselves we dont need to worry about putting a length for each message so long as we use a message ID system and deserialize properly.
It also seems channels get mixed into a single packet.

2

Ive tested and I do see that NetworkTransport.Send is delayed by ConnectionConfig.SendDelay. I have not tested if it combines packets while its waiting to send (not sure how to test that).

Since the HLAPI ChannelBuffer also waits to send, I am wondering if this means its possible for the ChannelBuffer to cause extra delay in sending if the NetworkTransport.Send timer and the ChannelBuffer timer is not properly synced.

Edit-
I think I can confirm that NetworkTransport.Send does indeed combine packets while its waiting to send. If I log NetworkTransport.GetOutgoingMessageQueueSize while calling NetworkTransport.Send, I can see the count go up for each message, however, when calling NetworkTransport.GetOutgoingPacketCountForHost, if the byte arrays I am sending each send is a small size such as 1 byte, then the OutgoingPacketCount only goes up by 1 even though I am sending 5 messages. However, if the byte array I am sending is a large size such as 1024, then the OutgoingPacketCount would go up by 5 or so.
The same is true for when reading the NetworkTransport.GetIncomingMessageQueueSize and NetworkTransport.GetIncomingPacketCount.

So since its pretty much confirmed that the NetworkTransport handles the sendDelay and packet combining, I am assuming I dont need to implement it unless needed in my own ChannelBuffer class. I probably dont even need a ChannelBuffer class.

3

So the HLAPI ChannelBuffer has this method

        public void CheckInternalBuffer()
        {
            if (Time.realtimeSinceStartup - m_LastFlushTime > maxDelay && !m_CurrentPacket.IsEmpty())
            {
                SendInternalBuffer();
                m_LastFlushTime = Time.realtimeSinceStartup;
            }
            //........
        }

Using the LLAPI, I setup a system where I send a message every second and the ConnectionConfig.SendRate is set to 5 seconds.
When I run this code every update after a connection was made…

    void CheckSendTimeSinceConnect()
    {
        if(Time.realtimeSinceStartup - lastSendTimeSinceConnect > sendDelaySeconds)
        {
            //It seems when we log here, the send size has not yet decreased...
            Debug.Log("SendTime on frame " + Time.frameCount + " - SendMsgQueueSize = " + GetSendSize());
            lastSendTimeSinceConnect = Time.realtimeSinceStartup;
        }
        //After a bit we see that finally the send size was decreased...
        //Just checking if its 0 is not such a great check, instead we should check if it was just lowered,
        //but since our send times are so slow, it seems fine to check for 0 in this case.
        if(GetSendSize() == 0) Debug.Log("SendSize is 0 on frame " + Time.frameCount);
    }

    int GetSendSize()
    {
        byte error;
        return NetworkTransport.GetOutgoingMessageQueueSize(targetSocketID, out error);
    }

From what I see, the custom timer check within CheckSendTimeSinceConnect is not synced with the actual send time of the NetworkTransport sending. It seems to be desynced by 1 second, but that time is probably just dependent on when the connection is made for the timer to actual start.

So I did another test and this time I kept track of the timer not just when the connection is made, but always. This means we are updating the lastSendTime even when we are not connected.
I found that its was still desynced.
I also tested when the timer is being kept track of when the socket was opened with AddHost, and still had a desync.
I noticed in ConnectionConfig.SendDelay docs it says
“When the first message to be sent is received by Unity Multiplayer, the delay begins…
The delay expires and the two packets are sent out. Unity Multiplayer then waits for a new message and, on receipt, starts a new delay.”
So I did another test and this time kept track after the first message was sent, and while the 2 timers seem to be very synced, they are still a few frames desynced. Perhaps I need to understand more how their delay system works.

I feel the HLAPI would have a delay in sending packets due to this.

Due to me needing to alter messages before sending, I do actually need to use some kind of ChannelBuffer to store my messages, however, I have no real way of properly sending the messages out right before the NetworkTransport sends everything.
Is it possible that we can have some kind of event OnNetworkTransportSending or a bool IsNetworkTransportSendingThisFrame or something that will be called right before the NetworkTransport starts doing it sending so that we can grab all our messages on that frame and send the most up to date data?
(I would also want the networktransport to be able to still merge messages and what not after that event is called so all the bunches of messages we just sent would still get merged to reduce bandwidth.)

Is the NetworkTransport sending time separate per connection? If so then a event being called for each individual connection wouldnt be of much use, or at least not simple. Hopefully the timer is a single timer for all connections.

Edit-
So I been understanding SendDelay all wrong. I think what I want is the ConnectionConfig.MinUpdateTimeout
SendDelay actually seems to work by when you send a message, the senddelay then activates and wont send messages for that set delay. I guess this is so you give some extra time for the NetworkTransport to combine messages into packets before sending.

Setting SendDelay to 0 and MinUpdateTimeout to 5 seconds, while doing the send message from within the custom timer I see that everything is being sent right away.
However, MinUpdateTimeout seems unpredictable, kinda like its a preferred rate, but unity will decide the actual rate. It says in the docs it can auto increase this delay if needed, but I noticed that with a MinUpdateTimeout of 5 seconds, if I send a message each second, those messages will get sent right away even though I said to not send until after 5 seconds.

I see there is a NetworkTransport.QueueMessageForSending and NetworkTransport.SendQueuedMessages
It advises to set SendDelay to 0 and use these methods to control when we actually send packets. This seems to be what I can use. Instead of getting a event call from when the transport sends, I can just have my owner timer that uses the queue and sends when I say to.

SendQueuedMessages seems to still merge messages into single packets, which is nice.

The problem is if I queue up 50 messages and click to send the queue, only 20 or so actually send and then it seems the queue is just stuck filled with 30 or so messages no matter how many times I tell it to send. Why is this?
If I set MinUpdateTimeout to 1 millisecond instead of 5 seconds, then the sendqueue seems to send all the messages just fine, otherwise the sendqueue just doesnt send anything even after the 5 seconds or after trying to manually send it. It does seem to start to go down itself, but it goes down like 1 message every 20 seconds?
Does sending MinUpdateTimeout so low cause any issues if I am not using the NetworkTransport.Send anyways. From my tests, the SendQueuedMessages seems to ignore the MinUpdateTimeout and allows you to send whenever you want?

Either way, NetworkTransport.QueueMessageForSending and NetworkTransport.SendQueuedMessages might be the way to go.

1 Like