Output data from PopEventForConnection to native array/slice instead of a DataStreamReader

I want to pop an event in a job and store a pointer to the internal sub array of that data so that I can later parse the data in other jobs.

I cannot use the DataStreamReader with ReadBytes in this case because I am inside a job and cannot allocate temp memory (with ReadBytes) and return that as it’s not allowed.
Passing in a pre-allocated block of memory to the job increases code complexity and I really don’t want to do that.

I’ve made a tiny modification to NetworkDriver to work around this. It’s probably unsafe.

My ParsePacketsJob
See how PopEventForConnection now outputs a NativeArray instead of DataStreamReader

while ((cmd = Driver.PopEventForConnection(Connections[index], out NativeArray<byte> bytes)) != NetworkEvent.Type.Empty)
                {
                    if (cmd == NetworkEvent.Type.Data)
                    {
                        var reader = new DataStreamReader(bytes);
                        PacketType type = Packets.ReadPacketType(ref reader);
                        reader.SeekSet(0);
                     
                        ReceivedMessages.Add((int) type, new PacketArrayWrapper()
                        {
                            Pointer = bytes.GetUnsafeReadOnlyPtr(),
                            Length = bytes.Length,
                            InternalId = Connections[index].InternalId
                        });
                    }
                    else if (cmd == NetworkEvent.Type.Disconnect)
                    {
                        Debug.Log("Client disconnected from server");
                        Connections[index] = default(NetworkConnection);
                    }
                }

Modification of NetworkDriver.cs
Added

            public NetworkEvent.Type PopEventForConnection(NetworkConnection connectionId, out NativeArray<byte> bytes)
            {
                return PopEventForConnection(connectionId, out bytes, out var _);
            }

            public NetworkEvent.Type PopEventForConnection(NetworkConnection connectionId, out NativeArray<byte> bytes, out NetworkPipeline pipeline)
            {
                pipeline = default;

                bytes = default;
                if (connectionId.m_NetworkId < 0 || connectionId.m_NetworkId >= m_ConnectionList.Length ||
                    m_ConnectionList[connectionId.m_NetworkId].Version != connectionId.m_NetworkVersion)
                    return (int)NetworkEvent.Type.Empty;

                var type = m_EventQueue.PopEventForConnection(connectionId.m_NetworkId, out var offset, out var size, out var pipelineId);
                pipeline = new NetworkPipeline { Id = pipelineId };

                if (type == NetworkEvent.Type.Disconnect && offset < 0)
                    bytes = m_DisconnectReasons.GetSubArray(math.abs(offset), 1);
                else if (size > 0)
                    bytes = ((NativeArray<byte>)m_DataStream).GetSubArray(offset, size);

                return type;
            }

I understand your use case and will bring it up to the team, but unfortunately I’m not sure this is a change we’ll be willing to make. The data stream API serves an important role for us in mediating access to our internal data structures.

To process data in a separate job, I’d recommend just passing the DataStreamReader itself to the other job. In your example code, that would mean modifying PacketArrayWrapper to store a DataStreamReader instead of a pointer and length. All readers obtained when popping data events are valid until the next update job is scheduled.

(Also small aside regarding the example code; I would advise against using NetworkConnection.InternalId. It’s not a unique connection identifier, since the internal ID of a connection can be reused after it’s closed. I’m not sure why it’s part of our public API, honestly. It would be preferable to store NetworkConnection structures directly. They’re meant to be small and usable as value types.)

1 Like

Aha. I now see where things went wrong. I assumed that DataStreamReader was only meant to be used in the current scope in a temporary way. Storing and passing DataStreamReader around feels weird but I now see that it’s a more elegant and proper solution to what I’m trying to achieve. Thank you for the response :slight_smile:

One problem with passing in DataStreamReader to a job is that DataStreamReader.m_bufferPtr produces errors because I am passing an unsafe pointer to the job.

I cannot use [NativeDisableUnsafePtrRestriction] because it has to be applied on the m_bufferPtr field itself inside of DataStream.cs

This attribute will not do anything and it will still produce errors about using an unsafe pointer.

    [BurstCompile]
    public unsafe struct ParsePublicSnapshotJob : IJob
    {
        [NativeDisableUnsafePtrRestriction] public DataStreamReader Reader;

Using Entities 0.50

Oh, good catch! I’ll file a task on our end to fix this. Thanks for raising this!

1 Like