Getting System.IO.Pipelines into Unity

I want to integrate System.IO.Pipelines into my Unity project for a good TcpClient implementation.

But I am stuck.

I saw this post: Do you support the use of System.IO.Pipelines?

So I went and fetched the packages from NuGet and put the DLLs into my project folder.

7900693--1006522--2022-02-16_pipelines_dll.png

And it ALMOST works.

It does not know what NetworkStream.ReadAsync(Memory, CancellationToken). Yet it should, because the DLLs are all there.

So what am I doing wrong, and how do I fix it?

Here’s the code (I took it from TcpPipelinesSample/Client/Program.cs at master · vbondaryuk/TcpPipelinesSample · GitHub). Be aware that I still have not ported this properly into Unity, because I am getting these errors:

using System;
using System.IO.Pipelines;
using System.Net.Sockets;
using System.Threading.Tasks;

namespace Tcp.Client
{
    public class TCPPipelineClient
    {
        public string host;
        public int port;
   
        public void Start()
        {
            var pipe = new Pipe();
            Task.WaitAll(
                TcpReaderAsync(pipe.Writer, host, port)
            );
        }

        private static async Task TcpReaderAsync(PipeWriter writer, string host, int port)
        {
            const int minimumBufferSize = 512;

            using (TcpClient client = new TcpClient())
            {
                Console.WriteLine("Connecting to server.");
                await client.ConnectAsync(host, port);
                Console.WriteLine("Connected.");

                using (NetworkStream stream = client.GetStream())
                {
                    while (true)
                    {
                        try
                        {
                            Memory<byte> memory = writer.GetMemory(minimumBufferSize);
                            int read = await stream.ReadAsync(memory); //ERROR: No overload for method 'ReadAsync' takes 1 arguments

                            if (read == 0)
                                break;

                            writer.Advance(read);

                            Console.WriteLine($"Read from stream {read} bytes");

                        }
                        catch
                        {
                            break;
                        }

                        // Make the data available to the PipeReader
                        if (!await writer.Flush())
                            break;
                    }

                    Console.WriteLine("Message was read");
                }
            }
            writer.Complete();
        }
    }

The NetworkStream is a class of the .NET framework itself, but the method is not available in the .NET version which Unity supports, see: https://docs.microsoft.com/en-us/dotnet/api/system.io.streamreader.readasync?view=net-6.0#system-io-streamreader-readasync(system-memory((system-char))-system-threading-cancellationtoken)

So you will have to use a different overload of the ReadAsync method, the one with ReadAsync (char[ ] buffer, int index, int count) works in Unity

That kinda undermines the whole point of using System.IO.Pipelines…
All examples I found are using the ReadAsync(Memory) method. Now I have to implement that functionality? Where can I find the source for that method to find out what it actually does?

The .NET code is available on Github you can find the method implementation there but I would assume that the method depends on some other method/feature that is not available in the “old” .NET version, otherwise they would have included it.

I don’t know what kind of project you want to make, the Unity 2021 beta and 2022 alpha support newer .NET versions, it’s obviously not recommended to use these versions for anything serious yet, but if you just want to prototype something then I guess it would be easier to use these versions instead of trying to reimplement framework methods.

Or you dig deeper and look if there is a different version (maybe made by the community) which backports these methods to the older version.

Thank you, I will look into that!

In the meantime I will use this solution, as it does what I want: https://codereview.stackexchange.com/questions/147935/extracting-complete-lines-from-a-data-stream

The Memory struct is one of the “newer” collection types (like Span) which can be used for accessing a managed array in a more controlled / direct and performant way. The whole Pipeline namespace wraps around those new types and are only accessible in .NET Standard 2.1 or .NET 5+. Unity currently supports .NET Standard 2.0 or .NET 4.x as you can see in the player settings of your project.

Yes, it does ^^. Relying on the latest tech always comes with some downsides. That’s why most minecraft modpacks run an ancient MC versions. When a new version comes out, there are barely any mods for that version.

The PipeReader / PipeWriter are just a neater way to solve the common problem of reconstructing a message of streaming data. All it essentially does is providing a way to shove new received data into a buffer (PipeWriter) and that data is appended to the end of the internal buffer. It also provides a way to access the data in the buffer (PipeReader) in order to search for a certain marker and to “slice out” the message and adjust the remaining buffer for the next message. It’s all explained over here. Note that the article was written in oct. 2021. So while native C# developers probably happily are using those new classes for quite some time now, you can’t expect to have support for them in Unity that quickly ^^.

According to your last question this is probably about reading messages out of a TCP stream? As I mentioned in your other question, using the BinaryReader and a binary high level protocol is usually the easiest solution. The BinaryReader in combination with a NetworkStream will automatically block until the requested data has arrived. Yes, the new Buffer, Memory and Span structs can reduce the amount of garbage generated, but there are always ways around that.

So I would suggest you’re looking for an alternative solution. Can you share any details about your planned highlevel protocol? Is it text based or binary? Keep in mind that text based protocols have to make sure the message seperating marker (often a new line “/n”) must not appear anywhere in the payload.