Update Texture2D System Memory Directly?

I have a 1024x1024 texture in RGB24 format, no mipmaps, so the texture itself takes up 3MB of memory. However, I need to update this texture frequently from code, so the texture has read/write enabled, and thus Unity keeps a “system memory” copy of the texture. So, the texture takes up a total of 6MB of memory.

However, to make changes to the texture over time in C#, I need another texture representation in C#, a byte array of size (1024x1024x3) = 3MB. At frequent intervals, the game will update this byte array, and then call texture.LoadRawTextureData(bytes), and then call texture.Apply().

So ultimately, my question: is this really the best way to do this? A 3MB texture that needs to be updated at runtime bloats to 9MB of used memory? I totally get the need for a copy of the texture in system memory, but is there any way for me to edit those bytes directly, rather than allocating yet another 3MB for a C# byte array? Is it possible to use C# unsafe context and get a byte* to the underlying system memory to modify that instead of allocating a C# byte array???

As I know, the best way to do this is using framebuffer structure, otherwise you will stall the render pipe and get very bad performance.

Which means that you need to implement your native render API with C or C++ and call these API in OnPostRender function by using GL.IssuePluginEvent();

In your native render API, you can get the pixel data, modify it and update the another texture.