How do I save a large PNG async?

Scripting C# & .NET
1

I really have 2 questions, and I think answering question #2 would negate the need for question 1. I’m working on a 2D game.

Question 1:
I have a FOW texture that is quite large, and in order to persist it, I’m saving it as a PNG as follows:

RenderTexture.active = FOWRenderTexture;
                
Texture2D texture = new Texture2D(FOWRenderTexture.width, FOWRenderTexture.height, TextureFormat.RGB24, false);

texture.ReadPixels(new Rect(0, 0, FOWRenderTexture.width, FOWRenderTexture.height), 0, 0);
texture.Apply();
                
byte[] bytes = texture.EncodeToPNG();
                
File.WriteAllBytes(filePath, bytes);
                
// Clean up
RenderTexture.active = null;
Destroy(texture);

This works well, except that it is a heavy process and hangs the game.

I tried pushing the save off the main thread using await Task.Run, but most of the saving function is required to be on the main thread (throws an error if inside the await). I can push the File.WriteAllBytes to a different thread, but apparently that’s not the heavy part, since it still hangs.

Any ideas for how to push this whole chunk off the main thread?

Question 2:
Alternatively, instead of saving the actual texture, I think it might be smarter to save some sort of simple set of map coordinates that can translate to and from the actual texture. So as the player walks around, maybe every time they enter a new 10x10 chunk, a simple representation of that chunk gets added to an array. Then I can just save and load the array, and when the game loads it uses the array to clear fog over chunks that have been entered.

Does this make more sense? I’m trying to figure out how to actually implement this.

Thanks for any thoughts.

2

EncodeToPng and/or ReadPixels are likely the time sink (use Profiler to verify).

Writing bytes to disk is already cached by the OS on fixed drives and thus returns even before everything is guaranteed to be physically written. Only on ejectable media write-cache is disabled (on Window at least, and by default).

As the ReadPixels method needs to load the texture data from the GPU onto main memory, this is also likely very slow as the manual indicates. The page also offers suggestions for alternatives.

3

Yeah it looks like you’re right.

I clearly need to learn how to use the profiler better.

I also think despite figuring this out, I’m better off storing either a List or if I can figure it out a HashSet and then using that to remake the fog clearance.

Thanks for the tips.

4

To help others that stumble on this thread like me, here is how I solved it in Unity 6000.0.38f. There is still a small stall to allocate the first NativeArray. I tried moving this onto the background thread as well but then get errors with the Async GPU readback.

Edit (20/08): I have made some changes to make the function:

  • now GraphicsFormat agnostic
  • more comments for explanations and recommendations
  • will create all folders along path if missing
  • As suggested by user CodeSmile, function will now handle failing to write to a file and multiple threads trying to write to the same file
public async Awaitable SaveRenderTextureAsync(string path, string fileName, RenderTexture rt)
{
    int width = rt.width;
    int height = rt.height;
    GraphicsFormat format = rt.graphicsFormat;
    int rtSizeBytes = (int)GraphicsFormatUtility.ComputeMipmapSize(width, height, format);

    // NativeArray could be allocated outside the function and reused for maximum performance
    NativeArray<byte> data = new NativeArray<byte>(rtSizeBytes, Allocator.Persistent);

    // AsyncGPUReadback request has to be started on the main thread
    var request = AsyncGPUReadback.RequestIntoNativeArrayAsync(ref data, rt, 0, format);
    await request;
    
    // switch to background thread for intensive encoding work
    await Awaitable.BackgroundThreadAsync();

    var encoded = ImageConversion.EncodeNativeArrayToPNG(
        data,
        format,
        (uint)width,
        (uint)height);
    
    string filePath = path + fileName + ".png";
    try
    {
        // create missing folders along path
        Directory.CreateDirectory(path);

        using (var fileStream = new FileStream(filePath, FileMode.Create, FileAccess.Write, FileShare.None))
        {
            fileStream.Write(encoded.AsReadOnlySpan());
            fileStream.Flush(true);
        }
    }
    catch
    {
        Debug.Log("Failed to write to file, was the file already open?\n" + filePath);
    }
    
    encoded.Dispose();
    
    await Awaitable.MainThreadAsync();
    
    data.Dispose();
}
5

Nicely done! Also, your way ahead of everyone with that version. :wink:

I see one potential issue with this code. If you were to call it a second time before the first call returns, two (or more) threads may try to write to the same file and thus throw an exception.

Also rather than FileStream you could simply call: await File.WriteAllBytesAsync(filePath, encoded);
Rule of thumb: don’t use streams if you aren’t actually streaming. :wink:

6

Thanks for your points, I have used them to improve my post.

I couldn’t remove the stream as you suggested with WriteAllBytesAsync as it does not accept data from the NativeArray<byte>. I could convert the data to a byte[], but I would like to avoid the data cloning, is it worth it to avoid a stream?

I have made creating the stream more specific to prevent a double write to the same file.

7

Does encoded.AsReadOnly() work? The docs mention “casts to array” implying no copy is done.

8

Getting a ReadOnlySpan from encoded.AsReadOnly() is how I am currently able to write to the stream, File.WriteAllBytesAsync() expects byte[] (or maybe ReadOnlyMemory according to C# docs).

I’ve done a little investigation into non-alloc casting but no luck and I will just leave it as is. I do not have the experience to say if a stream will cause trouble but did find this testing that it should be fast: