Async readbacks on a background Editor

Unity Engine
1

Hi all,

I’m trying to setup a pipeline where an external tool sends commands to the Editor to redraw a camera and return the output of that camera back to the tool through native code.

My initial setup was to manually call Camera.Render() to an RT + GetPixels every time I needed a refresh but obviously this would be too slow for a high framerate (target is around 60-90 FPS)

While trying to switch over to async readbacks - they end up working while the editor is focused but once the other tool has focus the callbacks to process the data never execute.

The setup I’m trying to use looks more or less like the following:

void Setup()
{
    coroutine = EditorCoroutineUtility.StartCoroutineOwnerless(
        EditorCoroutineUpdate()
    );
}

IEnumerator EditorCoroutineUpdate()
{
    while (true)
    {
        CaptureRenderTexture(); // <-- Executes regardless of editor focus
        yield return new EditorWaitForSeconds(rate);
    }
}

public void CaptureRenderTexture()
{
    cam.Render();

    AsyncGPUReadback.Request(
        cam.targetTexture, 0,
        0, width, 0, height,
        0, 1, TextureFormat.RGB24,
        OnAsyncReadback
    );
}

private void OnAsyncReadback(AsyncGPUReadbackRequest request)
{
    if (request.hasError)
    {
        Debug.LogError("GPU readback error.");
        return;
    }

    Debug.Log("Readback"); // <--- never executes when the editor is unfocused

    unsafe
    {
        // ... do native work ...
    }
}

I can get this to work when the editor is unfocused if I do an ASyncGPUReadback.WaitAllRequests() at the end ofCaptureRenderTexturebut then I incur a ~30ms semaphore wait on the GPU each frame.

Any other suggestions that may help here?

2

While working on this a bit more, I ended up encountering a memory leak with AsyncGPUReadback on 2021.3.11f1 LTS. Can pretty much distill it down to the following:

    IEnumerator EditorCoroutineUpdate()
    {
        while (true) {
            cam.Render();
            request = AsyncGPUReadback.RequestIntoNativeArray(ref bucket, rt);

            AsyncGPUReadback.WaitAllRequests();
            // or request.Update(), same problem regardless of the method 
            // of forcing async calls to finish when the editor is out of focus.

            if (request.done)
            {
                Debug.Log("req done");

                if (request.hasError)
                {
                    Debug.LogError("request err");
                }
            }

            yield return new EditorWaitForSeconds(rate);
        }
    }

Where bucket is a preallocated permanent NativeArray. Same behaviour can be observed with just AsyncGPUReadback.Request(), with or without callbacks, or running the CommandBuffer version of the method through something like an HDRP post processing volume pass (e.g. IN-18242 - HDRP PostProcessVolume Variant · GitHub) but, again, only when the rendering is happening while the editor doesn’t have application focus. So I guess I’m going with the slow way for now.