AsyncGPUReadback.RequestIntoNativeArray causes InvalidOperationException on NativeArray

Dimetrock · November 25, 2020, 5:24pm

Hi,

I have a class that calculates mesh data in compute shader and does additional work in a Job. I need to pull data from the GPU to the CPU without blocking my main thread. I have a private property Native array that is referenced by the job and I use it as a parameter when calling AsyncGPUReadback.RequestIntoNativeArray with a callback. When the callback is fired I know that I can continue with the job, but Unity throws an
exception:

InvalidOperationException: The Unity.Collections.NativeArray`1[System.Single] TerrainMeshDataJob.heights can no longer be accessed, since its owner has been invalidated. You can simply Dispose() the container and create a new one.

        public void ReadData()
        {
            ReadIsComplete = false;
            AsyncGPUReadback.RequestIntoNativeArray(ref _nativeHeights, _heightBuffer, Callback);       
        }
      
        private void Callback(AsyncGPUReadbackRequest request)
        {
            if(request.hasError) throw new Exception("AsyncGPUReadback.RequestIntoNativeArray");
            ReadIsComplete = true;
        }

So now I can’t use my array anymore and the whole point of using NativeArray is gone. Can anyone help me with this?

There is very little information on this, any information is welcome.
Unity version is 2020.1.14f1

tertle · November 25, 2020, 7:39pm

From memory the normal AsyncGPUReadback.Request native array it returns is only valid for 1 frame then gets disposed so you need to copy it to a new array or use it straight away.

I would have thought RequestIntoNativeArray would be doing this copy for you but maybe not and you still need to copy it.

richardkettlewell · November 25, 2020, 8:25pm

The NativeArray you provide to RequestIntoNativeArray should not be disposed/invalidated by Unity - it should be yours for as long as you want it.

If you think Unity has disposed it, it may be a bug. If you report it, we can check.

It’s a bit of a guess though - I’d need to see more of your code to say more

Dimetrock · November 26, 2020, 9:23am

I found a way to make it work:

        private NativeArray<float> _tempBuffer;
        private NativeArray<float> _nativeHeights;
       
        public void ReadData()
        {
           
            ReadIsComplete = false;
            _tempBuffer = new NativeArray<float>(_verticesLength, Allocator.Persistent);
           AsyncGPUReadback.RequestIntoNativeArray(ref _tempBuffer, _heightBuffer, Callback);
        }
       
        private void Callback(AsyncGPUReadbackRequest request)
        {
            if(request.hasError) throw new Exception("AsyncGPUReadback.RequestIntoNativeArray");

            _nativeHeights.Dispose();
            _nativeHeights = new NativeArray<float>(request.GetData<float>(), Allocator.Persistent);
            _tempBuffer.Dispose();
            ReadIsComplete = true;
        }

It seems to me that this is a bug and Unity disposes of the NativeArray when it clears the AsyncGPUReadbackRequest request and it should not do that.

richardkettlewell · November 26, 2020, 9:52am

I agree, this copy shouldn’t be necessary - the entire reason to make the request into your own array is to avoid an extra copy

Can you report a bug and reply here with the case number? I’ll make sure it gets fixed.

Dimetrock · November 26, 2020, 2:36pm

Sure, I will make an example project and report it. As soon as I am done I will reply with the case number here.

Dimetrock · November 26, 2020, 9:08pm

I’ve created a bug ticket: Unity Issue Tracker - AsyncGPUReadback.RequestIntoNativeArray causes InvalidOperationException on NativeArray

LooperVFX · December 26, 2020, 1:04am

@richardkettlewell thanks for keeping an eye on this issue and hope you are returning from a well deserved holiday. I am using Unity 2020.2.0f1 and also experiencing similar issues, But I found it may go deeper than this alone.
A related issue was reported by @jhelbig in this post: AsyncGPUReadback.RequestIntoNativeArray - What am I doing wrong?

The funniest thing is that the exception simply throwing when it shouldn’t may be the problem? i’m using AsyncGPUReadback.RequestIntoNativeArray with the callback argument, same as above and “owner has been invalidated” exception BUT curiously if i continue to step through execution manually everything still works!! I see the results of my Compute buffer being read back in the Native Array, handed off to Mesh API, etc. So obviously the data is being read back and making it all the way through the pipeline, just exception is just being thrown and halting execution when it shouldn’t.

What’s even stranger is that while trying to find a way to optimize this as best i can in the meantime, I found I am able to use AsyncGPUReadback.Request in the coroutine and then in the callback request.GetData into a NativeArray THAT I NEVER INITIALIZED, I merely defined it and nothing more. This mysteriously works great with no exceptions / errors / warnings. It works as if i initialized the NativeArray with Allocator.Persistent on Start or Awake, but I didn’t. and is reproducible even after restarting the Editor and in Builds, still no issue.
If i DO initialize the NativeArray with Allocator.Persistent on Start or Awake like I think I’m supposed to? then everything works great up until exiting play mode, then “A Native Collection has not been disposed, resulting in a memory leak” is thrown with stack trace pointing to the Allocator.Persistent initialization. In other cases when a proper Dispose is expected to occur i get the “[NativeArray…] has been set to undisposable and cannot be deallocated” as @jhelbig also reports.

In all cases, I am calling Dispose() on the persistently allocated NativeArray only once OnDestroy(). So it seems there is a bigger or related issue with not being able to Dispose persistently allocated NativeArrays in some situations possibly when they are used with AsyncGPUReadback and/or coroutines and callbacks. perhaps the “ownership” is not being handled as expected.

I also can’t ignore this blog post by Jackson Dunstan on “Sharing IDisposables” that seems to touch on the possible cause of the problems here, shared ownership as it is related to when / where we are permitted to Dispose of a IDisposable object or when it is marked as Disposable or Undisposable. If anything, it’s the same cast of characters. JacksonDunstan.com | Sharing IDisposables

Presently with holiday I don’t have time to put together a proper bug report and reproducible example for all these cases (I will in a few week if these issues haven’t already been stamped out) but I wanted to at least flag this additional very odd unexpected behavior to try and help connect the dots on these potentially interrelated issues.AsyncGPUReadback is great to have anyway and seems over the past year or so there’s now enough people using it now to get enough feedback and attention to sort out these issues and pain points.

Also heads up to @Soaryn on the above as I know you rely on AsyncGPUReadback as well from your report on that memory leak issue that was already fixed here: AsyncGPUReadBack Request Memory Issue

Happy holidays all!

wusticality · May 11, 2021, 9:17am

Hey richardkettlewell, I was curious what the status of this bug is as I seem to be running into it too. In my case I allocate a persisent NativeArray elsewhere that is never disposed. When I request into it via AsyncGPUReadback, attempting to access said array results in this error:

InvalidOperationException: The Unity.Collections.NativeArray`1[Unity.Mathematics.float2] TestSystem.JobData.values can no longer be accessed, since its owner has been invalidated. You can simply Dispose() the container and create a new one.

richardkettlewell · May 11, 2021, 9:20am

Hi, you can see the status here: https://issuetracker.unity3d.com/issues/asyncgpureadback-dot-requestintonativearray-causes-invalidoperationexception-on-nativearray

That said, it looks to have some weird version numbers on it (2020.1 is no longer supported/receiving new patches)
I will make sure that info is up to date.

wusticality · May 11, 2021, 9:59am

Hey @richardkettlewell , sounds like there’s a fix in progress and that it is indeed a bug then? For now I can perform a copy (which is what I was already doing) but it’s a large compute buffer and therefore a performance bottleneck so I’m super keen for the fix.

It’s interesting because it seems that the container isn’t actually disposed (as I would expect). I’m not sure why the “owner being invalidated” message is happening as this NativeArray is outside the ECS system. I tried to drill down into the C# code but as soon as I hit an extern it’s a black box haha.

Thanks again for your time!

wusticality · May 14, 2021, 12:31pm

Bump

richardkettlewell · May 14, 2021, 1:56pm

Is there some additional information you need?

wusticality · May 18, 2021, 1:15am

Hey @richardkettlewell - mostly just confirming that it’s a known bug, that’s all.

LooperVFX · May 18, 2021, 6:06am

Let’s keep this thread on topic, please. Feel free to start a new thread about this unrelated issue and I also highly recommend filing a bug report for best results.

richardkettlewell · May 18, 2021, 8:10am

It’s known.

LooperVFX · May 18, 2021, 8:02pm

Yes, the Issue Tracker link @richardkettlewell posted earlier already confirms this: https://issuetracker.unity3d.com/is…uses-invalidoperationexception-on-nativearray
Here is a breakdown of the issue tracker status::

Bug reports that are reproduced by the quality assurance (QA) team are added to the Issue Tracker, so anything that exists on the issue tracker is a known bug --or it wouldn’t be on the issue tracker. (see highlighted in yellow in image below)
When a bug fix is in progress or its release is imminent, this status bar (see circled in red in image below.) will reflect that. We can see the status currently is “Fix In Review for 2021.2.X” and “Planned for 2020.3.X, 2021.1.X” meaning the fix is imminent / almost ready, and that once it’s released for the latest Unity Editor version, it is also planned to be backported to the previous supported versions listed, The bug fix is just going through a review / quality assurance process before it can be released. ( more info in this thread )
So, the good news is not only is it a known issue, the status shows that the fix is in the development / QA process and is landing quite soon, We don’t need to post on this thread or on the issue tracker comments to garner support or prioritize the issue further. If we did, you can guarantee I’d be pushing for this issue to be prioritized along with you.

esgnn · July 6, 2021, 12:17pm

Dimetrock:

I found a way to make it work:

        private NativeArray<float> _tempBuffer;
        private NativeArray<float> _nativeHeights;
      
        public void ReadData()
        {
          
            ReadIsComplete = false;
            _tempBuffer = new NativeArray<float>(_verticesLength, Allocator.Persistent);
           AsyncGPUReadback.RequestIntoNativeArray(ref _tempBuffer, _heightBuffer, Callback);
        }
      
        private void Callback(AsyncGPUReadbackRequest request)
        {
            if(request.hasError) throw new Exception("AsyncGPUReadback.RequestIntoNativeArray");

            _nativeHeights.Dispose();
            _nativeHeights = new NativeArray<float>(request.GetData<float>(), Allocator.Persistent);
            _tempBuffer.Dispose();
            ReadIsComplete = true;
        }

It seems to me that this is a bug and Unity disposes of the NativeArray when it clears the AsyncGPUReadbackRequest request and it should not do that.

Stumbled upon the same issue, CopyTo workaround prevented the error as well.

wusticality · July 7, 2021, 2:12am

Although it works, it does introduce an expensive copy.

ryo0ka · July 21, 2021, 7:28am

My workaround: use CommandBuffer.

CommandBuffer class got an equivalent method & won’t yield the error log performing the same work.

// original procedure causing error logs
AsyncGPUReadback.RequestIntoNativeArray(ref buffer, texture, 0, TextureFormat.ARGB32, OnReadbackComplete);

// working, equivalent procedure using CommandBuffer
var cb = new CommandBuffer();
cb.RequestAsyncReadbackIntoNativeArray(ref buffer, texture, 0, TextureFormat.ARGB32, OnReadbackComplete);
Graphics.ExecuteCommandBuffer(cb);