Memory leak in NativeQueue

Unity Engine
, ,
1

Hi there,

I’m currently using jobs, burst and native containers quite extensively. I recently discovered that the code I’ve been writting causes Unity to consume 16 Gb of memory in about 15-20 minutes.

Upon further inspection, I isolated the cause to be NativeQueue. Searched the forums, and found this:

Turns out that when you call Enqueue(), some memory is allocated to never be released. Over time this leak is a real problem. You can reproduce it using this code, it will leak at a rate of 1 Gb every ten seconds, approximately:

using Unity.Collections;
using Unity.Mathematics;
using UnityEngine;

public class QueueLeak : MonoBehaviour
{
    private struct Data
    {
        public float4x4 data0;
        public float4x4 data1;
        public float4x4 data2;
        public float4x4 data3;
        public float4x4 data4;
        public float4x4 data5;
        public float4x4 data6;
        public float4x4 data7;
        public float4x4 data8;
        public float4x4 data9;
        public float4x4 data10;
        public float4x4 data11;
        public float4x4 data12;
        public float4x4 data13;
        public float4x4 data14;
        public float4x4 data15;
        public float4x4 data16;
        public float4x4 data17;
        public float4x4 data18;
        public float4x4 data19;
    }

    private NativeQueue<Data> queue;

    private void OnEnable()
    {
        //Allocate
        queue = new NativeQueue<Data>(Allocator.Persistent);
    }

    private void OnDisable()
    {
        //Deallocate
        queue.Dispose();
    }

    private void Update()
    {
        for (int i = 0; i < 10000; i++)
        {
            queue.Enqueue(default);
            queue.Dequeue();
        }
    }
}

Can anyone confirm it’s not an obvious misuse of the queue, so that I can file a bug report?

2 Likes
2

Upon inspecting the queue’s source during these two calls:

queue.Enqueue(default);
queue.Dequeue();

It seems the issue is that the amount of readers for the first block is always 2, so when Release() is called within Dequeue():

public unsafe static void Release(NativeQueueBlockHeader* block, NativeQueueBlockPoolData* pool)
{
      if (0 == Interlocked.Decrement(ref block->m_NumReaders))
      {
            pool->FreeBlock(block);
      }
}

the interlocked decrement can only get it to 1, never 0, and the block is never freed.

How does this happen? When you call Enqueue(), AllocateWriteBlockMT is called and numReaders is initialized to 1 for the block:

           if (currentWriteBlock == null)
            {
                currentWriteBlock = pool->AllocateBlock();
                currentWriteBlock->m_NextBlock = null;
                currentWriteBlock->m_NumItems = 0;
                currentWriteBlock->m_NumReaders = 1; //<----Here
                NativeQueueBlockHeader* prevLast = (NativeQueueBlockHeader*)Interlocked.Exchange(ref data->m_LastBlock, (IntPtr)currentWriteBlock);

                if (prevLast == null)
                {
                    data->m_FirstBlock = (IntPtr)currentWriteBlock;
                }
                else
                {
                    prevLast->m_NextBlock = currentWriteBlock;
                }

                data->SetCurrentWriteBlockTLS(threadIndex, currentWriteBlock);
            }

Now, when you call Dequeue(), internally it calls TryDequeue, which in turn:

  • calls GetFirstBlock()
  • reads its contents.
  • calls Release() on the first block.

At the end of GetFirstBlock(), right before returning the block, there’s this:

Interlocked.Increment(ref firstBlock->m_NumReaders);

Now m_NumReaders is 2.

After reading the block’s contents , Release() is called, which will use FreeBlock() if this condition is met:

if (0 == Interlocked.Decrement(ref block->m_NumReaders))

But the decrement only gets m_NumReaders to 1, the condition is not met, the block never freed, and so there’s a leak.

I’m not sure how to fix this myself without breaking the rest of the queue, but I hope it helps.

4 Likes
3

Turns out there’s already a bug report for this, and it has been confirmed:

The problem is actually more general, does not only affect the “dequeue to array” use pattern. Log in to vote the issue if you’re also being held back by it!

1 Like
4

Hi, thanks for the detailed analysis!
But it looks like this bug has been fixed in the following collections package [0.2.0] - 2019-11-22
This package requires Unity 2019.3 0b11+ could it be that you’re using an older version?
I believe the bug on issuetracker should have been closed when the fix landed, for some reason that didn’t happen and we’re following up on that.

1 Like