How to cleanup managed objects properly to prevent empty active heap spice from growing? (WebGL)

Unity Engine
, , , ,
1

Hi there,
I am wondering how to properly release objects in managed memory, specifically byte arrays and strings.

I have created the following minimal example to replicate a problem my team is facing in an actual project. It loads a videofile from streaming assets as a byte array, creates a string with a base64 encoding of the byte array and has a method trying to cleanup the created data.

Environment:
Unity 2022.3.31
Target: WebGL

 public class PayloadExperiment : MonoBehaviour
    {
        [SerializeField] private string filePath;

        private byte[] videoData;
        private string base64VideoData;


        /// <summary>
        /// Load a videofile from StreamingAssets as byte array
        /// </summary>
        public void LoadVideo()
        {
            StartCoroutine(LoadVideoCR());
        }

        private IEnumerator LoadVideoCR()
        {
            string videoPath = Path.Combine(UnityEngine.Application.streamingAssetsPath, filePath);

            Debug.Log($"Getting video from: {videoPath}");

            using (UnityWebRequest request = UnityWebRequest.Get(videoPath))
            {
                yield return request.SendWebRequest();

                if (request.result != UnityWebRequest.Result.Success)
                {
                    Debug.LogError("Error loading video: " + request.error);
                }
                else
                {
                    videoData = request.downloadHandler.data;
                    Debug.Log("Video loaded, byte array length: " + videoData.Length);
                }
            }
        }

        public void CreateBase64String()
        {
            base64VideoData = System.Convert.ToBase64String(videoData);
        }

        public void Cleanup()
        {
            videoData = null;
            base64VideoData = null;

            GC.Collect();
            StartCoroutine(UnloadAssets());
        }

        private IEnumerator UnloadAssets()
        {
            var unloadOperation = Resources.UnloadUnusedAssets();

            while(!unloadOperation.isDone)
            {
                yield return null;
            }

            Debug.Log("completed cleanup");
        }
    }

I use MemoryProfiler.TakeSnapshot to take snapshots in the WebGL build.
A: After LoadVideo() and CreateBase64String() were called
B: After Cleanup() has been called

Viewing A it in the fragmentation view of the memory profiler v 0.7.1-preview.1 I could Identify the string and byte datablocks based on their size.

After calling Cleanup() these blocks stay in the managed heap but move (at least partially) from the objects to the empty active heap space

Calling LoadVideo() and CreateBase64String() again results in more memory being allocated despite free blocks being available in the empty managed heap space.

image
image1582Ă—1169 71.6 KB

Even switching to an empty scene (and thus destroying the object PayloadExperiment was a component of) does keep the blocks in the empty active heap space.

My 2 Questions now are:

  1. When calling Cleanup() why are the blocks only moved to empty active heap space instead of being released back to the browser/OS? Will the empty active heap space ever be released back to the browser/OS at some point or will unity hold on to it until the application quits?
  2. Why is the empty active heap space not reused when calling LoadVideo() and CreateBase64String() a second time?

P.S. I would have included more profiler screenshots of the different states, but as a new user I am only allowed to include 1 :frowning:

2

Since you set the references to null, these should be collected … one would think.

on the Web platform, the GC can only run when no managed code is executing (which could potentially reference live C# objects). This occurs at the end of every rendered game frame.

… pay close attention to code that performs a lot of temporary allocations per-frame, especially if these allocations might exhibit a sequence of linear size growth. Such allocations may cause a temporary quadratic memory growth pressure for the garbage collector.

So be sure to wait for the next frame until you load another big chunk of data into memory, eg the next video byte[].

In any case, if you look at the memory profiler be sure to check the frames following the one that ran Cleanup to confirm that memory goes down. But you won’t observe garbage collection until after the current frame has rendered.

The above link also contains explanations about the heap memory in web builds.

The Player Settings also have the option of using the incremental garbage collector. Not sure if this is available on WebGL though but if it is, that would also alter memory usage behaviour.

At least in this script you aren’t using any Assets - string and byte[] are not assets. So this call is likely superfluous or may even cause other, unrelated assets to be unloaded which may need to be loaded again instantly (asset churn). This can add to memory pressure!

3

So be sure to wait for the next frame until you load another big chunk of data into memory, eg the next video byte[] .

In my profiling setup I trigger the methods and the memory snapshots through UI buttons so there should be a couple of frames between cleanup, snapshot collection and new data loading.

At least in this script you aren’t using any Assets - string and byte[] are not assets. So this call is likely superfluous

Yeah I thought so. I am just trying everything in this experiment to free up the memory again.

The Player Settings also have the option of using the incremental garbage collector. Not sure if this is available on WebGL though

No it is not available in WebGL. So I would assume the GC to collect the allocated memory for videoData and base64VideoData latest after the frame where I set the references to null.

5

The blocks aren’t “move to”, they become empty heap space because the objects in them got garbaged collected. I.e. your code worked in unloading them.

IIRC WebGL never returns memory to the OS, so these blocks of memory will just stick around forever. They should be able to be reused though.

Are there still contiguous sections of them that are of the appropriate size to host the new allocations?

Also, maybe remove the Debug.Logs, or at least the string concatenation in them. With those you are interspersing smaller allocations in between your big blocks, which is a good way to fragment the heap space in a way that makes reuse difficult.

I’d also recommend checking that same snapshot in an empty 2022/Unity6 project using The Memory Profiler Package version 1.1.1. it doesn’t have the visualization but it does have a Memory Map table sorted by address, so you should be able to find the same address range in there. The reason I suggest this is that there have been multiple fixes to how the Memory Profiler Package code crawls the managed heap. Because the snapshot data doesn’t contain the information on all managed objects that are alive, but instead comes with the managed heap bytes and the objects rooted via static fields or GCHandles (thread-local/stack based roots are not reported so there could be something on a thread’s stack that points into that address range but the profiler just wouldn’t know about it).

In 1.1.0 I fixed a slightly bigger mistake in how it goes from these roots over all fields of all objects it finds in order to build up the full object graph, which was that it ignored value type fields, which means any references held via references fields on structs where ignored. So there is a non-zero chance newer versions of the package might find objects in that heap section that would explain why they weren’t reusable.

6

IIRC WebGL never returns memory to the OS, so these blocks of memory will just stick around forever. They should be able to b e reused though.

Ok that is what I figured so far, which just leaves my second question, why in my case it is not reused.

I simplified my testscenario even more and just create and delete a string of a certain size. Again I trigger the public methods by simple UI buttons:

using UnityEngine;
using TMPro;
using System;

public class MinimalistResearch : MonoBehaviour
{
    private string data;
    private int creationSize = 20 * 1024 * 1024;

    [SerializeField]
    private TMP_Text sizeLabel;
    private void Awake()
    {
        sizeLabel.text = creationSize.ToString();
    }

    public void CreateData()
    {
        data = new string('A', creationSize);
    }

    public void IncreaseCreationSize()
    {
        creationSize *= 2;
        sizeLabel.text = creationSize.ToString();
    }

    public void DecreaseCreationSize()
    {
        creationSize /= 2;
        sizeLabel.text = creationSize.ToString();
    }

    public void Cleanup()
    {
        data = null;
        GC.Collect();
    }
}

I created 3 strings in total, cleaning up after each creation. All strings have the same size. Looking at the snapshots with the memory Map in memory profiler 1.1.1 I see this:

image
image908Ă—1900 269 KB

First I thought the “Reserverd” section that is kept after deletion of the first string is blocking the second string to reuse that 40 mb of memory. But the second string does not leave behind such section after deletion. Yet the third string does not reuse the 40 mb address range that was left behind by the second string. Any idea why?

7

I had the same first thought but it is a bit of a puzzle. There is the thing where Boehm is a heuristic GC that technically considers all pointer-sized fields as potential pointers so it could be that with ranges this big, some long somewhere happens to be a value that looks like a pointer to this address range?

Adding that parsing to the Memory Profiler is a next step after fixing super long opening times due to the afformentioned fix for struct held references taking a bit longer to deal with snapshots with billions of structs in arrays. Gonna have to do some performance testing before I can land a change that would crawl even more fields :sweat_smile:

8

Maybe try this with something slightly smaller to reduce random chance effects like that. Any allocation that’s multiple pages big (4KB per page, unless it’s iOS where it’s 16KB) would probably do.

9

Really though, allocations this big should, imo, not even end up on the Managed Heap. Use NativeArrays or other forms of “unsafe” memory buffers for these. Using managed memory for it is usually just causing trouble…

10

I tested it with a string of 1.2 mb. Then it looks fine. After three creations and deletions the empty heap sace only increases by the size of one string.

image
image868Ă—270 16.4 KB

What I do not understand though why the virtual machine memory is increasing that much. Is that an effect of taking memory snapshots? I also noticed that on the earlier tests with larger strings.

Really though, allocations this big should, imo, not even end up on the Managed Heap. Use NativeArrays or other forms of “unsafe” memory buffers for these. Using managed memory for it is usually just causing trouble…

Yeah that is a good point and probably the direction we will go in within our project. Although I still don’t understand why the managed heap is struggling with that.

11

It can be that, as some type information needs to be prepared in order to have all the info we need to process and display the info we dumped with the heap, but we capture the heap and VM status before we inflate the types we need, so a follow up snapshot will have slightly higher Virtual Machine memory usage. We reduced the impact of that, I’m just not sure ATM if that went into 2022 or just Unity6…

In general the use of generics and reflection is the leading cause for VM increases.