I find that If I create a big byte array, mono will NOT release the memory. E.g., the following codes will create a 300MB byte array, the memory cannot be freed, and I received out of memory exception in one minute. But If I only create a 30MB byte array, the memory will be freed successfully. This is very bad. Because we need to download a 300MB update file by using WWW class, and WWW class uses byte array. We are doomed if you cannot fix this bug.
Unity ver: 4.7.1f1
public class ByteArrCheck : MonoBehaviour
{
void Start()
{
StartCoroutine(CheckMemoryLeak());
}
IEnumerator CheckMemoryLeak()
{
int i = 0;
while (true)
{
byte[] arr = new byte[314572800]; // 300MB
arr = null;
System.GC.Collect();
System.GC.WaitForPendingFinalizers();
yield return new WaitForSeconds(2f);
Debug.Log(i++);
}
}
}
I see nothing in that code verifying that the garbage was not collected.
So Iām going to assume what you mean is that when you look in the profiler you are noticing that Unity does not release the memory back to the operating system.
Yeah, it doesnātā¦ thatās not the way Garbage Collection worksā¦
See, the way memory works in the CLR (the runtime mono runs in) is that it reserves memory from the operating system, this is called the āheapā. In this swath of memory it will stick your referenced objects and what not as your code runs. When GC runs (which it does on its own termsā¦ you donāt have to call it) it will go through the heap and clear out objects on the heap that arenāt needed anymore, and reorganize the left over objects to defragment the memory.
The entire time this happens the heapās total size is allocated from the operating system. That does not change.
UNLESS you go and attempt to use an amount of memory that the heap is not large enough to support. When this occurs it has to request more memory from the operating systemā¦ this is a slow process. This is why it allocates a large portion up front instead of with every object you create.
So, when you go to load a 300MB file into memory, which is larger than the heap can handle. It has to go request 300 megs from the operating system. It uses it. You then GC that data. But the CLR maintains that memory for future useā¦ you demonstrated to the CLR that you need this memory. Deallocating it back to the OS is expensive, and so is allocating it again if you need it again. So itāll maintain it as part of the heap for quite some time nowā¦
This isnāt a leak because the memory is still usable by your application. If you go and load yet another 300MB file after this one has been garbage collected, it wonāt ask the OS for another 300 megsā¦ it has it already, and it will use it again.
A memory leak is run away memoryā¦ itās memory that gets filled up and forgotten about, never to be utilized again until the application haults.
I donāt understand. There is no reference to that byte array. It should be collected by GC.
āSo, when you go to load a 300MB file into memory, which is larger than the heap can handle. It has to go request 300 megs from the operating system. It uses it. You then GC that data. But the CLR maintains that memory for future useā¦ you demonstrated to the CLR that you need this memory. Deallocating it back to the OS is expensive, and so is allocating it again if you need it again. So itāll maintain it as part of the heap for quite some time nowā¦ā
If you were right, the mono will take about 300MB memory, no matter how long the codes run. Because CLR has 300MB cached memory. But in fact, if you run the codes, you will receive out of memory exception in one minute.
The array is most likely getting allocated to the Large Object Heap(Large Object Space in mono) and not being freed/re-used because the LOH is a pile of shit. There is also no guarantee that memory will be freed when calling GC.Collect()ā¦ Even if nothing is referencing the object. With large data you are better off writing a little unmanaged library that will allow you to free the memory immediately.
P.S. Why are your updates so insanely big? You should create a binary diff patcher so you only have to send the modifications. Assets can be compressed with LZ4 or LZMA to significantly reduce the size as wellā¦
I have to create some AssetBundles from memory, which are large byte array. When I create those too many times, it will throw out of memory exception. I tried setting byte to null and GC.Collect(), but it did not work, the memory still not collected.
How to solve this kind of problem? Marshal.AllocHGlobal only creates unmanaged memory, but the AssetBundle.CreateFromMemory can only take byte[ ] as params.
Iām not sure how AssetBundles work in the engine. AssetBundle.CreateFromMemory could be keeping a reference to the byte array. The AssetBundle.CreateFromMemory method is also deprecated in 5.3.5. You should be using AssetBundle.LoadFromMemory now.
Thanks for your reply.
It should have nothing to do with AssetBundle.CreateFromMemory or AssetBundle.LoadFromMemory.
I did a test: I only new a large byte array (hundreds of MB), then set it to null and GC.Collect(), the memory of byte array is not collected. However, if I call GC.Collect() every frame, it will collect memory.
How to collect memory immediately?
You should never directly call GC.Collect unless you know exactly what youāre doing. There is a bigger underlying problem somewhere. The generational garbage collector wont free memory from the LOH as often. You can read more on monoās garbage collector here.
Is there something preventing you from creating multiple smaller AssetBundles? Smaller byte arrays will be collected much faster.
Iām facing this same issue. Running different Memory debugging tools it seems as though if a byte array is too big it is never recollected by Unity/Mono when Garbage Collecting.
I may be unexperienced, but to me, I see an infinite loop creating an infinite number of 300MB arrays that are never told they are no longer needed. Since the arrays are still āneededā, how can GC be run on them?
Same problem here. Cannot avoid allocating big value of data - need to create AudioClip during runtime from Wav (converted from mp3). And still found no way to clean allocated memory.