Problems with the garbage collector

[Hindrik]

Hello,

I am currently working on a procedural world generator.
I am using threads to make each island generate on another thread.
Each thread constructs a mesh, which is then send to the VRAM.
After all threads completed, i run a forced garbage collection to clear all the arrays. (around 1 gb)
This works fine, the profiler reports my managed heap to be 110mbs, where it was ~ 1gb before.
After the worldgeneration is done, it stores the arrays to disk and that’s it.
I cannot reduce the size of the arrays.
The problem however, is that my allocated ram stays at 1.14gb, even tough it uses far less. (around 120mb at most). The vram is loaded with around 450mb of meshes and that’s fine. This amount never goes down, so task manager reports a nice 1.2gb used. Which i am not very happy with, obviously.

Is there any way to reduce the reserved ram?

[lordofduct]

Not really.

That’s the way the mono runtime works. It allocates a chunk of memory for use as the heap. You created the necessity for 1 gig of memory. It had to resize its memory allocation with the operating system to force that (this is expensive), when done, it doesn’t know if your program will need it again. It just knows that you did need it, so it maintains it.

The newer versions of .Net and Mono are much better at avoiding such things as this and actually have a specific way of dealing with large objects as they often are unique. But we unfortunately get a very outdated version of mono with Unity.

Now, you refer to “clear all the arrays”, and 1 gig is a LOT of memory. I have to assume your completed mesh actually takes up an entire gig of memory. So of these arrays, can any of them be removed mid process before creating other arrays to avoid resizing up so large? Can some of them be compacted in any way?

Usually when dealing with large amounts of data like that you stream through the data rather than load it all into memory. Only the portion of the memory you need right at that very moment is actually in memory, and the next portion swaps with it at the next moment. I don’t know what you’re gig of memory actually is though, so I couldn’t even begin to say what you might do to stream through it, rather than create it all at once.

[Hindrik]

@lordofduct

Not really.

That’s the way the mono runtime works. It allocates a chunk of memory for use as the heap. You created the necessity for 1 gig of memory. It had to resize its memory allocation with the operating system to force that (this is expensive), when done, it doesn’t know if your program will need it again. It just knows that you did need it, so it maintains it.

The newer versions of .Net and Mono are much better at avoiding such things as this and actually have a specific way of dealing with large objects as they often are unique. But we unfortunately get a very outdated version of mono with Unity.

Now, you refer to “clear all the arrays”, and 1 gig is a LOT of memory. I have to assume your completed mesh actually takes up an entire gig of memory. So of these arrays, can any of them be removed mid process before creating other arrays to avoid resizing up so large? Can some of them be compacted in any way?

Usually when dealing with large amounts of data like that you stream through the data rather than load it all into memory. Only the portion of the memory you need right at that very moment is actually in memory, and the next portion swaps with it at the next moment. I don’t know what you’re gig of memory actually is though, so I couldn’t even begin to say what you might do t stream through it, rather than create it all at once.

One mesh is not that big, maybe 50-60mb max, but since i create many meshes at once, since it is WAAAY faster to do it on multiple threads, at most i can get 64 threads concurrently working on constructing the arrays for the meshes. Which the main thread then actually puts in a mesh and sends to the gpu. So yes, i could maybe decrease the amount of threads, but this would also heavily impact the generation time, which is around 10 secs now. I’ll see if i can split it in like two different parts and sent it in two times to the gpu.

If it isn’t an option i might have to see if i can write a C++ dll which handles it for me, but i am afraid that this will not be optimal due to the marshalling, espacially not when marshalling 1gb of data…

[lordofduct]

Ok, so when it comes to threading, here’s the deal.

Your actual CPU can only handle as many threads as it physically is capable of doing. Usually one per core, unless you have a CPU that allows hyper-threading (or similar technology). Although, technically hyper-threading isn’t a true separate thread, because it really is only parallel processing if the two parallel cpu operations at that very moment are complimentary to hyper threading. Otherwise one ends up having to wait for the other. You can kind of think of it as an extra ‘half’ thread.

Yet, you can actually spin up as many ‘threads’ in code that you want. This is because those threads aren’t actual threads on the CPU. You can have threads on a single core CPU. Every program running in Windows is technically its own thread (or group of threads).

This is called multi-tasking. The operating system allows each application to run code at the same time by allowing each thread intermittent access to the CPU. Making it appear like they’re running in parallel.

This is what happens when you spawn more threads than you have physical cores in your computer.

The average computer has 2 to 4 cores now a days (hyper-threading possibly, or even 8 cores for some high end machines, and more for servers… server type situations though are usually dedicating those cores to other things though). Spawning those threads to give you a perceived ‘speed’ boost won’t really actually give you a perceived speed boost once you surpass the actual number of threads the CPU can support. And don’t forget that those cores are also being used by other applications running, so you probably won’t be accessing all of them anyways.

Having thread counts beyond that core count (and why you normally don’t count the cores) is useful for reasons other than speed. Multithreading/Async processes has other uses. For example while downloading a file, you don’t want the system to hang, you want to also animate the screen. This has nothing to do with speed, and all to do with asynchronous behaviour.

So yeah, if your meshes are 50 or 60 megs. And you’re consuming a gig of memory. That’s what… 18 meshes? More if many are small? And you’re giving all those individual meshes their own thread?

Well, beyond 4 threads, you’re probably not seeing much gains. For the average casual gamer or cellphone, you’re probably not seeing much gains pass 2 threads!

I’d honestly go with 2 threads.

Note - you may see minor gains for the first couple of threads over the cpu core count. This can sometimes be the result of the operating system load balancing threads. A generic thread created in code isn’t necessarily assigned a specific core. But instead the OS might shift it between cores to balance out load. CPUs now a days with their variable speeds and the sort optimize in various ways that give over all system gains.

With that said though, expecting such gains is not guaranteed. And you shouldn’t write something in hopes of reaping those extra milliseconds of efficiency at the cost of a gig in memory!

Technically that memory cost could cause system slow-down.

And also you could technically get speed increases by recycling the arrays you use, rather than allocating for each array.

[Hindrik]

@lordofduct

Ok, so when it comes to threading, here’s the deal.

Your actual CPU can only handle as many threads as it physically is capable of doing. Usually one per core, unless you have a CPU that allows hyper-threading (or similar technology). Although, technically hyper-threading isn’t a true separate thread, because it really is only parallel processing if the two parallel cpu operations at that very moment are complimentary to hyper threading. Otherwise one ends up having to wait for the other. You can kind of think of it as an extra ‘half’ thread.

Yet, you can actually spin up as many ‘threads’ in code that you want. This is because those threads aren’t actual threads on the CPU. You can have threads on a single core CPU. Every program running in Windows is technically its own thread (or group of threads).

This is called multi-tasking. The operating system allows each application to run code at the same time by allowing each thread intermittent access to the CPU. Making it appear like they’re running in parallel.

This is what happens when you spawn more threads than you have physical cores in your computer.

The average computer has 2 to 4 cores now a days (hyper-threading possibly, or even 8 cores for some high end machines, and more for servers… server type situations though are usually dedicating those cores to other things though). Spawning those threads to give you a perceived ‘speed’ boost won’t really actually give you a perceived speed boost once you surpass the actual number of threads the CPU can support. And don’t forget that those cores are also being used by other applications running, so you probably won’t be accessing all of them anyways.

Having thread counts beyond that core count (and why you normally don’t count the cores) is useful for reasons other than speed. Multithreading/Async processes has other uses. For example while downloading a file, you don’t want the system to hang, you want to also animate the screen. This has nothing to do with speed, and all to do with asynchronous behaviour.

So yeah, if your meshes are 50 or 60 megs. And you’re consuming a gig of memory. That’s what… 18 meshes? More if many are small? And you’re giving all those individual meshes their own thread?

Well, beyond 4 threads, you’re probably not seeing much gains. For the average casual gamer or cellphone, you’re probably not seeing much gains pass 2 threads!

I’d honestly go with 2 threads.

Note - you may see minor gains for the first couple of threads over the cpu core count. This can sometimes be the result of the operating system load balancing threads. A generic thread created in code isn’t necessarily assigned a specific core. But instead the OS might shift it between cores to balance out load. CPUs now a days with their variable speeds and the sort optimize in various ways that give over all system gains.

With that said though, expecting such gains is not guaranteed. And you shouldn’t write something in hopes of reaping those extra milliseconds of efficiency at the cost of a gig in memory!

Technically that memory cost could cause system slow-down.

And also you could technically get speed increases by recycling the arrays you use, rather than allocating for each array.

I don’t really get what you mean by recycling the arrays? Each mesh has three arrays, one vor VC3’s (Vertices), one for ints (Triangles) and one for VC2’s. (UVs) and each mesh has a different amount of them. And as you said, i currently have up to 64 threads each creating at minimal 2 meshes and at most 10. (Each mountain is a mesh), and i will certainly optimize this and completely change this, but the mountain’s are very small and barely have any impact on ram usage. I also cannot combine them, since the main mesh is almost at its limits (65k vertices). And i know how threading works

But you are certainly right in that i don’t have it very optimized yet. My biggest concern is the memory usage right now. I really have to get that down. Especcially since i am planning to make my map twice the width and height as it is now. (4000x4000). I store each position in a byte array btw, so that is ~17mb at most and not a problem whatsoever.

Since my islands are 250x250 at most, i have 250 islands and 250 threads. That is a big problem, but this will be fixed later on, since my biggest concern is the ram and how to bring that size down. Since i predict my game will run up to 4gb if i create a full sized world. Afterwards it is back to only 30mb or so, so no big deal, but generating al those meshes at the same time is problematic. I however don’t want to get a crazy long generation time, even if it is just once. (I load the meshes from disk next time the game loads, this is working perfectly atm.)