Memory voes, is there really no way to manually allocate/free objects?

Unity Engine
1

Hi,

I’m working on a pretty complex game. At it’s core lies an AI system that simply uses a lot of objects and a lot of memory. There’s not much I can do about that. And the amount of memory will increase as I expand the domain.

Due to the large number of objects (I presume) the GC has a hard time - spikes are around 500ms on a powerful machine. I tested with the 2019 GC with mixed results. It makes the spikes almost tolerable however I’m worried about them increasing as I add more stuff to the game.

So my best approach seems to be to head to zero (or close as I can get) memory allocation. Getting to this conclusion was a long journey that ended in a predictable place.

I mainly see two tehniques to lower memory usage (beside the usual don’t use strings etc etc).

  1. “Global” variables. Ie. variables that even though used locally in a method you declare at the system or component level. That way we don’t need to keep reallocating them every time the method is called. It’s dirty coding but let’s say tolerable as long as you’re careful about recursive methods and so on.

  2. Where you can’t “global” something you need to pool it. And here lies the trouble. Pooling means:

a. Writting a pool system - fair enough, no biggie
b. Using the pool system to get new objects instead of calling new - again, fair enough, a little factory pattern never killed anybody
c. Returning the objects to the pool once they’re no longer used - well wait a minute - this is the exact same situation as if we didn’t have a memory manager - we just killed all/any productivity benefits the GC could bring us
d. Each poolable objects needs to have a reset() method. That is, once it’s returned to the pool you need to bring it to a pristine state before you return it as a new object next time someone wants an object of that type.

Now if you’re pooling lists you’ll just call Clear() on them and you’re done - easy peasy.

But what happens when you pool complex objects - like components (from an ECS system). Those are objects that can have 30-40 properties - some value, some reference types. Your reset method needs to set the default value for each property for value types and somehow reset the reference types if/when possible (ie. List.Clear() etc).

You have 50 components, you do it once, no biggie. But now you have to remember - every time you add a new property to a component - you now have to add it to the reset() method as well. You forgot? Good luck figuring that elusive bug 3 months down the road. And good luck training new team members on all the hacks.

This just seems to be a very very poor way to write code. I see no advantage over manual memory management - you still have to “free” your stuff. I see a lot of extra code, complication and maintenance issue caused by the GC. (Yes, I know it guards against memory leaks - I have a lot of experience writting manual memory managed stuff - there are tools to help, and memory leaks just aren’t a big problem with clean code.)

Sorry for the rant, I’m just hoping someone will point I’m missing something obvious and taking a hilariously wrong approach. GC for game development just seems very inferior to manual memory management.

A GC that would work as the default but allow you also manage memory (ie. letting the GC know not to concern itself with an object and giving you a way and the resposibility to free it) would be awesome!

Edit: can’t find a way to fix the typo in the subject :))

2

I think you’re confusing lowering memory usage with garbage collection.

No… a variable local to a method is allocated for a short moment on the stack. Once the method is finished the variable is immediately cleaned up.

void Foo()
{
    int i = 5;
    //do stuff
}
//int i is cleared up here

IF that variable is a class type, and you instantiate a new instance, that will allocate memory on the heap. And that should be avoided. Like this:

void Foo()
{
     var sb = new StringBuilder();
     //do stuff
}
//sb, the variable pointer, is cleared... but the StringBuilder still exists on the heap

Furthermore “global” has nothing to do with memory usage. Being global vs a class level field is going to take up the same amount of memory regardless.

Pooling ALSO doesn’t lower memory. That technically increases memory. Because now you’re holding onto objects that would have otherwise been cleared up.

The only thing pooling saves you is the cost of instantiation and cleaning up. This could be helpful if GC is causing frame stutters, or if you spawn a lot of objects that are expensive to instantiate (like prefabs, which have a bunch of work in Awake/Start).

This is viable for what I believe is your real issue. Not consuming too much memory, but that you’re instantiating too many objects that are then GC’d.

I’m not sure what you’re on about here.

If you have an object with 20 fields/variables on it… and you stop referencing that object. When GC cleans it up, it ALSO cleans up all of the things it references as well. Since it’s a dead reference. You don’t have to null out every field of an object to make the things it references eligible as well.

If you have this reference chain:

a → b → c → d

And you dereference b from a:

a X b → c → d

When GC cleans up b… well now nothing points to c since b is dead. So its also eligible. And since c is now dead, nothing references d and it is collected.

You don’t have to manually do all those steps. The GC is smarter than that!

Not sure what to say in the end to this…

I think you’re looking at GC a little backwards though.

…

I will say if you’re having GC spikes somewhere, you should open up the profiler and see what things are generating the most garbage. Then focus your efforts on maybe refactoring it so that it doesn’t create so much garbage.

Depending the version of the framework/garbage collector will impact the ways to go about it.

But if you show us any hot spots I bet we could help reduce it.

Personally in my games up to now I haven’t really used very much pooling and GC isn’t an issue. Our biggest bottle neck was the creation of collection types for temporary usage… arrays/lists/etc. How we deal with that is we have a TempCollection pool:
https://github.com/lordofduct/spacepuppy-unity-framework-3.0/blob/master/SpacepuppyUnityFramework/Collections/TempCollection.cs

https://github.com/lordofduct/spacepuppy-unity-framework-3.0/blob/master/SpacepuppyUnityFramework/Collections/TempList.cs

And we make sure to always use the non-allocating versions of methods when calling into the Unity API (for example GetComponents has a method that allows passing in a List to be filled rather than it returning a newly minted array).

But as for prefabs and other gameobjects… we’ve never really seen a lot of costly GC from them personally.

But the one I do hear about most is “bullets” if you have actual GO’s as bullets. In that regard having a pool for bullets definitely could help (since they’re so short lived).

But heck, GameObjects are mostly manually managed types. You have to call ‘Destory’ on them to get rid of them. They don’t heavily rely on GC (aside from the C# facades), since they’re primarily Unity objects whose state are stored on the Unity C++ side.

3

Hey Lordofduct. My initial post might have been a bit unclear but we’re actually on the same page :slight_smile:

My main concern is that just as you had to pool generics (which I have to as well), I also have to pool a set of objects specific to my project - internally I’m using an ECS architecture (nothing to do with Unity’s ECS) - and I have to pool the entities and hence the components they are … composed of.

Pooling those objects presents a challenge in that I have to give the pooled objects a “reset” method that adds complexity and maintanability issues to the code. Purpose of that method is to set all the object properties to their default values and it’s being called when the object is being returned to the pool. For example a List reset method would just call Clear().

My point was that in order to do proper pooling you have to manually return objects to the pool and reset their state. Since I have to know when an object isn’t being used anymore - I’d much rather have a way to just discard it at that point instead of having to add it to a pool and reset it’s state.

To sum it up: Isn’t it great that we have the GC? Now let’s work around it and try at all cost to avoid it running :slight_smile:

To add a bit of context - I’m using a lot of memory with a lot of objects - the GC takes ~ 500ms just to inspect that memory even if there’s nothing to clean. So the only proper option I see is to try and delay the GC as much as possible by allocating as little as possible memory each frame - hence my need for very aggressive pooling.

All the above refers to C#, non Unity clases. I do pool GameObjects and have absolutely no issue with that.

To clarify my post a little:

  • lower memory usage - I mean lower the memory allocated during the Update() method not in total. I agree that pooling actually increases the total memory and if you start to do pooling you’re even harder pressed to avoid the GC

  • “global” - notice the quotes :slight_smile: I’m refering to declaring a reference type in a larger scope than it’s needed. Say you need a List in a method. You can declare that list at the class level so that the List is only allocated in the class constructor and not every time the method is called. It works, but it’s dirty. And it’s a trap if that method ever becomes recursive.

At the end I think I only have 2 options:

a. Take the maintenance cost and do it
b. Export most of a the code to a DLL where I can do my own memory management

4

If you’re going to go to those lengths, you might as well jump onto ECS even though it’s in alpha. The API doesn’t look very user friendly yet, but doing your own memory management means C++, so then user-friendliness isn’t exactly an option to start with :stuck_out_tongue:

But, seriously, the entire deal with ECS is that manual memory management sucks, but waiting for the GC to run and having stuff randomly all over the heap also sucks. If you’re running into GC issues, and it’s not obvious how to avoid them in standard Unity C#, it’s probably a good fit.

Of course that would require rewriting the entire thing, so that might not be worth it.

stackalloc is a really nice way to do this without needing a list in the class body, and with a lot less overhead.

I haven’t done this much (I try to avoid pooling in other ways), but you could do this by putting the data in a struct, have that struct as a field, and just dump the struct when you Reset():

public class Foo {

    private struct FooData {
        public int bar;
        // add new fields here
    }
    private fooData data;

    public int Bar { get => data.bar; set => data.bar = value }

    public void Reset() {
        data = default;
    }
}

That guards against not resetting fields. Of course, if Foo could just be a struct to begin with, that’d be fine.

5

With Unity 2018.3 you can turn off the garbage collector.

https://docs.unity3d.com/2018.3/Documentation/ScriptReference/Scripting.GarbageCollector.html

With Unity 2019.1 you can manually run it for a period of time.

https://docs.unity3d.com/2019.1/Documentation/ScriptReference/Scripting.GarbageCollector.CollectIncremental.html

6

@Baste

I’m not sure ECS would help me here but I’m definitively eager to hear if :slight_smile: The reason I say that is that I’m fairly decoupled from Unity. I have 1 Monobehaviour running on 1 GameObject - that’s called GameManager. So all the stuff is happening in my own class hierarchy (my code is ECS as in the general pattern not as in Unity’s ECS implementation for Unity stuff).

stackalloc - yes, I had a look at Marshal and/or Unity’s Unsafe Utility. Problem is I’m dealing with managed objects. If it would be possible to have a C# class and say that’s it’s not managed … that’d be dreamy.

Structs are great, but my classes contain further references to managed objects (at the very least generic collections). So in my Reset() I have to reset the values for my value types and further “reset” my reference types (ie. Collection.Clear()). It’s not impossible. It just goes against all my spidey sense of what code should be => maintainable :slight_smile:

Also I’d be afraid of abusing structs for large data. Reference types have a reason to exist - they’re a lot faster to pass than copying large areas of memory.

@Ryiah

Thanks! My target is to only have the GC run once every 15 min at worst. Running the GC every 15 minutes means you have to generate very little if at all garbage every frame.

As to the GC in 2019 - what it offers will probably be great to take me the last 10-20% towards my goals. I’m eagerly looking forward to the 2019.1 release - I tried it on the beta but I get some random crashes I’m not prepared to dig into atm :slight_smile: