Confusion around memory, gc, and the profile

LesBloom · June 15, 2023, 2:54pm

Hi all,

I am trying to track down a memory related crash in my project. After seeing enough crash reports, I believe the memory leak happens on my scene changes.

I used the memory profile tool, changed scenes a few times, and started digging into the results.

I am now quite confused as to what is going on.

It appears as though none of my class allocations are ever being garbage collected.

For example, if I look at this ‘Game’ class.

1 of these should be created in a scene; max.
There should be no global / static references to it.
It should be (eventually) GCed after changing to the next scene.

This snap shot was taken after running through 3 scenes, and then moving into a blank, empty scene.

However, there are 3 instances still floating around in memory.

The reference info is quite confusing as well.

The middle pane shows that each Game class instance has about 160 ish references. That seems logical as a scene is running. There are many other classes within the scene that store pointers to it.
However, I would expect this all to get cleaned up after the scene is dumped.
The right inspector, however, shows 0 references. And that is true for each of the 3 Game class instances. I am not sure which is correct.

This same pattern is true for every class I have looked into. I have not found a single class (monobehaviours, scriptables, basic allocated classes, etc) that is not still referenced and in these snapshots.

Is this info wrong? Is this an issue with the Memory Profiler? I don’t think this is the case, because I have real crash logs showing a memory leak in my project.

I thought I understood C# and how the GC works, but maybe I am totally wrong? I thought that everything should be cleaned up once the scene is dumped; so long as there are no more references to anything.

Any help here would be greatly appreciated.
Thank you for your time
Les

Kurt-Dekker · June 15, 2023, 3:37pm

Remember it is a chain. If you have even one central manager and it holds a reference to some other thing, that other thing might hold a reference to Game and on down.

Obvious things to do is write some code to destroy all of your static-ish stuff between scenes, see if the answer changes above.

Also things like ScriptableObject instances can have a “static feel:” if one of your SOs refers to something, until that reference gets nulled, since that SO isn’t going away, that reference is gonna hold.

LesBloom · June 15, 2023, 3:53pm

Hello @Kurt-Dekker

Thank you for that response.

Regarding “Remember it is a chain”. My understanding is that the GC is smart and will only look at root references to determine if something should be collected or not. So, even if there is a tree of objects with thousands of sub references to and from each other; the entire tree will be collected once the ‘root’ object has no references.

If that is the case, then I should be fine, because Game is my ‘root object’ and it no long has references to it.

Or am I missunderstanding?

Thank you again

Kurt-Dekker · June 15, 2023, 4:25pm

Then as I wrote above, DESTROY EVERYTHING that is long-lived and see if the problem goes away when you change scenes.

Remember,

LesBloom · June 15, 2023, 4:29pm

Lol, fair enough. Thanks @Kurt-Dekker . I appreciate you taking the time

Cheers

Kurt-Dekker · June 15, 2023, 4:45pm

This isn’t about fixing the problem yet.

This is still about finding the problem.

Go around with your sword and stab (delete) everything until you find one that goes “sqawk!”

Also, this presupposes you’re using source control so it’s one click to restore.

PROPERLY CONFIGURING AND USING ENTERPRISE SOURCE CONTROL

I’m sorry you’ve had this issue. Please consider using proper industrial-grade enterprise-qualified source control in order to guard and protect your hard-earned work.

Personally I use git (completely outside of Unity) because it is free and there are tons of tutorials out there to help you set it up as well as free places to host your repo (BitBucket, Github, Gitlab, etc.).

You can also push git repositories to other drives: thumb drives, USB drives, network drives, etc., effectively putting a complete copy of the repository there.

As far as configuring Unity to play nice with git, keep this in mind:

https://discussions.unity.com/t/736093/3

I usually make a separate repository for each game, but I have some repositories with a bunch of smaller test games.

Here is how I use git in one of my games, Jetpack Kurt:

https://discussions.unity.com/t/807568/3

Using fine-grained source control as you work to refine your engineering:

https://discussions.unity.com/t/826718/2

Share/Sharing source code between projects:

https://discussions.unity.com/t/719810/2

Setting up an appropriate .gitignore file for Unity3D:

https://discussions.unity.com/t/834885/5

Generally the ONLY folders you should ever source control are:

Assets/
ProjectSettings/
Packages/

NEVER source control Library/ or Temp/ or Logs/
NEVER source control anything from Visual Studio (.vs, .csproj, none of that noise)

Setting git up with Unity (includes above .gitignore concepts):

https://thoughtbot.com/blog/how-to-git-with-unity

It is only simple economics that you must expend as much effort into backing it up as you feel the work is worth in the first place. Digital storage is so unbelievably cheap today that you can buy gigabytes of flash drive storage for about the price of a cup of coffee. It’s simply ridiculous not to back up.

If you plan on joining the software industry, you will be required and expected to know how to use source control.

“Use source control or you will be really sad sooner or later.” - StarManta on the Unity3D forum boards

MartinTilo · June 16, 2023, 11:06am

Hello and welcome to the confusing world of Unity’s Managed Shell objects. As a starter lecture, this article might help set the scene a bit.

I’d also recommend that you install a 2022.2+ Editor version, create an empty project, add the package com.unity.memoryprofiler @1.1.0-pre.1 version via the manifest and import that snapshot into there. There have been a substantial amount of bugfixes and UI/UX improvements since the version you’re using, including some that might have fixed bugs in how the references view is populated.

I could imagine, that there are some functions in your Game class that are subscribed to some events and that their managed shells are kept in memory that way, but looking at this with the newer package version might help clarify this.

Also, I suspect you’d want to look at this with the All Of Memory page instead of the Unity Objects page but give both a try

MartinTilo · June 16, 2023, 11:11am

Also, we’re aware that the references view isn’t really where we’d want it to be and 166 references is an amount at which it might have given up too early previously (I hope this is better in 1.0/1.1). But even in the newer versions, it will give up after 1000 processed referencing objects and just list out what it got so far, which isn’t ideal but just throwing it out there for awareness.

LesBloom · June 16, 2023, 2:08pm

Hi @MartinTilo

Thanks for all that info and the responses

“I’d also recommend that you install a 2022.2+ Editor version, create an empty project, add the package com.unity.memoryprofiler @1.1.0-pre.1 version”

I am working my way to that point. My project runs on LTS, and 2022 just released, but I am also trying to get this update out. So, upgrading Unity is going to have to be put on pause until my next update cycle. Just bummer timing is all. For now, my project needs to stay on Unity 2021 LTS

“I suspect you’d want to look at this with the All Of Memory page instead of the Unity Objects page but give both a try”

I was also using the default Summary View. I think that is what you mean by “All Of Memory”. I will look around though. Either way, yes, the tool has been helpful and there is a lot to utilize.

“we’re aware that the references view isn’t really …”

Yeah, I understand. It’s tricky making a way for us to view so much data in an easy way.

“As a starter lecture, this article might help set the scene a bit.”

I will look into this as well

Again, thank you to you both. I am knee deep in this now
Cheers

LesBloom · June 16, 2023, 2:18pm

What about serialized references between 2 Monobehaviours? Do I need to null them out in order for those references to be dropped when the scene changes?

For example:

class Game : Monobehaviour
{
public Car m_Car;
}

class Car : Monobehaviour
{
public Game m_Game;
}

Then, I add these in my scene. I set them to reference each other via the Unity Editor inspector. I launch my game, move to this scene, play around for a while, and then leave the scene.

I assumed I don’t need to null out m_Car or m_Game. Is that wrong?

Thanks

orionsyndrome · June 16, 2023, 6:19pm

I would guess that after forcing a gen-2 cleanup between scenes

GC.Collect(2, GCCollectionMode.Aggressive);

if you still have committed memory than your objects aren’t really without references.
I could be wrong. Edit: I am. These parameters do nothing, though GC.Collect() should do what I meant.

dlorre · June 16, 2023, 7:38pm

You can also null the objects at the end of the scene to encourage crashes. That way you will find if there are any references lingering around. With the Amiga we had debugging tools such as Enforcer and Lawbreaker that were doing things like that (also filling unallocated memory with 0xDEADBEEF).

Unifikation · June 16, 2023, 7:50pm

Side note.

After trying Scenes in various ways, I’ve come to the conclusion that the best way to use Unity is with just one scene, and to make your own categorisations of what is and is not within your own concepts of “scenes” and destroy everything you don’t need whilst showing some kind of transition dialogue to players in the hope that all clean up occurs during that period.

StarBornMoonBeam · June 16, 2023, 9:24pm

There is actually a nice additive scenes technique. Though I agree with the sentiment as most projects can be compact into a single scene. Some projects can even be packed into a single object within a single scene ( ! ). Though, a multi scene approach can be useful for a linear based progression game. For editing purposes limiting the scale of scene of simply organisational purposes multi scene approach to a project can simplify a fair amount of coding.

MartinTilo · June 16, 2023, 10:44pm

That’s why I did not suggest updating your project. Just create an empty project with the 1.1 package version installed in it and then use the “import” Button in the memory Profiler UI to import the snapshot from your 2021 project (or attach the 2022 Editor to your 2021 Player and capture the snapshot directly from that).

No, what I refered to was the new UI in the 1.0+ versions of the package.

If they live in the same scene and for the full duration of that scene? No. With the scene, their native objects are unloaded and if there is no other managed references to them from outside the scene, statics or DontDestroyOnLoad objects, then their managed shells should get garbage collected.

MartinTilo · June 16, 2023, 10:44pm

The Boehm GC is non-generational so these arguments do nothing special.

orionsyndrome · June 16, 2023, 11:42pm

Right, my bad. For some reason I thought there was still some semblance of generations, probably from looking at “incremental”. I know fully it has nothing to do with generational, that’s a brain fart on my part, but part of me wishes GC offered more options than just Disabled and Manual.

Unifikation · June 17, 2023, 12:33am

Absolutely, there is the issue of usefulness for processes of creation that’s a sticking point. For this problem I’ve come to super prefabs, that hold an “entire scene”. In this way it’s possible to edit as needed, with the added benefit of prefabs having an origin/transform, whereas Scenes do not.

StarBornMoonBeam · June 17, 2023, 8:27am

Sorry I don’t mean to be rude. I was following along with the thread for the story of the original content of the thread. And just wanted to say that there is no best way to do things. I didn’t want to discuss other ways.

Kurt-Dekker · June 17, 2023, 2:19pm

I hope you get past this phase of your Unity usage. I went through it too… it was the dark ages.

Once you discover additive scene loading and how you can slice-and-dice your content up for better control of what is loaded when, you will never make a project without additive loading.