Minimum 40KB of garbage when writing string to file

Performance & Profiling Runtime Performance
1

Hi there,

I have a simple question: when I save out my game settings to an xml file in my Application.persistentDataPath, Unity’s profiler notifies me that there have been 40KB of GC Alloc (it always ranges between 40KB and 45KB).
These allocations happen each time I save, and are always due to the constructor of StreamWriter, which is called implicitly when using the XmlWriter.

Can anybody tell me how I can avoid these allocations (I’m trying to reach a 0B GC Alloc goal, since I have to distribute this project one day and I don’t want to add single byte of garbage to anybody using it).

Until now I was very successful in allocating 0B per frame whilst maintaining a very simple and clean code (it’s not that difficult after all to cache and pool stuff beforehand). The only thing allocating right now are these stupid settings… Is there a way to recycle the XmlWriter by resetting it after usage? Or perhaps cache a StreamWriter in some way?
Please don’t tell me to use the PlayerPrefs :slight_smile:
I don’t know if I should even worry about these allocations, but considering that only changing the settings 10 times can already allocate almost half a megabyte of memory, I would really love to avoid that, since there will be plenty of other situations where garbage will be generated inevitably, so why not optimise where it’s possible?

Cheers, and thanks in advance for any constructive responses. Have a great spring everybody! :smile:

2

Why?

Technically… that’s impossible in Unity.

To have zero garbage ever, you’d have to never have a single object.

Which means… no GameObjects, no components… just structs.

I’m not sure exactly what your garbage is… but it’s probably strings, and other bits of boxed data, and conversions as you write to the xml.

I’m willing to be you do a few ‘ToString()’ calls in there if you’re serializing data. (or implicitly converting to string in some way)

This unavoidable. There is always temporary data that’s going to be created and then need to be deleted.

You shouldn’t be worrying this much.

Are you doing this mid game when framerate is super important?

I mean honestly… the act of ‘writing to disk’ is a fairly slow process compared to GC’n a few kilobytes of strings. I can’t really see you doing this during that brutal boss battle where a missed frame could be life or death… you usually do it on load screens, on menus, and while gameplay is minimal.

Just clear all references to unneeded stuff, and call GC.Collect once you’re done saving these settings, that way the garbage collection doesn’t creep up on you later in the game and rather is cleared out asap while you still know the game is in a state that it’s cool to do such a task.

…

I’m honestly unsure what you think garbage is… this isn’t ‘lost’ memory. It’s just used memory that is elligible to be cleared out for reuse. The act of cleaning up the garbage (an automatic task) is slightly expensive is all. It’s still inevitable, and you just need to manage keeping it from happening to frequently, or when framerate is critical.

1 Like
3

so literally all the time when it comes to games except loadscreens :stuck_out_tongue:

Anyway, allocations are avoidable even when converting to string.
instead of doing Write(myInt.ToString()); you could have a StringBuilder + some helper methods (StringBuilder alone won’t fix everything most likely).

Take a good long look at textmesh pro and how its SetText methods (the one that takes string and the one that takes a custom format string and numbers, as well as the one that just takes a single StringBuilder).

Now you also have to consider if it actually makes sense to do these optimizations, which is what @lordofduct was talking about I think.
Unless your stuff happens every frame (or periodic in general) there’s no reason to optimize for zero allocations.
All you’re doing is making your code harder to understand (even if only slightly! it will build up over time!) for an improvement that is measurable but not worth the cost (1. your time, 2. the harder to read code)

However we have too little information whats going on in your situation to really give more advice.

1 Like
4

Thank you SO much for the extensive reply first of all! I highly appreciate it.

I do know what garbage is, and I’ve always been a bit overly paranoid when it comes to automatic memory management (coming from a C++ background many moons ago). Until this point I was able to have my project perform with absolutely 0B of allocations by pre-allocating arrays and recycling them, caching the needed components beforehand, as well as passing compact structs of data when needed. I also put a lot of value in clean, readable and decoupled code. GameObjects that are needed often are pooled, etc…
The profiler just signals those ~40KB of gc alloc when

using (XmlWriter writer = XmlWriter.Create(xmlFilePath, xmlSettings))
{
        writer.WriteStartDocument();
        // All my xml elements & attributes are written in here...
        writer.WriteEndDocument();
}

and the reason for the allocation is StreamWriter…ctor (I just tried adding a test element with attributes to the xml and now it produces 107 KB of garbage). Am I doing something wrong perhaps?

Try and deep-profile just this bit of code:

void Start()
{
    using (XmlWriter writer = XmlWriter.Create(Application.persistentDataPath + "/test.xml"))
    {
        writer.WriteStartDocument();
        writer.WriteStartElement("testElement");
        writer.WriteAttributeString("testVal", "50");
        writer.WriteEndElement();
        writer.WriteEndDocument();
    }
}

this results in this:

The same thing happens btw with JsonUtility and File.WriteAllText…

Isn’t that strange? Why does it take so much memory to write to a file? :expressionless:

5

Thank you too for your reply!

Yes, I tried passing a StringBuilder to the XmlWriter.Create factory method, and I managed to cut down the garbage produced by the Xml creation to 251B, but then I’m left with a StringBuilder that needs to be written out to a file… which brings me back to File.WriteAllText(stringBuilder.ToString()), which produces exactly 40KB of garbage. ;D

6

40 KB sounds like the amount of space a serialized piece of XML would take up. About 400, maybe 500, lines of xml?

That string has to take up some memory somewhere.

…

Also, caching objects doesn’t mean you have 0b of garbage… you’ve just offest all the garbage to a later time. It’s still there… you just haven’t flagged it as eligible to be collected (by unreferencing it). It’s still taking up memory, and it’s going to be cleaned out at whenever point you stop referencing them.

…

You’re not going to get rid of the garbage generated by serializing objects. Serializing inhernetly has to create strings, and those strings take up memory.

You can of course lower the amount of memory used to it’s minimum… but it won’t be 0.

1 Like
7

Yes, 99% of my cached objects are indeed needed and referenced until the end of the session. The other things I did actually store in local variables, which take up minimal amounts (always around 50 to 100B). But those are REALLY not to worry about.
Did you try putting that code in one of your Start() methods? I can’t understand why it suddenly requires 107kb :expressionless:

8

Well…
I have a solution for that but it’s a really shitty one haha.

Create a file stream instead of using WriteAllText and then:
for(int i=0; i<stringBuilder.Length;i++)
fileStream.WriteByte(stringBuilder*);*

or something similar…

Sorry, I don’t know what else you could do.
The only way to truly have zero allocations is to roll your own methods to write into files.
That way you can stream values and stuff directly into the file (optionally post processing them into the right format, for example it wouldn’t be terribly hard to write a function that takes an int and writes the corrosponding UTF8 bytes into a given stream).

I think that’s what you’d have to do if you really don’t want to budge an inch away from perfectly 0 “wasted” bytes.
But I still doubt its worth the gains (unless obviously if you want to use it for advertising or to learn haha)

2 Likes