Wrong colors in memory Map - False positive fragmentation

Unity Engine
,
1

Hi,

We’ve been using the Memory Map to hunt down memory fragmentation problems and to analyze how our game memory is layed out.
On this project we have some native memory regions (pools) with a size of 16Mb.
At first glance, those pools seem as good as empty in the memory map view. The dark green is Native Memory (Reserved).

6066662--657284--upload_2020-7-8_10-4-8.png
6066662--657284--upload_2020-7-8_10-4-8.png1580×158 7.64 KB

Already here we can see that there are some black lines in our 16Mb pool section.
To the best of my knowledge, this could mean it’s Virtual Memory, as that’s black in the legend?

Anyway, we were baffled our memory regions were empty.
But if we look at the allocation list for this region, it turns out it’s not empty at all…

6066662--657290--upload_2020-7-8_10-6-46.png
6066662--657290--upload_2020-7-8_10-6-46.png1601×353 24.3 KB

If you sum all this, you get 18944Kb of data. with the rounding happening for the MB display, it’s fair to say that we are completely using up our memory buffer.
So, where is the light green “Native Memory (Allocated)” then? Shouldn’t it be completely light green?

Let’s drill deeper and look at those black bars:
We now use rows of 32Kb to try and see what those black bars are.

It turns out, those black bars seem to be my allocated memory.

6066662--657296--upload_2020-7-8_10-11-20.png
6066662--657296--upload_2020-7-8_10-11-20.png1618×180 11.9 KB

Everything selected in between the black bars show nothing at all in the allocation list:
6066662--657299--upload_2020-7-8_10-12-30.png
6066662--657299--upload_2020-7-8_10-12-30.png1602×150 8.22 KB

And then the last thing, the size of those black bars seem to be very wrong.
As you can see in the previous image, the little black bar is a size of 3.7Kb on a 34Kb row. It should be around 1/10th of the width of the bar.
But…
6066662--657305--upload_2020-7-8_10-14-53.png
6066662--657305--upload_2020-7-8_10-14-53.png1613×151 11.1 KB

This small section is 34.1Kb and it takes up about the same amount of space as the 3.7Kb section, while instead it should span more than the entire line.

The fact that these visuals don’t seem to make a lot of sense, makes it very hard for us to actually drill down on our memory problems and we’re almost out of time to fix them.

@MartinTilo could you confirm if my analysis is correct?

If so, I’m willing to modify the package locally to try and have an accurate visual representation but any help would be appreciated.

Thank you in advance,

Pieter

2

Hi Pieter,

I am, to be honest, not quite sure why there are no lighter green bars in your first two screenshots. This could be a bug in the visualization or with the UI where maybe the Display Filters (top right hand drop down) got into a bad serialization state and the Allocations are filtered out. I guess in that case you shouldn’t see the legend item for “Native Memory (Reserved)” though so that seems unlikely.

Could you please file a bug report with this? Just the snapshot file attached should be enough for us to reproduce but please ping me the bug ID so that our Customer QA doesn’t ask you for a full repro project (they can’t always make the right judgment calls in these cases as it requires more knowledge of the Memory Profiler system).

As to the black sections: your selections span the beginning of the next green section and that’s the one you are seeing in the table. The black sections are empty. So it would seem the selection only checks if the address is within the range of the selection, not any point between address + size.

I have to admit: that is UX wise a bit unfortunate. I guess there is a balance to be struck between avoiding that you accidentally overlap your selection into the previous Region/Allocation/Object by a pixel margin (and getting confused about seeing that previous item in your list), and the selected range showing any larger scope items that it overlaps with partially, so that the confusion you experienced here is avoided. I’ve added an issue tracker item for this one myself.

Now as to why those black sections are so big, I’m not too sure myself. I measured and made a calculated estimate that this gap could be ~194B (± pixel and measurement inaccuracies. This was literally me holding a ruler against the screen, dividing the width of the black section by the whole line width and multiplying with 32 KB, ergo, super scientific).

I’ll check back on this one with my colleagues, but for now, I don’t believe ~194B gaps of unused virtual address space should have any (significant) impact on the memory usage of you project.

3

Hi @MartinTilo
I really appreciate your fast and detailed replies on the forums. Thanks for that!

Issue submitted with ID: 1261948

Thanks,
Pieter

4

Thank you. I’ve found the main issue with the colors being wrong (linear vs non-linear color space) but still digging a bit further :slight_smile:

P.s. Here’s the public issue tracker link for this.

5

Hi @MartinTilo ,

Great to hear you’ve found the main issue with the color being wrong! I can’t wait to see that in the memory profiler package.

Unfortunately the public issue tracker link doesn’t seem to be public.

Kind regards,
Pieter

6

dmn, I was sure I had fixed it :smile:, now it is.

1 Like
7

FYI, the fix for the Linear Color Space issue has just landed with the release o com.unity.memoryprofiler@0.2.6-preview.1 . As noted out on the issue tracker item. We’ll continue to look into further issues with colors and Memory Map though.

8

Thank you for the update. Glad to see a long awaited update to the memory profiler :slight_smile:
I hope the “big internal refactor” went well.

1 Like
9

hah, that’s still ongoing :smile:
This release is mostly bug fixes and smaller improvements with initial refactor changes under the hood that fit in with these fixes.

1 Like