Deep Profiling on WebGL (Unity 2019.4)

[santiandrade]

Hi guys,

I’m trying to do a deep profiling on a local WebGL build but I can’t get it. I’m using Unity 2019.1.0f1 and follow the next steps:

• Go to Build Settings.
• Check “Development Build”.
• Check “Autoconnect Profiler”.
• Check “Deep Profiling Support”.

After the build is ready, I execute it and I get the next warning on the Unity Editor console:

"Autoconnected Player Stopping profiler. Profiler is not able to flush data to a file or socket and exceeded maximum allowed memory for buffering.
Please use Profiler.maxUsedMemory API or -profiler-maxusedmemory command line parameter to increase maximum allowed memory usage.
Using 16793600 bytes while Profiler.maxUsedMemory is 16777216 bytes."

As the warning message indicates, I try to use the Profiler.maxUsedMemory API to increase the maximum allowed memory. So I write this on the Awake of one of my scripts:

private void Awake()
{
     Profiler.maxUsedMemory = 20000000;
}

I repeat all the previous steps and, after execute again, I get the same warning in console:

“Autoconnected Player Stopping profiler … Using 16793600 bytes while Profiler.maxUsedMemory is 16777216 bytes.”

It’s like it’s ignoring the code I writed in the awake function.

Any help from someone?

Thank you in advance!

Note 1: I’m using Chrome.
Note 2: I have tried to profile without the “deep profiling” option and it works perfectly.
Note 3: I was reviewing the official documentation and, in theory, the deep profiling is allowed in Unity 2019.4 for WebGL builds: Unity - Manual: Build your WebGL application

[iscf_1]

I got the same problem un Unity 2019.4.4f1 without deep profiling. I think it’s a parameter in the build but it’s crazy there is no documentation int the profiler section about it, and no option to change it in the Player Settings.
I will try to do like in this page :

Unity - Scripting API: Profiling.Profiler.maxUsedMemory

But it’s so unintuitive to do a script for that.
If someone have the answer to override this with elegant solution, please tell us because it’s a burden to optimize witout the profiler.

EDIT : It works, maybe because i have set this script at first position in execution order.

[alexeyzakharov]

Hi!

@santiandrade

So I write this on the Awake of one of my scripts:

Are you sure the script is executed? If you add Debug.Log("**Profiler.maxUsedMemory:** " + **Profiler.maxUsedMemory**) to Awake - what does it print?

I would probably use [RuntimeInitializeOnLoadMethod] to ensure the setting is applied

class ProfilerSettingUpdater
{
    [RuntimeInitializeOnLoadMethod]
    static void InitProfiler()
    {
        Profiler.maxUsedMemory = 20000000;
    }
}

For WebGL it would be probably more convenient to add -profiler-maxusedmemory 20000000 to boot.config file. Also it is important to note that for WebGL the max memorySize should be adjusted to accommodate extra memory for profiling.

[alexeyzakharov]

@iscf_1

I think it’s a parameter in the build but it’s crazy there is no documentation int the profiler section about it, and no option to change it in the Player Settings.

Valid point. Would an existence of Profiler Player Settings help you to discover and configure profiler behavior?

Could you please explain your use case in more details?

• Do you use option to make profiler work and make sure you can get data?
• Do you use those in CI/automated builds?
• Which platforms do you use it for?
• What is more important for you by default - getting full data or making sure profiler has limited memory budget? Do you need this configurable?

[santiandrade]

Hi! Thanks for the answers,

@alexeyzakharov

I would probably use [RuntimeInitializeOnLoadMethod] to ensure the setting is applied

Using [RuntimeInitializeOnLoadMethod] I was be able to configure Profiler.maxUsedMemory to 30000000, but when I execute the build, the Unity editor always throws the warning saying:

“Autoconnected Player Stopping profiler. Profiler is not able to flush data to a file or socket and exceeded maximum allowed memory for buffering.
Please use Profiler.maxUsedMemory API or -profiler-maxusedmemory command line parameter to increase maximum allowed memory usage.
Using 30015488 bytes while Profiler.maxUsedMemory is 30000000 bytes.”

After that, I try to increment the assigned value to something greater than the warning is saying (30015488), but it always throws a new warning indicating that I need a bit more memory than I set. So I try to increment the value again… and so on, until the point where Chrome freezes when I try to load the app.

@alexeyzakharov

Also it is important to note that for WebGL the max memorySize should be adjusted to accommodate extra memory for profiling.

As the official documentation says: “This setting has been deprecated and does not have any effect on WebAssembly builds”.

Maybe my app needs more memory than the deep profiling can handle in a WebGL build?

Regards.

[iscf_1]

I got now on other problem with this but bug repport don’t work so I’m a bit frustrated.

I have tried to augment the memory buffer for profiling heavy build webgl in my project but the FPS in build drop and finally crash with
"
The browser could not allocate enough memory for the WebGL content. If you are the developer of this content, try allocating less memory to your WebGL build in the WebGL player settings.
"

Otherwise the profiler is down with
"
Autoconnected Player Stopping profiler. Profiler is not able to flush data to a file or socket and exceeded maximum allowed memory for buffering.
Please use Profiler.maxUsedMemory API or -profiler-maxusedmemory command line parameter to increase maximum allowed memory usage.
Using 16793600 bytes while Profiler.maxUsedMemory is 16777216 bytes.
"
I have done this script before it crash :

Unity - Scripting API: Profiling.Profiler.enabled

I tried with and without the script and the issue occur only when the build is in Developement with Autoconnect profiler, or if i enable this line

Profiler.enabled = true;

Maybe the issue is like there

Is it normal for Mono to slowly increase in size in the memory profiler?

In my game, obviously I want to keep memory usage as low as possible, and most importantly avoid memory leaks. However, looking at the memory profiler I've noticed that the value for mono slowly increases over time. Currently, it starts at about 14.3 mb, and then increases to about 21.0 mb after about 5 minutes or so, before going back to it's initial value (When I assume the garbage collector kicks in). Now, this definitely seems like it's probably not a huge issue, but if there's a way to avoi…

So before dertermining a use case for memory in player settings, maybe make some security or more documentation about it would be great.

And repare the bug repport.

EDIT: I have sent mail at bugs@unity3d.com with the files associated, for the problem with bug repport i have send also a mail and write there:
Can't send bug report

[iscf_1]

I have forgot that Build&Run alwais fail on my computer, Windows 10, so I use firefox with local files enabled.
I build for the profiler with development and autoconnect and it works.

It permit to add some context that if you want to optimize for WebGL, you must have some strong mind or be crazy with all this problems. And i don’t speak all the features like video player or UnityWebRequest that i have troubled since the start of my project that occur only in WebGL.

Anyway sorry for this post but i’m frustrated with this story.

[MartinTilo]

@iscf_1

EDIT: I have sent mail at bugs@unity3d.com with the files associated, for the problem with bug repport i have send also a mail and write there:
https://discussions.unity.com/t/755867

Did you get a reply to that? As discussed in that thread, our bug tracker server had an expired certificate that has been solved since. It would be really helpful if either you or @santiandrade could file a bug report through the normal process for this, now that it is back up.

[santiandrade]

@iscf_1

I have tried to augment the memory buffer for profiling heavy build webgl in my project but the FPS in build drop and finally crash with
"
The browser could not allocate enough memory for the WebGL content. If you are the developer of this content, try allocating less memory to your WebGL build in the WebGL player settings.
"

Otherwise the profiler is down with
"
Autoconnected Player Stopping profiler. Profiler is not able to flush data to a file or socket and exceeded maximum allowed memory for buffering.
Please use Profiler.maxUsedMemory API or -profiler-maxusedmemory command line parameter to increase maximum allowed memory usage.
Using 16793600 bytes while Profiler.maxUsedMemory is 16777216 bytes.
"

I’m getting exactly the same problem when I activate the “Deep Profiling Support” checkbox in the build options. Finally I found a middle solution that maybe can help you:

You can use samples for analize specific parts of code that you want:

Profiler.BeginSample("some id");
// code block to analyze...
Profiler.EndSample();

In this way, you can create a build (without “Deep Profiling Support” option) and when you play the build and open Profiler in your editor, you will find the id’s that you used on each BeginSamples(…) in the Profiler data. It is not the best solution, but at least I was be able to analyze better the performance of my app.

[MartinTilo]

More ideal because it has less overhead would be to use the ProfilerMarker API instead of Profiler.BeginSample/EndSample. Simply
because ProfilerMarker operates on an IntPtr, while Profiler.BeginSample/EndSample needs to send a string to native code on each call that needs to be looked up in a hashmap…
The ProfilerMarker API is also getting newer additions like MetaData support (stay tuned for more on that later in 2020.2.)

Still, even if this helps to work around the issue that deep profiling fails on WebGL, and that it is generally a good idea to place some ProfilerMarlers so you can get more info without the full deep profiling overhead, we’d still like to fix this up so that it’s a choice and not a tedious workaround. A bug report would help us get there faster and more reliably because we’d get something we can test against and see if there’s anything specific that keeps it from working.

[iscf_1]

@MartinTilo

More ideal because it has less overhead would be to use the ProfilerMarker API instead of Profiler.BeginSample/EndSample. Simply
because ProfilerMarker operates on an IntPtr, while Profiler.BeginSample/EndSample needs to send a string to native code on each call that needs to be looked up in a hashmap…
The ProfilerMarker API is also getting newer additions like MetaData support (stay tuned for more on that later in 2020.2.)

Still, even if this helps to work around the issue that deep profiling fails on WebGL, and that it is generally a good idea to place some ProfilerMarlers so you can get more info without the full deep profiling overhead, we’d still like to fix this up so that it’s a choice and not a tedious workaround. A bug report would help us get there faster and more reliably because we’d get something we can test against and see if there’s anything specific that keeps it from working.

** @MartinTilo Sorry for the late reply. I have send a conform bug repport and it’s case 1265204.**
Otherwise, in my case it’s not only the code I want to profile but draw call’s performances. So i don’t know if the ProfilerMarker API will help for this case.

[MartinTilo]

@iscf_1

Otherwise, in my case it’s not only the code I want to profile but draw call’s performances. So i don’t know if the ProfilerMarker API will help for this case.

Unless you are using an SRP, Deep Profiling would not affect what details you can see in the Profiler regarding Rendering and GPU performance. Are you interested in seeing Deep inspection in how an SRP is issuing commands? Draw call stats will be shown in the Rendering Profiler Module. The Frame Debugger can show you what Draw calls happened why, and the GPU Profiler Module would show the timings for them.

The SRPs already should have quite some ProfilerMarkers in them and if you need to, you can change their code locally to add more. So I’m not sure I understand the problem correctly…

[MartinTilo]

And thank you for the bug report @iscf_1

[iscf_1]

Sorry I wasn’t clear. I’m not using deep profiling, but even without it i got the memory leak problem. Deep profiling make it worse i think (i haven’t tested). But to have the details draw calls on spike, i use the profiler because it’s concise for batching information for exemple instead of the frame debugguer, i use the frame debugguer only for visualising how occur the problem, and no why or when.

[MartinTilo]

@iscf_1

but even without it i got the memory leak problem

With Memory Leak Problem, do you mean the issue that the Profiler will log something like this?

Profiler is not able to flush data to a file or socket and exceeded maximum allowed memory for buffering.

and while increasing the maxUsedMemory it eventually hits the ceiling and won’t start?

Or do you mean the issue you described in this thread ? If so, I am struggling to find the connection between Mono memory increasing, Draw Call details and Deep Profiling…

I guess besides my advice in that thread to turn on Deep Profiling to get details on GC.Alloc samples. Turning on Allocation Call Stacks should be the better way (less profiler overhead, deeper call stack) to find these GC.Allocs and get full call stacks on them. However, collecting the Call Stacks from built Players is only possible in 2019.3 and up so before that you can only use this option in the Editor.

[iscf_1]

It seems like the thread, but it hit ceiling navigator, not editor i think with

The browser could not allocate enough memory for the WebGL content. If you
are the developer of this content, try allocating less memory to your WebGL
build in the WebGL player settings.

The way the memory increase even in a empty project is abnormal, but not lead to problems if the project remain light I think.

[aromana]

Is there any update here? Deep profiling is still not usable for WebGL builds with the latest version of Unity 2020.2.

The issue I run into is as described by others – no matter how much memory I allocate to the Profiler, it is not enough for deep profiling and the profiler disconnects. If I put the memory usage effectively infinitely high, the web page will hang under all browsers, presumably due to exhausing javascript memory limits.

I am aware of the workaround suggested in this thread to manually tag my code with profiler markers, and that’s what I’ll have to do, but if deep profiling doesn’t work in WebGL, that should either be made clear or fixed so as not to waste folks time (as I have). It’s also a quite useful feature – showing the exact line of code an issue is on, usually, rather than just the general area – so I really hope it’s able to be improved!

Not knowing much about how the feature works… naively, it sounds like the solution might be to have a “slimmed down” profiler protocol for running in memory-constrained environments like WebGL to mitigate this issue, especially since it seems like the general trend is for more memory intensive profiling features going forward.

[jesusluvsyooh]

@alexeyzakharov

Hi!

Are you sure the script is executed? If you add Debug.Log("**Profiler.maxUsedMemory:** " + **Profiler.maxUsedMemory**) to Awake - what does it print?

I would probably use [RuntimeInitializeOnLoadMethod] to ensure the setting is applied

class ProfilerSettingUpdater
{
    [RuntimeInitializeOnLoadMethod]
    static void InitProfiler()
    {
        Profiler.maxUsedMemory = 20000000;
    }
}

For WebGL it would be probably more convenient to add -profiler-maxusedmemory 20000000 to boot.config file. Also it is important to note that for WebGL the max memorySize should be adjusted to accommodate extra memory for profiling.

this fixed it for me, needed an extra 0 on the end though
Edit, actually im not sure if it did work now…

However using args did the trick, when running build
-profiler-maxusedmemory 200000000