Profiling.Recorder?

Hi there!

I just noticed that a new thing appeared on the roadmap, namely a new Profiling API:

Can someone from UT explain this a bit more? Is there going to be an experimental build for this anytime soon? Very interested in this new feature!

Hi!

We are exposing builtin profiler counters you can see in the Profiler Window to runtime - UnityEngine.Profiling namespace.

Provided API is very simple in the first iteration and includes:

class Sampler
{
    public string name;
    public Recorder GetRecorder();
}

// Accumulator, every time sample activated it adds its value to the recorder.
public class Recorder
{
    public bool enabled;
    public long elapsedNanoseconds; // Accumulated time for the previous frame
}

class Profiler
{
    ...
    // Get builtin Sampler
    static public Sampler GetSampler(string name);
    // Return all available sampler names
    static public int GetSamplerNames(List<string> names);
};

Recorder allows to get accumulated time of Begin/End pairs of the specific profiler label for the previous frame.
This information is equivalent to the one you can see in Hierarchy or Timeline view of Profiler Window.

The usage example is:

using UnityEngine;
using UnityEngine.Profiling;

public class ExampleClass : MonoBehaviour
{
    Recorder behaviourUpdateRecorder;
    ExampleClass()
    {
        var sampler = Profiler.GetSampler("BehaviourUpdate");
        behaviourUpdateRecorder = sampler.GetRecorder();
        behaviourUpdateRecorder.enabled = true;
    }
    void Update()
    {
        if (behaviourUpdateRecorder.isValid)
            Debug.Log("BehaviourUpdate time: " + behaviourUpdateRecorder.elapsedNanoseconds);
    }
}

For now only development player will be able to collect data, but further the plan is to enable high-level metrics like BehaviourUpdate, Physics/AI/… update, loading later in non-development players.
The feature is on the way to trunk and hopefully will be available in the alpha builds - we are targeting 5.6.

Thanks a lot for your explanation! This looks like a very easy to use and useful API. I have one critical question for this though: Will this API be thread safe? I myself would really like to profile methods of background threads, however the profiler currently does not support this. Are there any plans for this or to change that?

+1 for multithreaded profiling: currently we built our own system to profile background logic threads.

Another use case where API improvement could help us: We currently capture snapshots in our benchmarking system, but to display the snapshot in the benchmarking result it requires a roundtrip through unity editor to load and process the snapshot. What we actually want is to capture a full snapshot (including user-defined sections) in a player and convert it into a human-readable format right away.

Thanks for the questions!

Yes elapsedNanoseconds property is thread safe.
But it returns the accumulated values (across all threads) of all Begin/End pairs for the specific profiler builtin label in the previous frame.
E.g. in situation like:

Main thread:     1           2 B-E 3
Worker Thread 1: 1 B-E B---E 2     3
Worker Thread 2: 1   B-E     2     3
Worker Thread N: 2      B----2-E   3

In frame 2 elapsedNanoseconds will be equal to the sum of 3 BE delta times and delta time between B and frame boundary. In frame 3 it is BE + frame 2 start to E time.

User-defined sections (dynamic samplers generated by Profiler.BeginSample/EndSample) are not supported in the first iteration. Also this functionality will be deprecated and replaced by Begin()/End() methods in Sampler class (with Sampler<T1,T2,…>.Begin(T1, T2, …) for custom data later on). We are currently working on it.

It would be nice to learn about your use-case more - How do you use the data? Do you hot-tune level based on immediate profiler data? Do you really need full snapshot to be in the player?

Thanks for your explanation Alex, really appreciate it! Another question related to multithreaded profiling (while we’re at it):

Last time I checked, Profiler.Begin/EndSample couldn’t be called from background threads. Are there any plans to change that? Ideally, I would like to see the measurements pop up in the profiler timeline view like they already do for Unity’s native Job scheduler. Like, additionally display the managed threads.

As far as I understand, the built-in profiler is the only tool to profile code in Unity. It’s great most of the time, but particularly for code profiling it’s not. The UI/UX is far behind such tools as ANTS Performance Profiler or JetBrains dotTrace.

Will “deep profile” be supported? Are there plans to add any extra information to the samplers, like corresponding source file names and line numbers? Perhaps, the general question is - are the planned profiler enhancements will make it possible to make a better viewer for code profiling results?

1 Like

We are working on a Profiler improvements which will allow capturing data from managed threads as well. However because of UI/UX challenges, you might need to express interest in the particular thread by calling API like Profiler.SetThreadName(string groupName, string threadName) in order to have view uncluttered (there might be dozens of threads).

Yes, deep profiler will be still supported. It might also get an upgrade in form of lightweight mode where you can set specific methods you want to instrument - so far we have a prototype for Window/OSX+Mono configuration.

Short-term plan is to extend generic samplers with an arbitrary (but strongly typed) data.
Talking about hierarchy/timeline view improvements - yes, this feature is on our radar as “Improve Hierarchy and Timelime view interaction - double click or right click on an overview entry opens the script at the position”.
These improvements might make profiling experience better.

Thank you! This sounds wonderful!

As far as I know, the .net Thread class does have a Name property, isn’t that an option to populate the displayed name field in the profiler view? Or do you mean to only select a few of the managed threads to display in this way? In the latter case, I would suggest to use a different naming scheme like Profiler.RegisterThread to make this intention more clear.

Lastly, do you have any estimate of when we might expect an experimental build for this? I’d be glad to provide feedback once you guys got a proof of concept!:slight_smile:

1 Like

Yes, the purpose of this API is to let you explicitly express whether or not you want to have this thread being displayed in the UI. Profiler.RegisterThread/UnregisterThread makes it clearer.

Recorder functionality will be available with an alpha build of 5.6. But other improvements are still in development and, unfortunately, we can’t say exact day yet, as we have planned those for 5.7. We’ll definitely keep you posted - once we have a stable experimental build :slight_smile: we can definitely share it.

Thanks for your interest!

Thanks! These will be a lifesaver.
Can you also add a clear on play button just like the console? It’s quite useless 99% of the time to have data from the previous run.

1 Like

Is this something that is already available in the beta right now ?

Added ClearOnPlay button to our todo list.

Recorder API will be available in 5.6 alpha 1.

2 Likes

Need a little guidance here. I am working with the new API here to collect and post process these metrics, but I am seeing labels that are not included in profiler window and data that does not match up quite as well as I would have hoped. As an example:

I collect the metrics and then look for hotspots (samplers with the most time spent.), idle reports as 8.6%, while the profiler Window tells me that the most time spent is in “BehaviourUpdate”.

I have a few questions regarding the metrics:

  • When running a workload (game, etc.), is it possible that a recorder becomes valid or invalid? If so is there a recommended approach to determine when this occurs?
  • I see “Idle” as a sampler metric, but do not see idle in the profiler window. Are there other samplers that are not available in the profiler that are in the Profiler?
  • Does the profiler capture editor metrics? I see samplers such as “GuiView RepaintAll” even with nothing in the scene.
  • On the heels of the previous question, is it possible to samplers into something more meaningful in order to profile only certain segments (i.e. CPU, GPU, etc.)?

Thank you so much for the assistance, I look forward to seeing the API grow and mature

Hi,
Thanks for the feedback - this is very valuable for us!

Recorder is valid, when there is correspondent label created at the time of Recorder.Get call. There are 2 use cases for that to consider:

  1. Static labels which are available immediately after engine init. We are working on making high level subset of the labels available in non-dev release builds (such as BehaviourUpdate, Camera.Render, etc.). For that we need a fail-safe if API is used with dev-player label. If static label is not available at startup - it won’t be available any further.
  2. Dynamic labels. Currently these are the ones created internally when calling C# methods (like MonoBehaviour.Update for the specific game class) or ones created with Profiler.BeginSample(string) or CustomSampler.Create calls. You can get recorder for such labels only after they were actually created/used. (Also Profiler.BeginSample/EndSample isn’t useful for Recorders as it has no information about the label in the EndSample call).
    We are adding API to get label type so you can see which labels are “static” and which are “dynamic” in the future.

“Idle” is an internal label which is used to measure Job Queue Worker Thread activity. Recorders collect information from all threads. Thus you can’t see “Idle” labels in Hierarchy View yet, but you can see them in the Timeline View.
We’ll use a better name for it.

Yes, recorders are intended to be independent from Profiler. In the Editor you can also capture Editor-only metrics, which again will be annotated as such later. “GuiView RepaintAll” is the Editor label.

There are no GPU metrics exposed yet, because they introduce a significant overhead to the render pipeline. However, we might add some very high-level metrics in the future. For now all recorders measure CPU time.
Making the labels more meaningful is definitely on our list - cleaning up names and providing tooltip with relevant information is very important.

Thanks for the reply. I have a few more items :slight_smile:

Tooltip information would be great and subsets would really help when the desire is to profile only specific areas. Also, it would be nice to get some sort of functionality to determine when new dynamic labels become available, I’m not sure if polling for recorder validity makes the most sense. I guess when subsets are added it would be nice to simply query a specific subset such as “BehaviorUpdate” and retrieve all MonoBehaviour labels. Also, I have to keep around two lists (Labels and Recorders) to make the correlation between label and data and output anything meaningful. Would be better if the recorder kept a string of the label name. As it currently stands, my approach is to:

  • Create a list of all available labels (and routinely check for new additions)
  • Retrieve each labels recorder
  • Query for recorder validity
  • Get data for each valid recorder
  • Output data using the list of labels and recorders

If you have any tips are advice, it is much apricated.

Yes, polling doesn’t do anything right now as if it was invalid during Get call, it stays invalid.
We can potentially do precreation of labels when you query recorder.
Another consideration is that we didn’t plan recorders as a replacement for profiler. As they are independent, we don’t want to make slower fast path when profiler is disabled. E.g. to get MonoBehaviour labels we have to get script name and actually create a Sampler class on a native side regardless of if we plan to use Recorder or not. That creates an overhead and memory overuse. 95% of dynamic labels are generated only when Profiler is enabled.
I completely agree that ideally you should get Recorder and then if the data is available, just query it. But it is tricky to do it without performance impact right now.

We tried this approach before and decided to remove name property from Recorder. The rationale behind that was that Recorder itself as well as all labels were stored on a native side. And every time we pass a string from native to managed side we do a managed allocation. And that’s very noticeable when you have e.g. onscreen HUD which displays some metrics and store only Recorders querying name every frame for rendering. So in this case API forces to use Tuple or two lists to avoid GC allocations :slight_smile:

We are working on a sample Profiler HUD project that uses Recorders to display main performance in the game. We’ll share it once ready.
Generally main use case for this API is to get overview what’s going on during the frame. How much time subsystems use and what are the bottlenecks when game is played. Especially release games.

Thanks for the feedback, the performance considerations make sense :slight_smile: Looking forward to see the Profiler Project and new changes as the become available. Is it possible top get a timeline on when these may be dropping (subsets for labels, high-level GPU stats, HUD project, etc.)?

Just discovered the Recorder/Sampler and CustomSampler classes, nice!

@alexeyzakharov One thing I very frequently find myself wanting is RAII samplers, for this kind of cases:

    void bla()
    {
        Profiler.BeginSample("bla");

        // do something...

        if (a)
        {
            Profiler.EndSample();
            return;
        }

        // do something...

        if (b)
        {
            Profiler.EndSample();
            return;
        }

        // do something...

        Profiler.EndSample();
    }

Keeping track of all the EndSamples before each return is tedious.

One (bad) way is to wrap the method body in another method, but that’s awfully intrusive for profiling code.

Ideally I’d like to simply do this:

    void bla()
    {
        var sample = new ProfilerSample("bla");

        // do something...

        if (a)
        {
            return;
        }

        // do something...

        if (b)
        {
            return;
        }

        // do something...
    }

… and sample would automatically EndSample when falling out of scope… but C# has no proper RAII (doesn’t call Dispose automatically).

The next best way (still intrusive) is to wrap the code in a using block:

    struct ProfilerSample : IDisposable
    {
        bool disposed;
        public ProfilerSample( string name )
        {
            disposed = false;
            Profiler.BeginSample(name);
        }
        public void Dispose()
        {
            if (!disposed)
            {
                Profiler.EndSample();
                disposed = true;
            }
        }
        public void End()
        {
            Dispose();
        }
    }

    void bla()
    {
        using (new ProfilerSample("bla"))
        {
            // do something...

            if (a)
            {
                return;
            }

            // do something...

            if (b)
            {
                return;
            }

            // do something...
        }
    }

I feel like something like this should be built-in… and if some UT wizard can find a magic trick to do it without a using block, that would be even better :slight_smile: !

1 Like

IIRC, the reason we don’t already have this is that it makes it impossible to strip out the profiler instrumentation in release builds.