Diagnostics symbol upload questions & criticism

[ylen_sami]

Hello,

We migrated to Diagnostics from Cloud Diagnostics sometime ago. Now the challenge is to get symbol files uploaded to the diagnostic service. For us its enough that release builds and builds that actually end up public tests have symbols uploaded. We do the submits by CI machine and its nice the Unity has topic on this subject here.

There are many problems with the approach though and state of the diagnostics service which by far is most important cloud services that Unity can actually provide.

Criticisms which explains the question:

1. Its very difficult to find right document page because naming “Diagnostics” vs “Cloud Diagnostics” it also somewhat breaks search engines as "Diagnostics is included in the deprecated “Cloud Diagnostics”.
2. As found in the earlier document of the CI flow the script actually uses /cloud-diagnostics/ REST API call. Which does not make 1) any easier.
3. usymtool is only provided in sh for linux. In my case I need to integrate this to python script. There are problematic approaches like sharing the auth token as environment variable which is also security mistreatment.
4. Depending on platform the usymtool binary is used on iOS in build directory and in android Unity directory and there are several binaries. Please only one common tool that handles all!
5. Weirdly the sh script is not downloaded from the CI document but from under unity services page which requires authentication. This could be explained with if the page included project ids etc directly to the script but it doesnt.
6. Service account configuration isnt self explanatory. There are different priviledge levels for whole organization and project which have different rules. Auth tokens are referred as"keys".
7. In services menu you can find the “Observation” stuff under LiveOps. In this site you cant access diagnostics tags within liveops and the site and you cant include the tags to liveops which obviously are under liveops but its in DevOps.
8. In the forum you cant make topic without editor version even you are speaking of services and REST API that is not related to editor.

As described the Diagnostics service is really a mess and I wonder how it can be like this because this is really one of the most important cloud features!!!

Questions:
a) usymtool.sh that is provided in the services portal and that is referred in the document. Can it be written in python fully (also the bit usymtool binary). Does any solution exists for that?
b) This seems rather perfect use case for Unity CLI does such solution exists?

Thanks and sorry for the rant but the quality is Unity is dropping and its really frustrating.

[jlemosz]

Hi there, thank you for the detailed feedback. I understand your frustration.

The transition between Cloud Diagnostics and the newer Diagnostics service, along with the split in UI placement (Observation vs. DevOps vs. LiveOps) and the use of legacy API endpoints, has indeed created a somewhat disjointed developer experience.

Here are the answers to your specific questions regarding usymtool and CI integration:

a) usymtool.sh that is provided in the services portal and that is referred in the document. Can it be written in Python fully (also the bit usymtool binary). Does any solution exist for that?

The short answer is yes for the wrapper script, but no for the binary.

There is currently no official or community-provided pure Python solution that handles the entire pipeline, but you could easily rewrite the .sh wrapper script in Python.

The usymtool.sh script provided by the dashboard is simply a wrapper that performs three steps:

1. Validates the required paths (Editor, project, IL2CPP, etc.) depending on the platform.
2. Authenticates with the Unity Services API using your service account credentials to fetch a time-limited token.
3. Invokes the usymtool binary and passes the token via the USYM_UPLOAD_AUTH_TOKEN environment variable.

You could write a Python script using the requests library to fetch the token and the subprocess module to construct the correct platform paths and invoke the usymtool binary.

However, you cannot easily rewrite the usymtool binary itself in Python. The usymtool executable is a compiled Go application. It handles complex, low-level tasks such as parsing DWARF, Mach-O, and ELF symbol tables, as well as mapping C++ file/line information back to the original IL2CPP C# lines. Reimplementing that symbolication logic natively in Python would be a massive undertaking.

If you rewrite the script in Python, you will still need to pass the token to the binary via os.environ when calling subprocess.run(). While passing secrets via environment variables is generally discouraged for long-lived system variables, injecting them temporarily into a child process’s environment dictionary is standard practice and prevents the token from being exposed in process lists (which occurs when passed via command-line arguments).

b) This seems rather perfect use case for Unity CLI does such solution exists?

There is no native command in the Unity CLI (unity command) or the Unity Gaming Services CLI (ugs command) specifically for uploading symbol files to Diagnostics.

While the new experimental Unity CLI has introduced extensive project, Editor, and build automation commands (such as unity build, which supports Android symbol generation flags), uploading those symbols to the Diagnostics service still strictly relies on the standalone usymtool Go binary bundled with the Editor.

For now, your best path forward for CI integration is to write a Python equivalent of the usymtool.sh/usymtool.ps1 script to handle your platform path discrepancies and token fetching, which then triggers the usymtool binary to do the heavy lifting.

[ylen_sami]

Thank you for the detailed answer. I will create somewhat the suggested solution in python as we dont have any other good solution at the time being.

Final comments though. I noticed that running the binary file in commandline actually outs somewhat unity editor log type output. Which suggests that the usymtool binary has been written as Unity script? If thats the case it should be somewhat easy deal to include it into the editor for the Unity CLI or another BuildPipeline call after the actual build or simply add it as commandline argument for the editor itself.

[jlemosz]

You’re welcome! While the log output from usymtool might look similar to Unity Editor logs, usymtool is not a Unity C# script.

As I mentioned previously, it is a standalone application written in Go. The log format you are seeing (e.g., time="..." level=info msg="...") is simply the default output format of standard Go logging libraries (like Logrus), which produces structured text that happens to look like Unity’s diagnostic logs.

Also, to address your suggestion about integrating it into the Editor, BuildPipeline, or CLI: it is actually already integrated natively into the Unity Editor.

Here is how it works under the hood and why the standalone scripts exist for CI:

When you enable Diagnostics in your Unity project, the Editor automatically hooks into the BuildPipeline. As a post-build step, the Editor internally constructs the correct paths, fetches a session token, and silently executes the usymtool binary in the background to process and upload your symbols.

If you build your project manually from the Unity Editor UI, you will see that post-build step run and successfully upload the symbols.

The native BuildPipeline integration works flawlessly when a user is logged into the Unity Hub/Editor. However, in a headless CI/CD environment using -batchmode, the Editor historically struggles to authenticate and retrieve the necessary token to upload symbols to Unity Cloud (frequently throwing 401 Unauthorized or Empty URI errors because it relies on standard user session tokens rather than CI service accounts).

Because the Editor’s native batch mode authentication has limitations for this specific cloud service, Unity provides the usymtool binary alongside wrapper scripts (usymtool.sh / usymtool.ps1). That decouples the symbol upload from the Editor’s BuildPipeline, allowing you to:

• Bypass the Editor’s personal user authentication limitations in headless mode.
• Authenticate securely using a Service Account (which the Editor’s standard -username / -password CLI flags don’t support well for that service).

So to summarize, while usymtool is already a built-in post-process step in the Editor, that built-in step expects a logged-in user session. For CI pipelines, extracting that step into a separate script (like the Python script you are planning to write) is currently the most reliable way to authenticate via a Service Account and guarantee your symbols reach the Diagnostics service.

[ylen_sami]

@jlemosz

Here is how it works under the hood and why the standalone scripts exist for CI:

When you enable Diagnostics in your Unity project, the Editor automatically hooks into the BuildPipeline. As a post-build step, the Editor internally constructs the correct paths, fetches a session token, and silently executes the usymtool binary in the background to process and upload your symbols.

If you build your project manually from the Unity Editor UI, you will see that post-build step run and successfully upload the symbols.

The native BuildPipeline integration works flawlessly when a user is logged into the Unity Hub/Editor. However, in a headless CI/CD environment using -batchmode, the Editor historically struggles to authenticate and retrieve the necessary token to upload symbols to Unity Cloud (frequently throwing 401 Unauthorized or Empty URI errors because it relies on standard user session tokens rather than CI service accounts).

Because the Editor’s native batch mode authentication has limitations for this specific cloud service, Unity provides the usymtool binary alongside wrapper scripts (usymtool.sh / usymtool.ps1). That decouples the symbol upload from the Editor’s BuildPipeline, allowing you to:

• Bypass the Editor’s personal user authentication limitations in headless mode.
• Authenticate securely using a Service Account (which the Editor’s standard -username / -password CLI flags don’t support well for that service).

So to summarize, while usymtool is already a built-in post-process step in the Editor, that built-in step expects a logged-in user session. For CI pipelines, extracting that step into a separate script (like the Python script you are planning to write) is currently the most reliable way to authenticate via a Service Account and guarantee your symbols reach the Diagnostics service.

This answer is what we really needed.

Its probably somehow related to the CI environment having problems to authentication. Its most likely related to this as the Hub does do weird things at times.

I am puzzled thoughj. If this is know issue why doesnt unityeditor batchmode have separately provided service account (token or auth) switches? Guess that would basically do same as writing the same stuff outside the editor code?

Its difficult to change now – in Unity Editor – but I just want to state that our CI machine creates many builds during one day and we do not need symbols uploaded for every possible build. I would like to suggest though that you add BuildOption to BuildPipeline.BuildPlayer that instructs the Unity builder NOT to sent the symbols. This should exists only because you now support the CI symbol upload. Once thats in there then could be another function that you could perhaps give service account information that the external script would not even be needed?

If users can include the service account information (token or auth) either in Unity batchmode argument or in C#. It would make it possible to actually handle the token handling either in CI script or C# CI code itself. Because of also firewall policies and network errors and logging it might be good that you could do it that way?

[jlemosz]

Below are the answers to your new questions:

If this is know issue why doesnt unityeditor batchmode have separately provided service account (token or auth) switches? Guess that would basically do same as writing the same stuff outside the editor code?

The lack of a -serviceAccountToken argument for the Editor comes down to legacy architecture.

The Unity Editor’s -username and -password batchmode arguments are deeply tied to the Unity Hub, Unity Licensing, and legacy Unity ID user sessions. When the Editor builds a project, the internal method that handles native symbol uploads (CrashReporting.GetUsymUploadAuthToken()) relies on a standard user login flow to populate project IDs and fetch user-level permissions.

Because Service Accounts bypass standard Unity user sessions and aren’t tied to Unity Licensing, retrofitting the entire Editor authentication stack to accept Service Accounts for this one service was deemed less practical than creating a decoupled CI tool. By extracting the upload logic into usymtool and providing wrapper scripts, we allowed CI/CD pipelines to bypass the Editor’s user-session limitations entirely.

Once thats in there then could be another function that you could perhaps give service account information that the external script would not even be needed?

While a specific BuildOption.DoNotUploadSymbols enum doesn’t exist, you can already achieve that behavior natively.

If you run your CI unity -batchmode command without passing the -username and -password arguments, the Editor’s automatic symbol upload will simply fail (usually throwing a 401 Unauthorized or Empty URI warning in the logs), but the build itself will still succeed, and the symbols will still be generated on disk.

That essentially gives you the “opt-out” behavior you want. You can let the Editor generate the symbols for every build, but only upload them when you explicitly choose to.

If users can include the service account information (token or auth) either in Unity batchmode argument or in C#. It would make it possible to actually handle the token handling either in CI script or C# CI code itself. Because of also firewall policies and network errors and logging it might be good that you could do it that way?

Since you want to control exactly which builds get their symbols uploaded, you could do the following:

1. Run your standard BuildPipeline.BuildPlayer without Editor credentials so the native upload is skipped.
2. In your C# CI script, evaluate if the current build is a release/public test candidate.
3. If symbols are needed, use UnityWebRequest to call the Unity Services API using your Service Account’s Basic Auth header to fetch a temporary token.
4. Use System.Diagnostics.Process to invoke the usymtool executable directly from your C# script.

Here is a conceptual example of how you can handle this entirely within Unity C#:

using System.Diagnostics;
using UnityEngine;
using UnityEditor;

public class CISymbolUploader
{
    public static void UploadSymbolsIfRequired(string buildOutputPath)
    {
        // 1. Check if this is a build we care about
        if (!IsPublicReleaseBuild()) return;

        // 2. Fetch the token (Implement your UnityWebRequest logic here)
        string token = FetchServiceAccountToken(); 

        // 3. Locate usymtool (Bundled in Editor/Data/Tools/ on Windows, Unity.app/Contents/Tools/ on Mac)
        string usymtoolPath = EditorApplication.applicationContentsPath + "/Tools/usymtool.exe"; 

        // 4. Setup the process
        ProcessStartInfo startInfo = new ProcessStartInfo
        {
            FileName = usymtoolPath,
            // Pass the required platform flags and paths 
            Arguments = $"-symbolPath \"{buildOutputPath}\" -forceUpload", 
            UseShellExecute = false,
            RedirectStandardOutput = true,
            RedirectStandardError = true
        };

        // 5. Inject the token securely into the environment variables
        startInfo.EnvironmentVariables["USYM_UPLOAD_AUTH_TOKEN"] = token;
        startInfo.EnvironmentVariables["USYM_UPLOAD_URL_SOURCE"] = "https://perf-events.cloud.unity3d.com/url";

        // 6. Run the tool and capture Unity-style logs natively
        using (Process process = Process.Start(startInfo))
        {
            process.OutputDataReceived += (sender, args) => UnityEngine.Debug.Log($"[usymtool] {args.Data}");
            process.ErrorDataReceived += (sender, args) => UnityEngine.Debug.LogError($"[usymtool] {args.Data}");
            
            process.BeginOutputReadLine();
            process.BeginErrorReadLine();
            process.WaitForExit();
        }
    }
}

By leveraging System.Diagnostics.Process and environment variables in C#, you get the benefits you mentioned: better network handling, adherence to your firewall policies, and the ability to keep all CI logic unified inside your C# codebase without relying on external Bash or Python scripts.

[ylen_sami]

@jlemosz

Because Service Accounts bypass standard Unity user sessions and aren’t tied to Unity Licensing, retrofitting the entire Editor authentication stack to accept Service Accounts for this one service was deemed less practical than creating a decoupled CI tool. By extracting the upload logic into usymtool and providing wrapper scripts, we allowed CI/CD pipelines to bypass the Editor’s user-session limitations entirely.

I understand that Unity users could still perhaps implement this inside Unity script like your example does. This will probably be our approach as it makes sense to do it inside the builder script.

@jlemosz

If you run your CI unity -batchmode command without passing the -username and -password arguments, the Editor’s automatic symbol upload will simply fail (usually throwing a 401 Unauthorized or Empty URI warning in the logs), but the build itself will still succeed, and the symbols will still be generated on disk.

We do pass -username and -password to the batchmode. I am not sure about the following if it still happens but I recall we added the -username and -password switches some years ago to pass some license error that failed the whole build process. Unity has internally changed so I am not sure if that still happens.

However I would think that this is probably not functionality you internally want in license and payment side. And it can also possibly screw things like adding “made with unity” splash etc which is not fine so there has to be different way.

Personally I am bit puzzled why you simply dont add for example the BuildOptions approach as it would give users option to reduce stress on your services – if users wanted to include the data on important builds. I know its too late to do it now but actually the symbol upload should be disabled by default because I suspect it has caused some thread/deadlock behaviour in batchmode where Unity does not exit correctly (probably due to corporation firewall settings). It should be either manual call or BuildOptions with warning that it can fail due to multiple reasons.

@jlemosz

Since you want to control exactly which builds get their symbols uploaded, you could do the following:

1. Run your standard BuildPipeline.BuildPlayer without Editor credentials so the native upload is skipped.
2. In your C# CI script, evaluate if the current build is a release/public test candidate.
3. If symbols are needed, use UnityWebRequest to call the Unity Services API using your Service Account’s Basic Auth header to fetch a temporary token.
4. Use System.Diagnostics.Process to invoke the usymtool executable directly from your C# script.

Here is a conceptual example of how you can handle this entirely within Unity C#:

This is really helpful. Thank you for the example. Though I do point out that there is no valid way to prevent Unity to push the symbols with BuiildPipeline.BuildPlayer. If there would be valid way to prevent symbol upload during BuildPipeline.BuildPlayer this would be perfect approach.

Though I would also like that usymtool could have commandline arguments that it wouldnt parse the stuff from environment variables. Environment variables dont log and they lead to wondering stuff everytime. Its better that the used parameters would show in the commandline calls. Also there been many security issues in different build environment plugins (Jenkins) for which reason I dont like this approach either.

---

Further question: We own enterprise license and we are wondering also if its possible somehow handle the authenticating differently using floating licenses etc. I believe there was some kind of CI license type existing?