No activity received from the player when running tests on standalone build via Teamcity

[LCGC_Developers]

We have a couple of playmode tests which measure performance using the Testing and Performance frameworks.

When running the command locally the tests runs and test results are produced. When running that same command through Teamcity it doesnt work and return exit code 3.

I tried solutions posted in this thread ( How to run Target Platform tests in Docker? ) but none of them worked.

Has anyone had any luck running tests on a standalone build through Teamcity?

The command:

"Unity.exe" -projectPath "./" -batchmode -runTests -profiler-enable -testPlatform StandaloneWindows64 -testResults "./performance_testresults.xml" -editorTestsCategories Performance -logfile "./log.log"

The error logs that Teamcity produces:

DisplayProgressNotification: Build Successful
Unloading 0 Unused Serialized files (Serialized files now loaded: 0)
Loaded scene ‘Temp/__Backupscenes/0.backup’
Deserialize: 8.245 ms
Integration: 53.772 ms
Integration of assets: 0.004 ms
Thread Wait Time: 0.026 ms
Total Operation Time: 62.048 ms
Unloading 0 unused Assets / (0 B). Loaded Objects now: 11476.
Memory consumption went from 2.57 GB to 2.57 GB.
Total: 46.677500 ms (FindLiveObjects: 1.098700 ms CreateObjectMapping: 1.336100 ms MarkObjects: 44.201800 ms DeleteObjects: 0.039200 ms)

DisplayProgressbar: Test Runner -
Executing IPostBuildCleanup for: Unity.PerformanceTesting.Editor.TestRunBuilder.
UnityEngine.StackTraceUtility:ExtractStackTrace ()
UnityEngine.DebugLogHandler:LogFormat (UnityEngine.LogType,UnityEngine.Object,string,object[ ])
UnityEngine.Logger:Log (UnityEngine.LogType,object)
UnityEngine.Debug:Log (object)
UnityEditor.TestTools.TestRunner.TestRun.Tasks.BuildActionTaskBase1/<ExecuteMethods>d__9<UnityEngine.TestTools.IPostBuildCleanup>:MoveNext () (at Library/PackageCache/com.unity.test-framework@2.0.1-pre.18/UnityEditor.TestRunner/TestRun/Tasks/BuildActionTaskBase.cs:57) UnityEditor.TestTools.TestRunner.TestRun.Tasks.BuildActionTaskBase1/d__8<UnityEngine.TestTools.IPostBuildCleanup>:MoveNext () (at Library/PackageCache/com.unity.test-framework@2.0.1-pre.18/UnityEditor.TestRunner/TestRun/Tasks/BuildActionTaskBase.cs:41)
UnityEditor.TestTools.TestRunner.TestRun.TestJobRunner:ExecuteStep () (at Library/PackageCache/com.unity.test-framework@2.0.1-pre.18/UnityEditor.TestRunner/TestRun/TestJobRunner.cs:171)
UnityEditor.TestTools.TestRunner.TestRun.TestJobRunner:ExecuteCallback () (at Library/PackageCache/com.unity.test-framework@2.0.1-pre.18/UnityEditor.TestRunner/TestRun/TestJobRunner.cs:109)
UnityEditor.EditorApplication:Internal_CallUpdateFunctions ()

(Filename: Library/PackageCache/com.unity.test-framework@2.0.1-pre.18/UnityEditor.TestRunner/TestRun/Tasks/BuildActionTaskBase.cs Line: 57)

Asset Pipeline Refresh: Total: 0.021 seconds - Initiated by RefreshV2(NoUpdateAssetOptions)
[00:00:00] Finished 1 Bake Ambient Probe job (0.00s execute, 0.00s integrate, 0.06s wallclock)
AcceleratorClientConnectionCallback - disconnected - 10.0.1.103:10080
AcceleratorClientConnectionCallback - disconnected - 10.0.1.103:10080
Exception: Test execution timed out. No activity received from the player in 600 seconds.
at UnityEditor.TestTools.TestRunner.TestRun.Tasks.PlayerHeartBeatTask+d__3.MoveNext () [0x00130] in D:\BuildAgent\work\game_windows_client\Library\PackageCache\com.unity.test-framework@2.0.1-pre.18\UnityEditor.TestRunner\TestRun\Tasks\PlayerHeartBeatTask.cs:39
at UnityEditor.TestTools.TestRunner.TestRun.TestJobRunner.ExecuteStep () [0x0015e] in D:\BuildAgent\work\game_windows_client\Library\PackageCache\com.unity.test-framework@2.0.1-pre.18\UnityEditor.TestRunner\TestRun\TestJobRunner.cs:171
UnityEngine.DebugLogHandler:Internal_LogException(Exception, Object)
UnityEngine.DebugLogHandler:LogException(Exception, Object)
UnityEngine.Logger:LogException(Exception, Object)
UnityEngine.Debug:LogException(Exception)
UnityEditor.TestTools.TestRunner.TestRun.TestJobRunner:ExecuteStep() (at Library\PackageCache\com.unity.test-framework@2.0.1-pre.18\UnityEditor.TestRunner\TestRun\TestJobRunner.cs:214)
UnityEditor.TestTools.TestRunner.TestRun.TestJobRunner:ExecuteCallback() (at Library\PackageCache\com.unity.test-framework@2.0.1-pre.18\UnityEditor.TestRunner\TestRun\TestJobRunner.cs:109)
UnityEditor.EditorApplication:Internal_CallUpdateFunctions()

(Filename: Library/PackageCache/com.unity.test-framework@2.0.1-pre.18/UnityEditor.TestRunner/TestRun/Tasks/PlayerHeartBeatTask.cs Line: 39)

An unexpected error happened while running tests.
UnityEngine.StackTraceUtility:ExtractStackTrace ()
UnityEngine.DebugLogHandler:LogFormat (UnityEngine.LogType,UnityEngine.Object,string,object[ ])
UnityEngine.Logger:Log (UnityEngine.LogType,object)
UnityEngine.Debug:LogError (object)
UnityEditor.TestTools.TestRunner.Api.CallbacksDelegator:RunFailed (string) (at Library/PackageCache/com.unity.test-framework@2.0.1-pre.18/UnityEditor.TestRunner/Api/CallbacksDelegator.cs:64)
UnityEditor.TestTools.TestRunner.TestRun.TestJobRunner:ExecuteStep () (at Library/PackageCache/com.unity.test-framework@2.0.1-pre.18/UnityEditor.TestRunner/TestRun/TestJobRunner.cs:215)
UnityEditor.TestTools.TestRunner.TestRun.TestJobRunner:ExecuteCallback () (at Library/PackageCache/com.unity.test-framework@2.0.1-pre.18/UnityEditor.TestRunner/TestRun/TestJobRunner.cs:109)
UnityEditor.EditorApplication:Internal_CallUpdateFunctions ()

(Filename: Library/PackageCache/com.unity.test-framework@2.0.1-pre.18/UnityEditor.TestRunner/Api/CallbacksDelegator.cs Line: 64)

Too many instant steps in test execution mode: Error. Current task ConditionalTask.
UnityEngine.StackTraceUtility:ExtractStackTrace ()
UnityEngine.DebugLogHandler:LogFormat (UnityEngine.LogType,UnityEngine.Object,string,object[ ])
UnityEngine.Logger:Log (UnityEngine.LogType,object)
UnityEngine.Debug:LogError (object)
UnityEditor.TestTools.TestRunner.TestRun.TestJobRunner:ExecuteCallback () (at Library/PackageCache/com.unity.test-framework@2.0.1-pre.18/UnityEditor.TestRunner/TestRun/TestJobRunner.cs:120)
UnityEditor.EditorApplication:Internal_CallUpdateFunctions ()

(Filename: Library/PackageCache/com.unity.test-framework@2.0.1-pre.18/UnityEditor.TestRunner/TestRun/TestJobRunner.cs Line: 120)

Test run completed. Exiting with code 3 (RunError). An unexpected error happened while running tests.
UnityEngine.StackTraceUtility:ExtractStackTrace ()
UnityEngine.DebugLogHandler:LogFormat (UnityEngine.LogType,UnityEngine.Object,string,object[ ])
UnityEngine.Logger:Log (UnityEngine.LogType,object)
UnityEngine.Debug:Log (object)
UnityEditor.TestTools.TestRunner.CommandLineTest.Executer:ExitApplication (UnityEditor.TestTools.TestRunner.CommandLineTest.Executer/ReturnCodes,string) (at Library/PackageCache/com.unity.test-framework@2.0.1-pre.18/UnityEditor.TestRunner/CommandLineTest/Executer.cs:91)
UnityEditor.TestTools.TestRunner.CommandLineTest.Executer:ExitIfRunIsCompleted () (at Library/PackageCache/com.unity.test-framework@2.0.1-pre.18/UnityEditor.TestRunner/CommandLineTest/Executer.cs:84)
UnityEditor.EditorApplication:Internal_CallUpdateFunctions ()

[jasal82]

I cannot even get it to produce the testresults.xml without using a TestRunCallback. I’m sorry I have to say that but the whole testing framework is a complete mess. We wasted days trying to get the unit tests running (which use WaitForEndOfFrame, you can imagine…) and we’re still not there.