In one of our projects, we can’t run our (edit mode) tests in batch mode on our CI (Jenkins) server, because for some reason Unity decides that before running the test, the current (Untitled) scene needs to be saved. But in batch mode that results in cancelling the tests altogether.
Part of the log file:
[Project] Loading completed in 335.912 seconds
Project init time: 335.785 seconds
Services packages init time: 0.000 seconds
Package Manager init time: 0.998 seconds
Asset Database init time: 7.336 seconds
Global illumination init time: 0.269 seconds
Assemblies load time: 1.975 seconds
Unity extensions init time: 0.001 seconds
Asset Database refresh time: 205.563 seconds
Scene opening time: 0.077 seconds
GraniteSDK > Be sure to place a Granite Manager in your scene. The editor automatically registered a GraniteManager but this will not happen in-game.
UnityEngine.DebugLogHandler:Internal_Log(LogType, LogOption, String, Object)
UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[ ])
UnityEngine.Logger:Log(LogType, Object)
UnityEngine.Debug:LogWarning(Object)
GraniteSDK.EditorUpdate:Initialize()
GraniteSDK.EditorUpdate:UpdateEditorCamera()
GraniteSDK.EditorFunctions:OnEditorUpdate()
UnityEditor.EditorApplication:Internal_CallUpdateFunctions() (at C:\buildslave\unity\build\Editor\Mono\EditorApplication.cs:200)
(Filename: C:/buildslave/unity/build/Editor/Mono/EditorApplication.cs Line: 200)
Hashing assets (1 files)… 0.000 seconds
file read: 0.000 seconds (0.005 MB)
wait for write: 0.000 seconds (I/O thread blocked by consumer, aka CPU bound)
wait for read: 0.000 seconds (CPUT thread waiting for I/O thread, aka disk bound)
hash: 0.000 seconds
Connect to CacheServer 10.10.3.14:8126
Disconnect from CacheServer
Refreshing native plugins compatible for Editor in 0.61 ms, found 4 plugins.
Preloading 0 native plugins for Editor in 0.00 ms.
Running tests for EditMode
Executing tests with settings: test mode = EditMode
UnityEngine.DebugLogHandler:Internal_Log(LogType, LogOption, String, Object)
UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[ ])
UnityEngine.Logger:Log(LogType, Object)
UnityEngine.Debug:Log(Object)
UnityEditor.TestTools.TestRunner.CommandLineTest.Executer:InitializeAndExecuteRun(String[ ]) (at C:\buildslave\unity\build\Extensions\TestRunner\UnityEditor.TestRunner\CommandLineTest\Executer.cs:42)
UnityEditor.TestTools.TestRunner.CommandLineTest.TestStarter:UpdateWatch() (at C:\buildslave\unity\build\Extensions\TestRunner\UnityEditor.TestRunner\CommandLineTest\TestStarter.cs:48)
UnityEditor.EditorApplication:Internal_CallUpdateFunctions() (at C:\buildslave\unity\build\Editor\Mono\EditorApplication.cs:200)
(Filename: C:/buildslave/unity/build/Extensions/TestRunner/UnityEditor.TestRunner/CommandLineTest/Executer.cs Line: 42)
Cancelling DisplayDialogComplex: Scene(s) Have Been Modified Do you want to save the changes you made in the scenes:
Untitled
Your changes will be lost if you don’t save them.
This should not be called in batch mode.
UnityEditor.SceneManagement.EditorSceneManager:SaveCurrentModifiedScenesIfUserWantsTo()
UnityEditor.TestTools.TestRunner.EditModeLauncher:Run() (at C:\buildslave\unity\build\Extensions\TestRunner\UnityEditor.TestRunner\TestLaunchers\EditModeLauncher.cs:28)
UnityEditor.TestTools.TestRunner.Api.TestRunnerApi:Execute(ExecutionSettings) (at C:\buildslave\unity\build\Extensions\TestRunner\UnityEditor.TestRunner\Api\TestRunnerApi.cs:25)
UnityEditor.TestTools.TestRunner.CommandLineTest.Executer:InitializeAndExecuteRun(String[ ]) (at C:\buildslave\unity\build\Extensions\TestRunner\UnityEditor.TestRunner\CommandLineTest\Executer.cs:43)
UnityEditor.TestTools.TestRunner.CommandLineTest.TestStarter:UpdateWatch() (at C:\buildslave\unity\build\Extensions\TestRunner\UnityEditor.TestRunner\CommandLineTest\TestStarter.cs:48)
UnityEditor.EditorApplication:Internal_CallUpdateFunctions() (at C:\buildslave\unity\build\Editor\Mono\EditorApplication.cs:200)
[C:\buildslave\unity\build\Editor/Platform/Windows/EditorUtility.cpp line 1255]
(Filename: C:/buildslave/unity/build/Extensions/TestRunner/UnityEditor.TestRunner/TestLaunchers/EditModeLauncher.cs Line: 28)
It’s not entirely clear to me what causes the Untitled scene (which is empty, not loaded from disk) to have changed but that may be due to the Granite plugin we use.
The dialog appears when I run Edit Mode tests in the editor too, and cancelling it causes the tests not being run there as well. Only choosing “don’t save” will actually start the tests.
How can I work around this issue? Or is this a bug that can be fixed in the test framework? In batch mode, the default choice in this case should be “Don’t Save” instead of “Cancel”, I think.