We have released v1.4 of UTF for Unity 2023.3 and above. This version includes user experience improvements and bug fixes. The 1.4 version of UTF will be the default for Unity 2023.3 and above.
Many of the changes have been backported from the UTF 2.0 experiment. It is our intention to backport most of the remaining changes from 2.0 over time, including its bug fixes and performance improvements.
What’s New Update in the interface
This version includes a revised Test Runner window and several usability improvements:
Added a third tab to the Test Runner window, for running in a player explicitly. This makes it easier to run a subset of tests, as well as retaining the test results from the latest player run.
Moved the run and action buttons to the bottom of the window, to separate them from the filters.
The stack traces are clickable, and will open the relevant file in the external editor.
When searching, the parents of the matching tests are expanded, to make it easier to see the context of the matching tests.
Other notable features added Ignore tests based on arguments
This version introduces the ParameterizedIgnoreAttribute which allows ignoring tests based on arguments which were passed to the test method of a parameterized test
API updates
In this version, we introduced several API changes:
Added API for saving results
Added API for canceling test runs
Added overloads of LogAssert.Expect which allow users to expect a log message without specifying a severity, which will match logged messages of any severity.
Fixes
This version includes many bug fixes and performance improvements. For a full list of changes and updates in this version, see the Unity Test Framework package changelog.
Known Issues
Using OneTimeSetup and OneTimeTeardown with async methods will make the call hang.
If you encounter unexpected issues using UTF 1.4, please file a bug from the bug reporter in the editor.
Installation steps for versions 2023.2 and older
In older versions, the default version of UTF is 1.1.33, however you can still upgrade to 1.4 version by going to the package manager and upgrading it manually, by selecting the Test Framework package and installing the desired version.
Alternatively you can manually edit your /Packages/manifest.json file and add “com.unity.test-framework”: “1.4.2” to the dependencies.
Installing this worked! That’s great, we’ve been stuck on 2.0.1-pre.18 for ages, as any switch to a 1.x version would cause our project to not compile due to internal compiler errors until we switched back to 2.0.1-pre.18 and deleted the library.
Is there any entries for testing on the Roadmap page? I’m still waiting for any work put into making tests that needs to run in a specific scene not be incredibly painful.
The refactoring and improvements of the TestRunnerApi have not yet been backported from 2.0. We do not yet know when that will happen. There is indeed no way of retrieving a test tree with a specified filter.
hey @Warnecke , thanks for the constant feedback here.
I’m using the version “1.4.3”, and running async Task methods on Editor Mode will hang most of the time. I have to constantly move the mouse and click on any view of the Unity Editor to get the Tests to get finished.
Do you know if that is a known issue and if the team is working on a solution for it?
Okay, I’ve made a simple utility to solve that issue until it gets solved on the Unity side.
I basically added a logic to trigger an infinity loop to execute EditorApplication.QueuePlayerLoopUpdate() on every tick whenever the TestRunner starts to run Editor mode tests and then break the loop when the TestRunner tests get finished. Script follows below for anyone experiencing the same issue:
using System.Threading;
using System.Threading.Tasks;
using UnityEditor;
using UnityEditor.TestTools.TestRunner.Api;
using UnityEngine;
namespace TestRunnerUtils
{
[InitializeOnLoad]
public static class TestRunnerHangingOnEditorTestsFixer
{
private static TestRunnerApi api;
private static CancellationTokenSource cancellationTokenSource;
static TestRunnerHangingOnEditorTestsFixer()
{
if (Application.isBatchMode)
{
// This script should not be used in batch mode.
return;
}
SetupListeners();
}
private static void SetupListeners()
{
Object.DestroyImmediate(api);
api = ScriptableObject.CreateInstance<TestRunnerApi>();
api.RegisterCallbacks(new TestRunnerCallbacks());
}
private class TestRunnerCallbacks : ICallbacks
{
public void RunStarted(ITestAdaptor testsToRun)
{
Debug.Log($"TestRunner in {testsToRun.TestMode} mode started");
if (testsToRun.TestMode != TestMode.EditMode)
{
return;
}
cancellationTokenSource?.Cancel();
cancellationTokenSource = new CancellationTokenSource();
TriggerEditorUpdateLoop(cancellationTokenSource.Token);
}
public void RunFinished(ITestResultAdaptor result)
{
Debug.Log("TestRunner finished");
cancellationTokenSource?.Cancel();
}
public void TestStarted(ITestAdaptor test)
{
// Don't need to do anything here.
}
public void TestFinished(ITestResultAdaptor result)
{
// Don't need to do anything here.
}
private static void TriggerEditorUpdateLoop(CancellationToken cancellationToken = default)
{
TriggerEditorUpdateLoopAsync(cancellationToken).ConfigureAwait(false);
}
private static async Task TriggerEditorUpdateLoopAsync(CancellationToken cancellationToken = default)
{
while (cancellationToken.IsCancellationRequested == false)
{
await Task.Yield();
EditorApplication.QueuePlayerLoopUpdate();
}
}
}
}
}
@Warnecke One really important feedback I would like mention is please dun backport PlayerHeartBeatTask at 2.0 which is regression that will fail my long running test. I have test needs to take a lot of time to run.
Do async tests work with [UnityTest]? Or now that we don’t need to use IEnumerator, can we remove [UnityTest] and put [Test] instead? Because in the docs it doesn’t appear nor is it mentioned anywhere:
async Task works only with [Test].
and there is no need to remove [UnityTest] because otherwise everyone would have to refactor their code to change IEnumerator to async Task and that is not good.
Also, there is no reason to remove the IEnumerator option only because now we can use async Task.
In my case it is because my tests are async but I was using UniTask.ToCoroutine (before you could run async tests officially) and I want to remove that dependency.
In some of my old projects, I was also using a task.ToCoroutine() extension I created, then I started to convert them to use async Task methods with [Test], and it turned out to be an easy refactoring.
What would be the issue with [UnitTest], in your case, when you start changing your UniTask.ToCoroutine to async Task methods?
There is no mention of OneTime/Setup OneTime/TearDown attributes or the known issues of the first post of this thread:
What is the status? It’s fixed on any 1.4.x package version (In the changelog there is no mention of it)? Can these attributes be used with “async Task”, or not @Warnecke ? Thanks.
Also, if I try to use the [Retry] attribute (I’m using also Test, UnityPlatform and Timeout attributes for what it’s worth) with async Task I get the following error:
