Does the Unity test framework support running test methods that are marked as ‘async’ (e.g: using the await keyword) ?
This test is shown in the test runner (but doesn’t properly work since the test exits without waiting for the task to complete):
[Test]
public async void ThisTestSucceeds()
{
await Task.Delay(5500);
Assert.IsTrue(false);
}
Currently that is not supported. It might be possible when we upgrade the nunit version, but we do not have a timeframe for that yet.
@Warnecke could you satisfy my curiosity and inform me why is that not a priority? Not being able to unit test async/await paths really limits the scope of what you can test, specially with recent libs.
workaround:
public static IEnumerator AsCoroutine (this Task task)
{
while (!task.IsCompleted) yield return null;
// if task is faulted, throws the exception
task.GetAwaiter ().GetResult ();
}
then use IEnumerator-based tests
[UnityTest] public IEnumerator Test() {
yield return Run().AsCoroutine();
async Task Run() {
actual test code...
}
}
Basically the same as M_R suggested, but I am using UniRX.Async where you can go
[UnityTest]
public IEnumerator AwaitablePlayModeTest()
{
yield return Task().ToCoroutine();
async UniTask Task()
{
// test with async/await
}
}
I think, you need to wait a frame using UniTask.Yield() at the beginning to have the task scheduled at the right time in the Unity game loop. Not sure.
This is a bit more compact than C# Task and more performant, because UniTask is optimized for Unity.
I vote for integrating async/await everywhere in Unity natively! 
@Warnecke is there an update on this? Async tests important to me as well.
Async Await Support has extension methods that convert a task to an IEnumerator. Here’s an example:
[UnityTest]
public IEnumerator LoadTerrain()
{
yield return _().AsIEnumerator();
async Task _()
{
Task loadingOperation = fixture.LoadTerrainAsync();
Assert.AreEqual(GameState.TerrainLoading, fixture.State);
await loadingOperation;
Assert.AreEqual(GameState.TerrainLoaded, fixture.State);
Assert.That(SceneManager.GetActiveScene().path,
Does.EndWith($"{ScenePath}.unity"));
}
}
And the LoadTerrainAsync method (which essentially loads a scene asynchronously):
public async Task LoadTerrainAsync()
{
state = GameState.TerrainLoading;
Debug.Log($"Loading scene '{scenePath}')");
await SceneManager.LoadSceneAsync(scenePath);
state = GameState.TerrainLoaded;
}
One advantage of running a test in this way is if LoadTerrainAsync method throws an exception, it gets passed up to the test runner as an AgregateException. If the method was a Coroutine, the exception would get swalled and all I know is that the state didn’t change as expected but no indicator as to why. Since it’s also possible to await methods that return IEnumerator (thanks to extensions in the AsyncAwaitUtil plugin), I think it’s in principle possible to only make use of async/await in test code but not use it the production code (should you want that).
The coroutine workaround will only work for playmode tests as [UnityTest] is not available for editor tests, I think.
[UnityTest] attribute is available for EditMode and PlayMode tests, although with some limitations in the former.
It sounds like Unity isn’t going to add these any time soon so I’ve added them myself in my fork of the test framework: GitHub - 8bitforest/com.unity.test-framework: [Mirrored from UPM, without any changes. Maintained by Needle. Not affiliated with Unity Technologies.] 📦 Test framework for running Edit mode and Play mode tests in Unity.
It adds [AsyncTest], [AsyncSetUp], [AsyncTearDown], [AsyncOneTimeSetUp], and [AsyncOneTimeTearDown]
The methods with these attributes will need to return “async Task”. “async void” will not work:
[AsyncTest]
public async Task TestWithDelay()
{
Debug.Log("Starting test...");
await Task.Delay(5000);
Debug.Log("Test finished!");
}
They are implemented using IEnumerator’s in the backend just like the answers above, but this way improves the syntax so they are regular async methods, and should be easy to switch over when/if Unity implements them themselves.
And as a bonus my fork also includes the mysteriously missing [UnityOneTimeSetUp] and [UnityOneTimeTearDown]
TestTask:44
m_Context.CurrentResult.RecordException(m_TestTask.Exception!.InnerException); => shouldn’t be ? instead of !
Creates a compilation error. Or is it a c#8 trick ?
But then it actually works, very greatful.
One issue I’ve noticed is that tests doesn’t fails when consol has errors. Not sure because wasn’t able to reproduce it clearly
Another feedback, if the AsyncSetup fails with an exception then it’s considered as “Passed” silently.
I have the case while awaiting a task wrapping a RecompileScript that fails
The UniTask library is also a way to handle unit testing async:
Yeah I fallback into that one, it’s working.
Oh heavens, thank you!!!
I was going to use Coroutines, but such tests have to 100% completely wait for each one to individually run… which will just make my tests take forever and freeze the editor unnecessarily.
<3 async / await
Wait… your code example (method called TestWithDelay()) isn’t working in my edit mode tests.
And no shame, here’s my quick and dirty testing code (as used in the screenshot above)
using System.Collections;
using System.Threading.Tasks;
using NUnit.Framework;
using UnityEngine;
using UnityEngine.TestTools;
using UnityEditor;
namespace XXX.Editor.Tests {
public class XXXTests {
[UnityTest]
public IEnumerator ExampleCoroutine() {
float startTime = (float) EditorApplication.timeSinceStartup;
while (EditorApplication.timeSinceStartup - startTime < 1)
yield return null;
Debug.Log("!?");
startTime = (float) EditorApplication.timeSinceStartup;
while (EditorApplication.timeSinceStartup - startTime < 1)
yield return null;
startTime = (float) EditorApplication.timeSinceStartup;
while (EditorApplication.timeSinceStartup - startTime < 1)
yield return null;
}
[UnityTest]
public IEnumerator ExampleCoroutine2() {
float startTime = (float) EditorApplication.timeSinceStartup;
while (EditorApplication.timeSinceStartup - startTime < 1)
yield return null;
Debug.Log("!?");
startTime = (float) EditorApplication.timeSinceStartup;
while (EditorApplication.timeSinceStartup - startTime < 1)
yield return null;
startTime = (float) EditorApplication.timeSinceStartup;
while (EditorApplication.timeSinceStartup - startTime < 1)
yield return null;
}
[AsyncTest]
public async Task TestWithDelay() {
Debug.Log("Starting test...");
await Task.Delay(5000);
Debug.Log("Test finished!");
}
[AsyncTest]
public async Task Example() {
await Task.Delay(1000);
Debug.Log("????");
await Task.Delay(1000);
await Task.Delay(1000);
await Task.CompletedTask;
Assert.Pass("!?");
}
[AsyncTest]
public async Task NUnit1012SampleTest() {
bool result = await Task.FromResult(true);
Assert.That(result, Is.True);
}
[AsyncTest]
public async Task<int> TestAdd() {
return await Task.FromResult(2 + 2);
}
}
}
Note the 2 coroutine examples are the same – I just wanted to verify that they couldn’t run “at the same time”.
Did you too, convert the Tasks into a “coroutine representation”, since it’s telling me edit mode tests can only yield return null?
I was thinking that you updated the NUnit Framework version to one that supports async Task… but I appear to be incorrect.
I think you’ve misunderstood what async/await support in NUnit is intended for. It has nothing to do with parallel test execution, which is a completely different feature: https://docs.nunit.org/articles/nunit/technical-notes/usage/Framework-Parallel-Test-Execution.html
However, that feature uses separate threads, and if you are doing anything that works with Unity APIs, it probably wouldn’t work for you anyway, as they mostly require you to work on the Unity main thread. If you are not working with any Unity APIs, and want to put in the extra effort, you could have your code and tests live in standard .NET assemblies, and leverage all the NUnit 3 features available from a separate non-Unity project.
Ah…
Right, I forgot about the implications of running on the Unity main thread, since these tests usually interact with UnityEngine/related API.
I guess I see why they didn’t prioritize it… but it’s still a stretch to say I still wouldn’t want to test async (non-parallel) code in my Unity tests.
I suppose if I wanna test it though, it’ll need to be pure .NET/C# projects then.
Thanks for explaining!
Oh what I was thinking of was both async tests,
and what I found today:
[Parallelizable] marks unit tests in NUnit as ones that can run at the same time as others.
Since their docs note it requires NUnit 3.0, I assume it won’t work with Unity Test Framework / Unity Test Runner though.
@ModLunar TestFramework 1.1.24 is using NUnit 3.5 :
From : About Unity Test Framework | Test Framework | 1.1.33
Also, my problem is that for whatever reason, my test is taking something like 20 second to execute where in play mode, it’s almost instant.
I’m using the IEnumerator work around :
Task task = sut.Load();
while (!task.IsCompleted)
yield return null;
if (task.IsFaulted)
throw task.Exception;
It’s so crazy that UTF is not supporting the async task test like NUnit does.