Async await in Unittests

[ISeidingerReflekt]

I am struggling with writing unittests for async methods since the unittest itself can not be async. I am trying to work around this by using task.Wait() but this seems to result in a deadlock in certain situations.
See the following code:

        [Test]
        public void TestAwait()
        {
            var task = GetTestTask();
            task.Wait();
            Assert.AreEqual(1, 1);
        }

        private async Task GetTestTask()
        {
            UnityEngine.Debug.Log("1");
            await Task.Delay(TimeSpan.FromMilliseconds(200));
            UnityEngine.Debug.Log("2");
            await Task.Run(async () => await Task.Delay(TimeSpan.FromSeconds(1)));
            UnityEngine.Debug.Log("3");
            await Task.Delay(TimeSpan.FromMilliseconds(200));
            UnityEngine.Debug.Log("4");
        }

I guess that some of the code in GetTestTask will run on a different thread and try to sync up with the main thread while the mainthread is blocked because of task.Wait().
How can i work around this issue, and is this a bug or a feature?
Any help appreciated!

[GoodGoodMan]

I have same problem.

I also tried the following way.

[Test]
public void AwaitTest()
{
Task<bool> task = GetTask(); // Task will be fired from GetTask method.
while(!task.IsCompleted) {}
Assert.IsTrue(task.Result);
}

But it doesn’t work.

[tsibiski]

Alternatively, you could use the Trilleon Automation Framework that supports both Unit Testing and Integration Automation tests. It uses Coroutines to simulate asynchronocity, but it does not use multi threading or actual async (with a socket connection for server communication being the exception to that). But given your sample, it doesn’t look as if you need real async. You just need something that can wait for work to be done before continuing the test execution. (Trilleon uses “WaitFor” which passes in a predicate that can express any boolean state, such as “is this work done yet?”).

You can try it out from here. GitHub - disruptorbeam/trilleon: Automate all the things..

It is an entire framework, and will take care of this problem for you. However, your Unit Tests written in Trilleon will not run on compilation. It is not designed identically to something like NUnit. It however allows you to unit test IEnumerators and anything that requires gameplay context to validate.

[StephenHodgson-Valorem]

I’m surprised that just using public async Task MyAysyncTest() doesn’t work out of the box with NUnit

[xmedeko]

Same problem, Unity GUI hangs and I have to kill the Unity process. It’s a serious bug and should be solved.

Have reported bug: https://fogbugz.unity3d.com/default.asp?1129455_2qh1olgn71f7hr6l

[Mathijs_Bakker]

Unity uses an old Nunit framework. I don’t know which version. But it does not offer proper async Task support.
You can use a workaround that’s inefficient but works: Execute the async test logic on a different thread pool thread, and then (synchronously) block the unit test method until the actual test completes.

The example test below will fail nicely:

[Test]
public void TestAwait()
{
    Task.Run(async () =>
    {
        await GetTestTaskAsync();
    }).GetAwaiter().GetResult();
    
    Assert.AreEqual(1, 2);
}

public async Task GetTestTaskAsync()
{
    Debug.Log("1");
    await Task.Delay(TimeSpan.FromMilliseconds(200));
    Debug.Log("2");
    await Task.Run(async () => await Task.Delay(TimeSpan.FromSeconds(1)));
    Debug.Log("3");
    await Task.Delay(TimeSpan.FromMilliseconds(200));
    Debug.Log("4");
}

TestAwait (1.427s)

Message: 
Expected: 2
But was: 1

Output: 
1
2
3
4

[xmedeko]

@Mathijs_Bakker yes it works! Thanks.

A bit simplified it is: Task.Run(() => GetTestTaskAsync()).GetAwaiter().GetResult();

[pld]

Here is the pattern I use; it avoids deadlock if the code you’re testing needs to wait for work that’s posted to the Unity SynchronizationContext.

  [UnityTest]
  public IEnumerator TestSomeAsyncThing() {
    var task = Task.Run(async () => {
      /* non-boilerplate code goes here */
    });
    while (!task.IsCompleted) { yield return null; }
    if (task.IsFaulted) { throw task.Exception; }
  }

Edit: to propagate exceptions

[JakHussain]

@pld

Here is the pattern I use; it avoids deadlock if the code you’re testing needs to wait for work that’s posted to the Unity SynchronizationContext.

  [UnityTest]
  public IEnumerator TestSomeAsyncThing() {
    var task = Task.Run(async () => {
      /* non-boilerplate code goes here */
    });
    while (!task.IsCompleted) { yield return null; }
    if (task.IsFaulted) { throw task.Exception; }
  }

Edit: to propagate exceptions

I see your very useful boiler plate and raise you a static method:

public static IEnumerator Execute (Task task)
{
    while (!task.IsCompleted) { yield return null; }
    if (task.IsFaulted) { throw task.Exception; }
}

Used like this:

[UnityTest]
public IEnumerator AddReceiver ()
{
    yield return AsyncTest.Execute (manager.AddReceiverAsync (workingCredentials.streamID));

    Assert.True (manager.ReceiverCount == 1);
}

[jBardoe]

@Mathijs_Bakker works like a charm!
And if you need to get a result out of a task, it works in the similar way:
(You can change UnityTest to just Test if your NUnit version requires it to be like that)

[UnityTest]
public void TestAwait()
{
    var task = Task.Run(async () =>
    {
        return await GetTestTaskAsync();
    });
  
    Assert.AreEqual(1, task.Result);
}

public async Task<int> GetTestTaskAsync()
{
    await Task.Delay(TimeSpan.FromMilliseconds(200));
    return 1;
}

[Mathijs_Bakker]

@jBardoe

@Mathijs_Bakker works like a charm!

Good!

[AlexStark]

Based on @Mathijs_Bakker I write some generic helper methods

public static class UnityTestUtils {

        public static T RunAsyncMethodSync<T>(Func<Task<T>> asyncFunc) {
            return Task.Run(async () => await asyncFunc()).GetAwaiter().GetResult();
        }
        public static void RunAsyncMethodSync(Func<Task> asyncFunc) {
            Task.Run(async () => await asyncFunc()).GetAwaiter().GetResult();
        }
}

Usage:

        [Test]
        public void Test()
        {
            var result = RunAsyncMethodSync(() => GetTestTaskAsync(4));
            Assert.That(result, Is.EqualTo(4));
        }

        public async Task<int> GetTestTaskAsync(int a) {
            await Task.Delay(TimeSpan.FromMilliseconds(200));
            return a;
        }

        [Test]
        public void Testthrow() {
            Assert.Throws<InvalidOperationException>(
                           ()=> RunAsyncMethodSync(() => ThrowTaskAsync(4)));
        }

        public async Task<int> ThrowTaskAsync(int a) {
            await Task.Delay(TimeSpan.FromMilliseconds(200));
            throw new InvalidOperationException();
        }

[Rib]

Here’s another variation that I arrived at…

I have these static Await utilities, similar to @ and @JakHussain …

        public static IEnumerator Await(Task task)
        {
            while (!task.IsCompleted) { yield return null; }
            if (task.IsFaulted) { throw task.Exception; }
        }
        public static IEnumerator Await(Func<Task> taskDelegate)
        {
            return Await(taskDelegate.Invoke());
        }

Which can be used to ‘await’ for an async delegate or an async Task within a coroutine like…

        [UnityTest]
        public IEnumerator TestSomeAsyncThing() {
            yield return Await(async () => {
                await UniTask.Delay(TimeSpan.FromSeconds(5)); // example async thing
            });
        }
        async Task AsyncTestMethod()
        {
            await UniTask.Delay(TimeSpan.FromSeconds(5)); // example async thing
        }
        [UnityTest]
        public IEnumerator TestSomeOtherAsyncThing() {
            yield return Await(AsyncTestMethod());
        }

The main reasons I went with this approach:

1. Calling GetResult() or reading Result, as above, blocks the main thread which is the thread most of the Unity code I want to test is designed to run from.
2. It avoids using Task.Run() (which will run code via a thread pool), again because most of the Unity code I’m looking to test is designed to run on the main thread (or will handle spawning work on threads itself where appropriate)
3. The static Await() utilities feel similar to using the await keyword but within a coroutine
4. It supports the convenience of using an async lambda for writing tests

[ModLunar]

Check out this potential solution, although I (as commented in that thread) couldn’t get it to work even with the fork of Unity Test Framework…?

Support for async / await in tests

Async / await is supposed to be supported there.
You can try by adding their package using the Git URL:
GitHub - 8bitforest/com.unity.test-framework: [Mirrored from UPM, without any changes. Maintained by Needle. Not affiliated with Unity Technologies.] 📦 Test framework for running Edit mode and Play mode tests in Unity.

[thewerku]

Hi. Team owning the Unity Test Framework package (UTF, also known as Test Runner) has recently made available a pre-release version of v2.0 - you can read the announcement here . One of the new features we added, and are looking to get users’ eyes on it, are async tests. We’d appreciate any feedback and comments, if you decide to test it out. Thanks!

[ModLunar]

@thewerku

Hi. Team owning the Unity Test Framework package (UTF, also known as Test Runner) has recently made available a pre-release version of v2.0 - you can read the announcement here . One of the new features we added, and are looking to get users’ eyes on it, are async tests. We’d appreciate any feedback and comments, if you decide to test it out. Thanks!

Thanks so much for sharing!
I’ll take a look at this soon, for feedback, would replying to this thread, or somewhere else be preferred?

[thewerku]

Thanks! It would be great if you could post in the main thread where we made the announcement . That’s where a lot of discussion is happening and we’re actively monitoring it with the team.