Add coroutine version of OneTimeSetUp

Unity Engine
,
1

You already provide the UnityTest and UnitySetUp/Teardown attributes which work with coroutines. It would be nice to also have a way to run a one-time setup (once per test class) that works with coroutines, i.e. this would be a good place to load a test scene, then have a number of test methods do different tests in the same scene.

11 Likes
2

Isn’t it possible to achieve this by loading the scene in the TestFixture’s constructor? (it doesn’t have to be a coroutine).

3

Probably. The unloading only works asynchronously though. The synchronous method is deprecated.

And there might be other use cases for a UnityOneTimeSetUp attribute.

2 Likes
4

Precisely, I was searching this option for a bunch of tests that have to be checked when the complete flow has been finished

5

I’m also trying to wrap my head around this at the moment. How is it supposed to work to

  • Load a scene or two until they have finished loading
  • Run a test with those scenes
  • Run another test with those same scenes
  • Finish, unload etc.

Using [UnitySetUp] I can use yield to wait until the scene is loaded which is actually necessary for both synchronous and LoadSceneAsync. But then it will be run before every test.
Using a TestFixtureSource as I understand it would not even allow loading the scene synchronously because it won’t be loaded before the next frame, it is semi-asnychronuous (Unity - Scripting API: SceneManagement.SceneManager.LoadScene). It seems crucial to me to have [UnityOneTimeSetUp] where all scene setup can be done for a test suite.
A workaround I have now found is to simply check if the scenes have loaded already in the SetUp call. Not very clean though.
Also I haven’t figured out how to determine that all tests in the class have run and I can unload all scenes and GameObjects from DontDestroyOnLoad in [TearDown] since it will be necessary if I wan’t to have another test suite that should start fresh.

1 Like
6

I’m wondering if a more complete framework for loading and unloading test scenes would be beneficial, or if everyone’s requirements are too different to find a good foundation. I imagine setting up separate test scenes where you want to run isolated tests would be a quite common use case.

7

In my opinion that’s the way to go. The Test Framework has improved so much, but there are still some flaws.

Loading test scenes for me is one of them. If you want to load a test scene you have to add it to BuildSettings and then remove it.
Maybe something like TestSceneManager.LoadAsync

1 Like
8

Yes, please can the Unity development team look into this. I’m trying to figure out what the ‘best practice’ way of loading scenes for testing. Ideally I’d like to do this once, but it’s not clear how this can be done with an async OneTimeSetup method.

1 Like
9

I just needed the same thing, and while Unity adds a proper way, I hacked it in.
To make it work, just replace the two attached files inside the Library\PackageCache\com.unity.test-framework@1.1.14 folder:

The relevant parts are:

Defining the attributes in UnitySetupAttribute.cs

[AttributeUsage(AttributeTargets.Method, AllowMultiple = false, Inherited = true)]
public class UnityOneTimeSetUpAttribute : NUnitAttribute
{
}

[AttributeUsage(AttributeTargets.Method, AllowMultiple = false, Inherited = true)]
public class UnityOneTimeTearDownAttribute : NUnitAttribute
{
}

Variables to hold the UnityOneSetup and UnityOneTimeTearDown in CompositeWorkITem.cs (line 24)

MethodInfo _unityOneTimeSetup, _unityOneTimeTearDown;

Capturing the methods (line 167)

if (_suite.TypeInfo != null)
{
    _unityOneTimeSetup = Reflect.GetMethodsWithAttribute(_suite.TypeInfo.Type, typeof(UnityOneTimeSetUpAttribute), true)
                                .FirstOrDefault(x => x.ReturnType == typeof(IEnumerator));

    _unityOneTimeTearDown = Reflect.GetMethodsWithAttribute(_suite.TypeInfo.Type, typeof(UnityOneTimeTearDownAttribute), true)
                                   .FirstOrDefault(x => x.ReturnType == typeof(IEnumerator));
}

Running them before and after the tests (line 72)

PerformOneTimeSetUp();
if (_unityOneTimeSetup!=null)
{
    yield return Reflect.InvokeMethod(_unityOneTimeSetup, Context.TestObject);
}

line (103)

if (_unityOneTimeTearDown!=null)
{
    yield return Reflect.InvokeMethod(_unityOneTimeTearDown, Context.TestObject);
}

PerformOneTimeTearDown();

Its a hack, but should work for most cases.

BTW, to avoid having Unity override the changes everytime unity starts, just copy the package from the library folder to the Packages folder.

Hope that helps

6055715–655325–CompositeWorkItem.cs (13.8 KB)
6055715–655328–UnitySetUpAttribute.cs (555 Bytes)

5 Likes
10

Thanks for this Inter-Illusion. I’ve added your changes to the git repository https://github.com/DanStevens/com.unity.test-framework.git. To use with Unity, all one has to do is go to Window > Package Manager, click the ‘+’, select Add package from git URL and enter the repository URL above.

3 Likes
11

Is it working on 2019.4?
It compiles, but doesnt work for me. This code doesnt seem to execute anything:

        [UnityOneTimeSetUp]
        public void OneTimeSetup () {
            Debug.Log("UnityeOneTimeSetUp");
        }

@superpig
We really need this as default in the Test Tools.
Here the use case: client and servers. The server start is a bit slow to do every [UnitySetup], and it cant be done via [OneTimeSetup] because many of it’s dependencies are initialized on Start(), so it needs to wait a yield return null to work.
Also, clients need to reconnect for every test, and the server also needs to wait for those connections every single test. This is too slow and a pain to synchronize. A [UnityOneTimeSetUp] solves this.

12

I have not been able to get @Inter-Illusion 's solution to work with Unity’s Test Framework 1.1.14 or 1.1.18. Please bear in mind that I am a Unity and C# noob. I put UnitySetupAttributes.cs in \Library\PackageCache\com.unity.test-framework@1.1.1x\UnityEngine.TestRunner\NUnitExtensions\Attributes and CompositeWorkItem.cs in \Library\PackageCache\com.unity.test-framework@1.1.1x\UnityEngine.TestRunner\NUnitExtensions\Runner. When I use UnityOneTimeSetup and UnityOneTimeTearDown, I get compilation errors. I want to use the official Unity test framework, not danstevens. Can someone help me figure out why this isn’t working for me?

Thanks!

13

I, too, would like this feature. It would greatly simplify my test suite.

My test suite consists of a bunch of scenarios, followed by a bunch of Asserts to ensure the world is in my desired state. I could run my scenario to completion in a hypothetical [UnityOneTimeSetup], then run the actual Asserts in some trivial test methods.

14

Would also really love to see this. I completely agree with @akuno . Here’s a concrete example of what he described that I would really love to be able to handle out of the box.

    /// <summary>
    /// Contains tests for our <see cref="PlayFabClientAPI"/>
    /// </summary>
    public class PlayFabClientAPITests
    {
        [OneTimeSetUp]
        public IEnumerator SetupTest()
        {
            var playFabAuthorizationService = new PlayFabAuthorizationService();
            var unityTestUser = new PlayFabAuthorizationService.AuthorizedUser("UnitTester","UnitTester");
            var signInTask = playFabAuthorizationService.SignInAsync(unityTestUser);
            yield return signInTask.AsIEnumerator();
        }
     
        [UnityTest]
        [TestCase(7496062027744821083,"Betty", ExpectedResult = null)]
        [TestCase(1312412027744821083,"Todd", ExpectedResult = null)]
        [TestCase(5326062027744124354,"Miami", ExpectedResult = null)]
        public IEnumerator GetCatalogItem_LevelDataId_PlayFabCatalogItemFound(long levelDataId, string levelName)
        {
            //Get CatalogItem from server & verify not null...requires a sign in before PlayFabClientAPI.GetCatalogItems can be used.
        }
1 Like
15

I can’t believe this hasn’t been added yet. I need this for my project as well. Surely it can’t be that hard for Unity to add this to the official package.

EDIT: I have created my own fork here: GitHub - 8bitforest/com.unity.test-framework: [Mirrored from UPM, without any changes. Maintained by Needle. Not affiliated with Unity Technologies.] 📦 Test framework for running Edit mode and Play mode tests in Unity. I have confirmed this works on 2020.2.1f1, and is based off the latest version of the official package. ( @akuno @pwdstaraster a little late, but might help you guys)

EDIT 2: I also added [AsyncTest] [AsyncSetUp/TearDown] and [AsyncOneTimeSetUp/TearDown] attributes that can be added to methods returning “async Task”!

6 Likes
16

I also need this feature. Is anybody from Unity team aware of this feature request?

17

I don’t think so. It would be good to know if this feature is at least in their roadmap

18

+1

19

+1

20

Should we keep adding +1 in this thread until Unity sees it?