Confusion between unit tests, EditMode tests and PlayMode tests

I have a couple of questions I really need some help with.

  1. A coworker who's basically in charge of overviewing unit tests told me that EditMode tests are unit tests and PlayMode tests are (exclusively) integration tests. Is this correct?

  2. He wants the EditMode tests to have 100% coverage. Is this realistic? Up until recently this also included testing private methods using reflection, but he dropped that idea after some debating.

  3. Recently it was decided that instead of testing lines of code, we want to test behaviours instead. The idea is: "I don't care if this method calls a method named DoSomething or DoSomethingElse. What I care about is the result. If tomorrow we rename DoSomething to ExecSomething, why should the test fail when the behaviour is still the same?"

But I find it problematic to test a behaviour in a unit test (EditMode) when the behaviour/result is seen in another class. For example, consider the method LoginManager.EnableLogin:

public class LoginManager : MonoBehaviour
{
    private LoginView login;
    ...
    public void EnableLogin()
    {
        login.EnablePanel();
    }
    ...
}

public class LoginView : MonoBehaviour
{
    protected new GameObject gameObject; // this is necessary to allow set/get using reflection in the tests
    ...

    private void Start()
    {
        gameObject = base.gameObject;
        ...
    }

    public void EnablePanel()
    {
        gameObject.SetActive(true);
    }
    ...
}

In the test, the behaviour we want to check is that LoginView.gameObject is set active. This is the test:

public class LoginManagerTests
{
    private LoginManager manager;

    private ILoginView loginView;

    [SetUp]
    public void SetUp()
    {
        manager = new LoginManager();

        loginView = Substitute.For<ILoginView>();
        SetValue(manager, "login", loginView);
    }

    [Test]
    public void EnableLogin_ShouldEnableLoginPanel()
    {
        //Arrange
        GameObject loginGameObject = new GameObject();
        loginGameObject.SetActive(false);
        SetValue(loginView, "gameObject", loginGameObject);

        //Act
        manager.EnableLogin();

        //Assert
        Assert.AreEqual(
            true,
            GetValue<GameObject>(loginView, "gameObject").activeInHierarchy
            );
    }
}

Is this really a proper unit test? It seems a bit dirty...
And then the question arises: Shouldn't this kind of stuff be tested in PlayMode? In the test just load the Login scene, call loginManager.EnableLogin() and check if the panel becomes active. Voilà, behaviour checked. Am I missing something?

[quote=“alexisnavarro24”, post:1, topic: 897828]

  1. A coworker who’s basically in charge of overviewing unit tests told me that EditMode tests are unit tests and PlayMode tests are (exclusively) integration tests. Is this correct?
    [/quote]Not really. I think what your coworker is getting at is that a unit test shouldn’t need to be a playmode test because it shouldn’t be dependent on interacting with engine systems - if the system under test is truly isolated then it should be possible to test the behaviour you want purely in edit mode. However, there’s nothing stopping you from running that same test in play mode, it’s just a bit of a waste of time. It’s also completely possible to write integration tests in edit mode.

[quote]
2. He wants the EditMode tests to have 100% coverage. Is this realistic? Up until recently this also included testing private methods using reflection, but he dropped that idea after some debating.
[/quote]It may be realistic in the sense that ‘it can be done’, but it’s debatable as to whether it’s a good idea.

[quote]
3. Recently it was decided that instead of testing lines of code, we want to test behaviours instead. The idea is: “I don’t care if this method calls a method named DoSomething or DoSomethingElse. What I care about is the result. If tomorrow we rename DoSomething to ExecSomething, why should the test fail when the behaviour is still the same?”

But I find it problematic to test a behaviour in a unit test __(__EditMode) when the behaviour/result is seen in another class. For example, consider the method LoginManager.EnableLogin:
[/quote]Yes, I would not call this example a unit test because it’s testing behaviour across two units (LoginManager and LoginView). If you follow traditional unit test rules, you need two tests: one which tests “when LoginManager.EnableLogin is called, then the EnablePanel method on its LoginView member is called exactly once”, and one which tests “when LoginView.EnablePanel is called, then the SetActive member of its GameObject members called exactly once with parameter ‘true’.”

[quote]
And then the question arises: Shouldn’t this kind of stuff be tested in PlayMode? In the test just load the Login scene, call loginManager.EnableLogin() and check if the panel becomes active. Voilà, behaviour checked. Am I missing something?
[/quote]You certainly could do that, and it brings a different kind of value. For example, that test might fail if the LoginManager/LoginView objects in your Login scene aren’t configured correctly. As a result the scope of possible reasons for the test to fail is broader - it’s no longer only about the code, but about the Scene, and potentially about other things in the Scene as well, etc. There are pros and cons to that: pro, you potentially catch a broader range of issues with your test; con, when your test does fail you will need to do more investigation to see why. Execution time will also be higher compared to an EditMode test.

2 Likes