So, in my game code, I have some systems that initialize via methods that utilize the RuntimeInitializeOnLoadMethod attribute. However, it turns out that methods with this attribute will also execute when running Play Mode unit tests. What ends up happening during a unit test is that my game systems initialize and one of them executes a scene load that interferes with/breaks the tests and puts the test runner into a locked state where the tests never finish.
Now, since we’re indeed entering Play Mode in these unit tests, I can somewhat see why it makes sense that methods with this attribute would execute. However, what it means is that if you do unit tests (or performance benchmarking using Unity’s performance benchmarking API) you’re locked out of writing any logic in those methods that could potentially interfere with and break your tests. This seems limiting in a way that isn’t helpful.
I’ve looked all over the API and, unless I’ve missed it, there doesn’t seem to be a reliable way to detect at runtime whether or not we’re running a unit test. If there were, I’d probably settle for the compromise of making my attributed methods check for that… though I emphasize that that’s a compromise because polluting game code with unit test checks is not clean. A better solution would be in order.
Try implementing ITestRunCallback in a TestRunner assembly. When RunStarted gets called, write a bool to EditorPrefs and in your runtime init method check if that key is set. You need to #if UNITY_EDITOR that code though. And reset the key when the run stopped.
However I kinda expect the InitOnLoad method to run before TestStarted gets called. If that is the case I’d try to create another RuntimeInit method but in the TestRunner assembly and change the EditorPrefs key there. The order of execution in that case I’m not aware of though, it may be undefined.
Yeah, unfortunately I can confirm that all RuntimeInitializeLoadType options used with RuntimeInitializeOnLoadMethod will execute before the first test callback is executed in that interface.
In regards to your alternative: SubsystemRegistration is the first RuntimeInitializeLoadType that is executed… and since all of my game code only uses AfterAssembliesLoaded or later, what you suggested could be an okay workaround for now. However, as you pointed out, there’s probably an order-of-execution issue that would arise if anyone wanted to initialize anything at the SubsystemRegistration phase. Also, I’d need to use PlayerPrefs instead of EditorPrefs, as we need to be able to run these tests on builds occasionally.
Really hoping Unity sees this, as this seems like quite the drawback.
I use RunTimeInitializeOnLoadMethod to create a single point of entry. From there, I hang everything else on a ScriptableObject, with a reference to another type of ScriptableObject, with references to whatever you’d like to code into that asset from there. So, when it’s time to run a test you just swap that one single reference from pointing to the object that defines everything you want to initialize when launching a released app, to everything you want to initialize and inject instead into your mock dependencies as interfaces. It’s just, snap, change one reference and run tests. Snap, change back to normal and play again. You can technically change your whole project from a fighting game, to a racing game, to a dating game, or whatever you want to initialize on play by changing out that one single reference, if you set it up that way. Everything in any scene is just static scene stuff that always goes with that little diorama, no matter the context or what type of game is being played in that scene. If you really wanted to bind an NPC to a spot in that scene no matter what, as opposed to a spawnpoint / encounter metadata object for hinting at important locations in that environment, you could still just change that one root reference in your project to define what AI is injected into each character. They could be swapped from an aggro enemy, to a quest giver, to an ally companion by just swapping the object that gets initialized when you press play. So, you can easily create a mock testing context object to initialize instead. It just has to be set up for that kind of workflow, and it all works off of RunTimeInitializeOnLoadMethod. I just use one entry point, and make everything easily modifiable in the editor from there.
[RuntimeInitializeOnLoadMethod(RuntimeInitializeLoadType.BeforeSceneLoad)]
private static void InitializeThis() {
#if UNITY_INCLUDE_TESTS
var sceneName = UnityEngine.SceneManagement.SceneManager.GetActiveScene().name;
var type = Type.GetType("UnityEngine.TestTools.TestRunner.PlaymodeTestsController, UnityEngine.TestRunner");
var controller = FindFirstObjectByType(type);
if (sceneName.StartsWith("InitTestScene") && controller != null) { return; } //Only way to do it for now
#endif
...
}
For anyone who’s still stuck on this, the above workaround works well for Play Mode tests!