LocalizationBehaviour and async tasks causing issues for play mode tests

Unity Engine
, ,
1

Hello

I’m writing Unity tests that test my actual scene using the [UnityTest] attribute and SceneManager.LoadSceneAsync. Between consecutive tests, I delete all existing GameObjects, so the next test can load fresh state again.

Everything worked fine, but as I wrote more tests I’m running into errors with the localization package because it seems there are asynchronous actions starting in the background that I cannot seem to hook into or wait for. Most of it also seems to be static, so it probably doesn’t reset state properly between tests. This is resulting in tests that fail only if other tests ran first, and running them again in isolation magically make them pass.

I’m getting the error below, but it is not located inside my application code nor in my test code, which also makes it impossible to try-catch (though ironically it is still failing my test). It seems that the localization package is trying to start some kind of background process:

OnStartupWhenNotLoggedInAuthenticationIsRequested (0,989s)
---
Unhandled log message: '[Exception] MissingReferenceException: The object of type 'LocalizationBehaviour' has been destroyed but you are still trying to access it.
Your script should either check if it is null or you should not destroy the object.'. Use UnityEngine.TestTools.LogAssert.Expect
---
UnityEngine.MonoBehaviour.StartCoroutine (System.Collections.IEnumerator routine) (at <cfc1ad890650411e946cff2e6f276711>:0)
UnityEngine.Localization.LocalizationBehaviour.ReleaseNextFrame (UnityEngine.ResourceManagement.AsyncOperations.AsyncOperationHandle handle) (at Library/PackageCache/com.unity.localization@1.0.3/Runtime/Utilities/LocalizationBehaviour.cs:29)
UnityEngine.ResourceManagement.AsyncOperations.AsyncOperationBase`1+<>c__DisplayClass57_0[TObject].<add_CompletedTypeless>b__0 (UnityEngine.ResourceManagement.AsyncOperations.AsyncOperationHandle`1[TObject] s) (at Library/PackageCache/com.unity.addressables@1.19.4/Runtime/ResourceManager/AsyncOperations/AsyncOperationBase.cs:291)
DelegateList`1[T].Invoke (T res) (at Library/PackageCache/com.unity.addressables@1.19.4/Runtime/ResourceManager/Util/DelegateList.cs:69)
DelegateList`1:Invoke(AsyncOperationHandle`1) (at Library/PackageCache/com.unity.addressables@1.19.4/Runtime/ResourceManager/Util/DelegateList.cs:73)
UnityEngine.ResourceManagement.ResourceManager:Update(Single)
MonoBehaviourCallbackHooks:Update() (at Library/PackageCache/com.unity.addressables@1.19.4/Runtime/ResourceManager/Util/MonoBehaviourCallbackHooks.cs:26)
---
Loading scene 'Main' for testing
MissingReferenceException: The object of type 'LocalizationBehaviour' has been destroyed but you are still trying to access it.
Your script should either check if it is null or you should not destroy the object.

I’ve been scouring the localization classes for entry points so I can wait for all async tasks to finish executing, and was able to partially remedy the issue by using Reflection to hook into DelayedActionManager.IsActive so I can wait for it, which admittedly is horrible, but everything is internal and I’m at my wits’ end at this point. This fixed the problem initially, but where it initially failed on the second test, it now fails on the fourth test, with exactly the same error.

An alternative solution would be to try and disable the localization package entirely during testing purposes, but I’m not sure if that is possible and at some point I might need to test that things get translated properly, so it would be a poor solution at best.

It is not clear to me if this is a localization package bug, or if I’m doing something wrong here. I’ve tried many things already, such as waiting for a couple of seconds (horrible), waiting for a couple of frames, disabling locales, adding a dummy locale provider, and so on, but to no avail.

In case it matters, I’m using Zenject in my scene and write tests as well, but I don’t think it is causing the issue.

2

That error looks like a bug we fixed recently. It was caused by entering and exiting play mode. We have a fix for it in the next release.

3

Hi @karl_jones and thanks for your reply.

That’s great to hear! Will this be a new version of the unity localization package (or perhaps Unity itself)? If so, I’ll try to update, retest as soon as it’s released, and post the results here.

For others looking into this problem, I’ve managed to “delay” the issue further for now by adding LocalizationSettings.Instance.ResetState(); to TearDown. That, and I’m waiting for
LocalizationSettings.InitializationOperation in my tests, as well as still using the Reflection hack to wait for delayed operations to finish if there are any. This workaround likely doesn’t fix it fully, but it got me to being able to run more tests without the problem exposing itself, which is good enough until a final fix arrives.

4

The fix will be in the next patch release. In about 1-2 weeks.

5

I wanted to confirm we do indeed have better results with the latest 1.0.5 package - the error around the LocalizationBehaviour is gone for the time being, and we were able to remove our reflection workaround.

We’re hitting a separate issue now where GetLocalizedStringAsync(...).Task on the main thread never seems to complete - running GetLocalizedStringAsync(...).WaitForCompletion() or simply using GetLocalizedString(...) works fine, it just seems as if the task itself is never marked as completed or failed sometimes (the event handlers also aren’t invoked in that case).

We’ve currently managed to work around this, so it’s not blocking, and it may well be that we are doing something wrong here, so feel free to ignore until I am able to post more details :-).

In any case, thanks for the response.

6

Th

That sounds like an Addressable bug, could you please file a bug report so they can look into it?