Vision Pro app gets killed when main thread hangs for more than 2 seconds

Unity Engine
, , ,
1

Hi,

I am running into an issue that no one else seems to have trouble with on the internet, while developing on Vision Pro :

While migrating an XR app that ran on other devices, we noticed that any time the main thread hangs (loading big resource, awaiting methods…) for more than a very precise 2 seconds threshold, the OS kills the app (it’s not a crash, the app gets closed properly, but without any warning nor message)

This is systematic and causes issues with multiple scenarios :

  • Bypassing the immersive splashscreen crash by using a splashcreen scene, but loading an important scene afterwards, is impossible

  • Using asynchronous methods that can have “big” treatment times (more than 2sec) with is very common while downloading 3D resources on runtime

I have managed to bypass this by rewriting everything in coroutines as I went, but sometimes it’s not at all the best way to do it compared to “await”.

Is anyone aware / blocked by this as well ?

Thanks

2

Maybe it is related to this discussion?

3

Oh thanks, I had missed this thread (i searched for keywords similar to my issue before posting i promise :frowning: )

I am on Unity 6 and this message suggests a fix should be in already. Maybe @mtschoen can confirm : for fully immersive metal rendering mode on Unity 6, the 2sec threshold is still an issue ?

4

I’m also interested in any updates on fixes/workarounds for the 2 second issue. We’re experiencing it on Unity 6 + visionOS XR Plugin 2 in a Metal Rendering app. We could probably integrate the workaround given in the above linked thread, but I believe it only works with Unity 2022 and the embedded 1.1.6 version of the visionOS XR Plugin.

@mtschoen Is it possible to get an updated version of this workaround example for Unity 6 and visionOS 2? Or even better, do you have an ETA on when the modifications in the workaround will be integrated in Unity directly?

5

Hi, after checking it seems like the workaround is already in Unity 6. But the 2sec threshold is still an issue…

6

This is correct. The trampoline code in Unity 6 and 2022.3.26+ should include the fixes I described in that linked post. Indeed, the 2 second threshold is an issue, and will be until Apple decides to change it. Put simply, if your app doesn’t provide a new frame after 2 seconds (i.e. if some blocking function is running on the main thread for >2 seconds) the OS will kill the app. If you want them to change this, please submit feedback to Apple through their Feedback Assistant.

Slightly earlier in that thread I provided a demo project which demonstrates how to dismiss the immersive space before the long-running main thread code executes and re-open it afterward. This assumes that you have control over whatever is causing the lag spike. If it’s happening randomly or you have no way of detecting the lag spikes, you’re out of luck. By design, Apple isn’t letting your app stall for more than 2 seconds.

7

As you say in your more recent post, the lifecycle fixes are in Unity 6 and all plugin versions greater than 1.1.6. The “long load” example I provided should also still work on the latest Unity and plugin versions.

8

Hm. Is the “dismiss immersive space” technique working when using fully immersive metal app like we do ?

9

Yep! That sample project uses the VR app mode, which was renamed to Metal for Unity 6/2.x packages.

1 Like
10

Oh. I’ll check this out then. Tbh when I read “swift loading screen” and “dismissing immersive space” I immediatly thought it couldn’t do the trick for VR apps

11

Yeah the terminology can get a little confusing between the words that Apple use and the words we use in Unity. The immersive space I’m referring to does actually apply to both modes. In any case, I think my linked post should have all the info you need.

12

So if I’m understanding correctly, the long load example should run in Unity 6 with 2.x version of the Vision XR plugin without the need to replace UnityAppController.mm or UnityAppController+Rendering.mm in the generated Xcode project?

I’m trying that out but get an error about LoadingWindow being undefined, and then the application goes to background and doesn’t return.

Accessing Environment<OpenWindowAction>'s value outside of being installed on a View. This will always read the default value and will not update.

No Scene with id 'LoadingWindow' is defined

Accessing Environment<DismissImmersiveSpaceAction>'s value outside of being installed on a View. This will always read the default value and will not update.

Wait a second...

UnityEngine.DebugLogHandler:Internal_Log_Injected(LogType, LogOption, ManagedSpanWrapper&, IntPtr)

UnityEngine.DebugLogHandler:Internal_Log(LogType, LogOption, String, Object)

UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[])

UnityEngine.Logger:Log(LogType, Object)

UnityEngine.Debug:Log(Object)

<LongLoadCoroutine>d__4:MoveNext()

UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)

UnityEngine.MonoBehaviour:StartCoroutineManaged2_Injected(IntPtr, IEnumerator)

UnityEngine.MonoBehaviour:StartCoroutineManaged2(IEnumerator)

UnityEngine.MonoBehaviour:StartCoroutine(IEnumerator)

LongLoadTest:SimulateLongLoad()

UnityEngine.Events.InvokableCall:Invoke(Object[])

UnityEngine.Events.InvokableCall:Invoke()

UnityEngine.Events.UnityEvent:Invoke()

UnityEngine.UI.Button:Press()

UnityEngine.UI.Button:OnPointerClick(PointerEventData)

UnityEngine.EventSystems.ExecuteEvents:Execute(IPointerClickHandler, BaseEventData)

UnityEngine.ExpressionEvaluator:EvaluateTokens(String[], T&, Int32, Int32)

UnityEngine.EventSystems.ExecuteEvents:Execute(GameObject, BaseEventData, EventFunction`1)

UnityEngine.XR.Interaction.Toolkit.UI.UIInputModule:ProcessPointerButton(ButtonDeltaState, PointerEventData)

UnityEngine.XR.Interaction.Toolkit.UI.UIInputModule:ProcessTrackedDevice(TrackedDeviceModel&, Boolean)

UnityEngine.XR.Interaction.Toolkit.UI.XRUIInputModule:DoProcess()

UnityEngine.XR.Interaction.Toolkit.UI.UIInputModule:Update()

ARPredictorRemoteService <0x127572720>: Query queue is not running.

AR data provider state changed. New state is Paused.

UnityEngine.DebugLogHandler:Internal_Log_Injected(LogType, LogOption, ManagedSpanWrapper&, IntPtr)

UnityEngine.DebugLogHandler:Internal_Log(LogType, LogOption, String, Object)

UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[])

UnityEngine.Logger:Log(LogType, Object)

UnityEngine.Debug:Log(Object)

UnityEngine.XR.VisionOS.VisionOSSessionProvider:DataProviderStateChangeHandler(IntPtr, AR_Data_Provider_State, IntPtr, IntPtr)

AR data provider state changed. New state is Paused.

UnityEngine.DebugLogHandler:Internal_Log_Injected(LogType, LogOption, ManagedSpanWrapper&, IntPtr)

UnityEngine.DebugLogHandler:Internal_Log(LogType, LogOption, String, Object)

UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[])

UnityEngine.Logger:Log(LogType, Object)

UnityEngine.Debug:Log(Object)

UnityEngine.XR.VisionOS.VisionOSSessionProvider:DataProviderStateChangeHandler(IntPtr, AR_Data_Provider_State, IntPtr, IntPtr)

Presenting a drawable without a device anchor. This drawable won't be presented.

AR data provider state changed. New state is Paused.

UnityEngine.DebugLogHandler:Internal_Log_Injected(LogType, LogOption, ManagedSpanWrapper&, IntPtr)

UnityEngine.DebugLogHandler:Internal_Log(LogType, LogOption, String, Object)

UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[])

UnityEngine.Logger:Log(LogType, Object)

UnityEngine.Debug:Log(Object)

UnityEngine.XR.VisionOS.VisionOSSessionProvider:DataProviderStateChangeHandler(IntPtr, AR_Data_Provider_State, IntPtr, IntPtr)

Hand anchors can only be queried when the hand tracking provider is running.

Hand anchors can only be queried when the hand tracking provider is running.

Presenting a drawable without a device anchor. This drawable won't be presented.

-> applicationDidEnterBackground()

AR data provider state changed. New state is Paused.

UnityEngine.DebugLogHandler:Internal_Log_Injected(LogType, LogOption, ManagedSpanWrapper&, IntPtr)

UnityEngine.DebugLogHandler:Internal_Log(LogType, LogOption, String, Object)

UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[])

UnityEngine.Logger:Log(LogType, Object)

UnityEngine.Debug:Log(Object)

UnityEngine.XR.VisionOS.VisionOSSessionProvider:DataProviderStateChangeHandler(IntPtr, AR_Data_Provider_State, IntPtr, IntPtr)
13

Let me take a look. It’s been a minute since I posted that project so I have to remind myself how it all works. But yeah, you shouldn’t need the modifications to UnityAppController anymore. The LoadingWindow bit may require you to add some code to the Xcode project, though. I can’t remember how I set that up…

Edit: I failed to read my own readme :sweat_smile: you do need to customize the app controller in the Xcode project.

14

OK so I loaded the old project back up, and at first I also forgot to read my own readme.txt and was confused about why it wasn’t working. I don’t see the log message No Scene with id 'LoadingWindow' is defined so I think you may be missing something there.

The following logs are expected, even though they are the Error log type:

Accessing Environment<OpenWindowAction>'s value outside of being installed on a View. This will always read the default value and will not update.
Accessing Environment<DismissImmersiveSpaceAction>'s value outside of being installed on a View. This will always read the default value and will not update.

You’re also getting -> applicationDidEnterBackground() like I was at first, until I realized I needed the changes in UnityAppController+Rendering.mm. Although the Unity6 trampoline (Xcode project) has the latest fixes to app lifecycle, etc. there are some hacks I did in those files specific to this example.

For clarity, here’s what I ended up with, grafting those changes over to the version of that file from 6000.0.30 (it’s an older one but that file is pretty stable these days):

- (void)repaintDisplayLink
{
    if (_LayerRenderer == nil)
    {
        if (!_didResignActive) {
            UnityDisplayLinkCallback(_displayLink.timestamp);
            [self repaint];
        }
    }
    else
    {
        auto isRendererRunning = _LayerRendererState == cp_layer_renderer_state_running;
        auto isAppActive = [[UIApplication sharedApplication] applicationState] == UIApplicationStateActive;

        if (isAppActive)
        {
            if (isRendererRunning)
            {
                // Disable pause for long load example
                /*
                if (UnityIsPaused())
                {
                    UnityWillResume();
                    UnityPause(0);
                }
                 */

                if(self.engineLoadState >= kUnityEngineLoadStateRenderingInitialized)
                {
                    UnityDisplayLinkCallback(0.0);
                    [self repaint];
                }
            }
            else
            {
                _LayerRendererState = cp_layer_renderer_get_state(_LayerRenderer);

                // Added for long load sample
                UnityDisplayLinkCallback(0.0);
                [self repaint];
            }
        }
        else
        {
            if(_didResignActive)
            {
                UnityDisplayLinkCallback(0.0);
                [self repaint];
            }
            // Disable pause for long load example
            /*
            else if (!UnityIsPaused())
            {
                UnitySetPlayerFocus(0);
                UnityPause(1);
            }
             */
            else
            {
                _LayerRendererState = cp_layer_renderer_get_state(_LayerRenderer);
            }
        }

        if (_ShouldDispatchRepaint)
            dispatch_async(dispatch_get_main_queue(), ^{ [self repaintDisplayLink]; });
    }
}

You shouldn’t need any changes to the base UnityAppController.mm file anymore.

If you want to permanently apply these changes to the Xcode project, you’ll either need to modify the file in a build postprocessor, or modify the file in your Unity installation so that all future builds have these changes made. Bear in mind that the “hack” in question is to disable pausing; Unity will always run scripts and audio etc. while the app is open. If you need this long-load fix and pausing, you’ll need to modify the sample to suit your needs. Something along the line of an extra boolean in the AppController to signify “is long load” should do it.

15

Hi, thanks, I get the “No Scene with id ‘LoadingWindow’ is defined” too, and I think it’s because we work with 2.x version of visionos xr and there is no “UnityMain.swift” to modify anymore in the package

16

Update @Cannos : I got it working by adding this at the end of the “UnityCompositorSpace.swift” file : (from the old UnityMain file)

    }.immersionStyle(selection: .constant(VisionOSImmersionStyle), in: VisionOSImmersionStyle)
        .upperLimbVisibility(VisionOSUpperLimbVisibility)
        .persistentSystemOverlays(VisionOSPersistentSystemOverlays)
    WindowGroup(id: "LoadingWindow") {
        Text("Loading...")
    }.defaultSize(width: 0.2, height: 0.15)

But I stumble across the “no input after returning to the app” mentionned in the readme file, and I’m not sure where to apply these modifications

17

I applied the UnityAppController+Rendering.mm changes to disable pausing that @mtschoen mentioned above, as well as the the UnityCompositorSpace.swift change from @Sashell. After that I had the same issue with no input after returning.

So then I commented out the contents of applicationWillResignActive in UnityAppController.mm as mentioned in the readme. After that I had input when returning. Though if I try to Simulate Long Load an additional time after returning the app crashes. I haven’t looked too deeply into that yet.

18

Ah, good point. I hadn’t tested input. So disregard my comment about not needing the UnityAppController changes.

I’m fairly certain I tried it multiple times back when I first threw this together. However, this isn’t something we test on each version as they go out. The reality is that this is just a hack to demonstrate how you might accommodate things like a one-time shader warmup at the start of the app. Another approach to try on 2.x packages is to use a Hybrid app where you start your app in a volume, do your long shader warmup or whatever with RealityKit as your renderer, and then switch your volume camera to Metal mode. Fundamentally you must not have a CompositorLayer (Apple terminology for what we call Metal mode) if your app stalls for more than 2 seconds.

I know this is frustrating, but we’re stuck against this hard line Apple is taking about the 2 second deadline. Please submit feedback to Apple that you need this deadline relaxed or removed. All I can do is show you potential ways to hack around it.

19

Hi again,

I’ve tried to comment the content of UnityAppcontroller too, but I came across another issue : opening a flying system window (control center, or the “force quit apps window”) closed my app.

So i got back into the readme file, and tried as mentionned to activate the background running in the input settings, and reverted the modifications to UnityAppController.mm. It worked !

So, we indeed do not need any modification to this file, but instead :

Project Settings > Input System Package > Settings : “Background Behavior” = Ignore Focus.

image
image480×140 16.9 KB

And finally, my app can launch without being killed instantly. Thanks everyone

20

Oh, nice! I’m glad I wrote that readme, because I’ve already forgotten a lot of what went into that sample project. I think the core issue is that there’s still probably a place where a call to UnitySetPlayerFocus or something is slipping through, but I’m glad you found your answer. Cheers!