Our fully immersive VR app has been crashing on device after running for a few minutes. In some cases, it crashes on launch. In both scenarios, the app is allocating a ton of hand anchors every frame, which leads to the app running out of memory.
We’re running on Unity 2022.3.11f1 on the latest plugin (0.6.3) on Xcode 15.1 Beta 3.
I attached some screenshots of the memory captures on Xcode while running on device. Every single allocation in the first screenshot went through the following method: NativeApi.HandTracking.ar_hand_anchor_create().
I looked through the VisionOSHandProvider code (in the visionOS XR plugin package), and this issue seems to be related to the TODO in the following code snippet.
Hi there! Thanks for reporting the issue, and sorry you’re having trouble.
I was able to replicate the problem and the fix is relatively easy. If you just re-use those pointers that come out of ar_hand_anchor_create, you should be able to avoid the leak. I introduced a pair of IntPtr instance variables to that class, and replaced the code you highlighted with
if (m_LeftHandAnchor == IntPtr.Zero)
m_LeftHandAnchor = NativeApi.HandTracking.ar_hand_anchor_create();
if (m_RightHandAnchor == IntPtr.Zero)
m_RightHandAnchor = NativeApi.HandTracking.ar_hand_anchor_create();
And of course use those instead of leftHandAnchor in the subsequent calls to ar_hand_tracking_provider_get_latest_anchors and GetHandData.
There’s also a new block at the beginning TryCreateNativeProvider to reset those variables, but it’s not strictly necessary unless you stop/start the AR session.
You should be able to “embed” the xr.visionos package (copy it from Library/PackageCache/com.unity.xr.visionos@0.6.3 to Packages/com.unity.xr.visionos locally and apply these changes as a quick workaround. The next version we ship should have this fix applied.
Thank you so much! I verified this fixes the out of memory crashes.
Unfortunately, this doesn’t fix our issue of crashing on launch, when opening the app through the Vision Pro Home menu (it always runs fine when launched from the debugger). I’ve noticed a few issues related to app lifecycle. When quitting the app via the control center menu (the icon that shows up at the top of your view), the app doesn’t seem to properly quit until it runs out of memory, at which point the gray “Loading” panel closes. It also seems the “OnDestroy” MonoBehaviour methods are not getting called on application quit.
These scenarios are much harder to debug. The only useful pieces of information I’ve been able to get are this assert:
Assertion failed: (0 && “Failed to allocate IOGPUDeviceShmem.”), function -[IOGPUMetalDeviceShmem initWithDevice:shmemSize:shmemType:], file IOGPUMetalDeviceShmem.m, line 43.
and this stack trace, which causes the out of memory crash on quit:
Try adding -force-gfx-direct (or -force-gfx-st) to the app command line args (via Edit Scheme, add command line args). The -direct version in particular should give you an actual stack trace, instead of something blowing up on our gfx worker thread based on queued up work. (I’m 99% sure these are implemented on Metal; if they don’t seem to change much, they might not be!)
But, dumb question – do you have the splash screen disabled?
Got a different message with the command line args. To get to this point, I launched the app through the debugger, then quit it via the control center (which is when the AR data provider state changed to “Paused”), then closed the “Loading” panel.
After a couple of minutes of staying attached to the process, the MTLCommandEncoder messages showed up. Note that the Memory started increasing continuously AFTER I tried to quit the app.
Ah – uncheck/disable “Show Splash Screen” for now. There’s a known bug if the splash screen is enabled; fixing it just hasn’t been as quick because the code is in the core unity player vs. code that lives in our package. (Though – that stack trace doesn’t look like the splash screen issue, but it should still be disabled.)
Awesome! It seems like that fixed the initial crash on launch.
Unfortunately, the crash on quit is still happening. The real issue here is we’re unable to re-launch the app for a few minutes after quitting. Regardless, I think this should unblock us for now. Thanks again for your help!
Just in case this helps, I did spend some time looking into this, and I found that after quitting the app, UnityIsPaused() in the below snippet never returns true. I set breakpoints on all the callbacks that should trigger when the app quits, but they were never hit. The app will continue to try to render, which seems to be the cause of the memory leak and crash. Maybe?
Yeah, we have some issues with app lifecycle that we’re investigating. Unity isn’t getting the signal that the app is backgrounded, so ARKit stops and we don’t know to re-start it. For now, you’ll have to force quit the app (hold both buttons on the device for ~4 seconds) or stop/start it with Xcode to re-launch it once it goes in the background.
Ooh! I didn’t know that trick to force quit the app. Thanks for the info, and please let me know when you have updates on this.
The strange part is the app code is getting the update that the AR session stopped (in VisionOSSessionProvider), but it’s not piped through on the native side of the code.
Yeah. The OS stops ARKit data providers, and we get notified about that via callback. But the core engine itself which controls the render thread, etc. is controlled by a different mechanism. We’re actively working on a fix for this, but because the change (likely) has to happen in the core engine code, it will require updating to that new version when it is released. No ETA yet but I’ll keep you posted.
