So I have a hybrid app. App starts in PolySpatial. Here control centre, minimising the app and taking the headset on and off works fine. The PolySpatial part of my app is where I have the main menu with hover effects.
Then I switch to Metal mode to render my scenes so I can have box projected reflection probes and other visual effects.
In Metal mode if you take the headset off and put it back on Unity will just hold that frame forever but stay open. There are also other issues, with everything working fine, if I open the control centre it will also hold the last frame, but also close the app (so I assume the 2 second timeout for composite mode is doing this). Lastly if you minimize the app and re-open it you see nothing but the camera pass through feed until you force shut the app and restart.
Specific’s about taking the headset on and off: I’m on Unity 6.0.26, VisionOS 2.0.4 (I also tried 6.0.30 and 2.1.2 but got the same result). I had a look at Classes/UnityAppController+Rendering.mm, I notice the code in “repaintDisplayLink” in Unity 6.0.26 is different then your post here:
Possible it’s different to catch more edge cases, but in any case when I take the headset off it freezes. If I try the code from your post it does not freeze which is great, but the other issues still exist. So I don’t know if there has been a regression there?
Any tips on how to work around the app not being able to be minimized and how to stop the app breaking when bringing up the control centre? I did have a terrible idea where if someone minimizes the app I could just fully close it. But that wouldn’t account for the control centre.
I also have an issue where when a load scenes sometimes the app closes. But I assume this is related to the 2 second timeout so I assume if I fix the control centre maybe my scene loading will work. I’m loading the scenes from downloaded asset bundles so I can’t pre-load the shaders before hand (my guess was that the shader compilation is the thing hanging rendering for too long, could be wrong).
Maybe found a clue, if I comment out this function I can open the control centre and take the headset off.
I still have an issue where I can’t minimize the app, when I re-open it the app just get’s stuck on the “loading” SwiftUI that is meant to come up before metal rendering mode starts. You have to hard reset to get past it.
Not sure if commenting out that will crease issues when I try and go back to RealityKit
Uh… well yeah that would prevent the pause, but also completely breaks VR
Can you remind me if you’ve submitted a bug for this yet? It’s possible that this is just a bug with Hybrid apps in general, but it’s important to know that I’m looking at exactly the same scenario you’re encountering. Even if you encounter this issue with the basic visionOS template or sample scenes, it will help to have a bug in the system with an attached project that can reproduce the issue. Could you please do that (Help > Report a Bug...), if you haven’t already or share the incident ID with me if you have?
I’ll try out a Hybrid build on the versions you listed above to see if I can replicate the issue.
The VR/Metal mode relies on us setting up that reference to the compositor layer. That’s how we know whether or not the compositor is active and we can render to it. Does Metal rendering work if you cut out that code? I’d be surprised if it does…
It does yes. My app starts in polyspatial where I have a main menu. Then transitions to Metal when you enter a scene with 3D content. That all works somehow.
Any ideas how I can get the applicationDidEnterBackground callback to work? It doesn’t seem to ever get called.
I know this is a little nuclear but I was thinking of just shutting down Unity if someone closes the app.
I still need to test if I can get back to PolySpatial
Quick update on this one. I was able to replicate what you’re seeing, and it seems to be specific to Hybrid mode, but I can’t quite figure out why. An app built for Metal mode doesn’t have this issue when you open Control Center. For Metal app mode, in the method you referenced above, _didResignActive is true when you open control center. But in a Hybrid app, it is false.
It’s a kind of wonky workaround, but if you just modify line 75 to be if(_didResignActive || isRendererRunning), you should be able to trigger control center without issue. I’ll spend some more time trying to figure out what’s going on tomorrow and rewrite this logic so it makes a bit more sense and ensure I’m not missing some other edge case.
I’m not entirely sure if this is a regression (possibly a change in behavior on visionOS) or if it was always an issue with Hybrid mode. I had no reason to think that app lifecycle issues like this would work any differently in Hybrid mode since it should be doing the same things as Metal mode. I can’t remember if I specifically tested this situation where you open control center when dong Metal rendering in Hybrid mode.
Thanks for looking into it! Yes unfortunately I need to stick with Hybrid because I want hover effects throughout my main menu and I want the extra rendering features when showing my 3D scenes.
I tried your fix, I think I have something slightly similar (although much messier that your suggestion) so thanks for that. Unfortunately it does not fix my issues with minimising the app. Your fix lets me take the headset off, put it back on and open the control centre without issue. But if I go to the VisionOS dash and back to the app I get trapped in a bounded loading state forever (until I force quit).
At the moment I’m trying to figure out if there is a way I can discern between the app being in the background when the control centre is open & the app being minimised so I can try and force the compositor to refresh only in that second case (although I do not yet know how to force it to refresh, one step at a time).
Oh also I was wrong about the repaint loop not running all the time. It does seem to.
So I have the worst solution ever that might pass Apple’s cert. Figured out a way to know if the app was minimised as apposed to the headset being removed or the control centre opening, checking if no scenes exist. I restored the rest of this script, these are my only changes.
Also kept your suggestion to make sure the app keeps rendering when the control centre is up or the headset is off. It is much cleaner than the one I wrote and I wouldn’t be surprised if what I wrote broke other things.
If you know a way to restore from a minimized sate into Metal mode for a hybrid app or a better way to do this I’d be very interested. But at least this unblocks me.