IN-60565 Fully immersive apps always emit audio from grey window and mutes when window is dismissed

It looks like fully immersive app audio always sounds like its emitting (in 3d) from the grey window that appears when you first launch the app. We’ve been told it’s harmless to dismiss this window for now (since it’s not supposed to be there), but when you dismiss it, all audio from the app mutes. I don’t think it’s possible to get audio to return once the window is dismissed. This means we have to leave the window blocking the experience if we want to hear audio, which is a bummer. I have submitted a bug for this, see IN-60565.

From other posts on the forums, its seems this happens with Unity’s built-in audio as well as WWise. I can also confirm it happens with FMOD as well.

Beyond just fixing the audio muting issue, we’re concerned we will be unable to create spatial 3D audio for individual emitters like we’re used to. Will this possible once the bug where it emits from the window is fixed?

Idk if this will work for you/FMOD, but after dismissing the loading window I can bring back audio (this time fully diotic/2d) by bringing down the control widget (the little down arrow that always floats and steals input if you glance at it) and dismissing it again.

oh interesting! Unfortunately, that workaround doesn’t help with FMOD as far as I can tell. But an interesting clue about what’s going on tho…

Any update on this issue? This is making it hard test our experience (either you have a window blocking the experience or no audio) and impossible to work on 3D spatialized audio.

I have created an easy repro project for this using the built-in audio system in Unity (also added this to bug IN-60565):

Simply run and note how the audio emits from the grey window. Then dismiss the window and audio is gone for good. Any help is much appreciated!

This is also a major issue for us. It would be nice to know when this issue is expected to be resolved - and if you good folks at Unity are actively working on it. :slight_smile:

Hi there! I have good news on multiple fronts!

First of all, I was able to fix the mute issue by eliminating the loading window entirely. If you “embed” the com.unity.xr.visionos package and apply this patch (it may not work automatically but you should get the gist), the app can launch directly into the immersive space without opening the loading window.

diff --git a/Packages/com.unity.xr.visionos/Editor/VisionOSBuildProcessor.PostProcessor.cs b/Packages/com.unity.xr.visionos/Editor/VisionOSBuildProcessor.PostProcessor.cs
index 6bf788ed1..36a619616 100644
--- a/Packages/com.unity.xr.visionos/Editor/VisionOSBuildProcessor.PostProcessor.cs
+++ b/Packages/com.unity.xr.visionos/Editor/VisionOSBuildProcessor.PostProcessor.cs
@@ -17,6 +17,8 @@ static partial class VisionOSBuildProcessor
 #if UNITY_VISIONOS
         const string k_SceneManifestKey = "UIApplicationSceneManifest";
         const string k_SupportsMultipleScenesKey = "UIApplicationSupportsMultipleScenes";
+        const string k_SessionRoleKey = "UIApplicationPreferredDefaultSceneSessionRole";
+        const string k_SessionRoleValue = "CPSceneSessionRoleImmersiveSpaceApplication";
         const string k_HandsTrackingUsageDescriptionKey = "NSHandsTrackingUsageDescription";
         const string k_WorldSensingUsageDescriptionKey = "NSWorldSensingUsageDescription";
 
@@ -132,8 +134,10 @@ static void FilterPlist(string outputPath, VisionOSSettings settings, VisionOSSe
                 if (appMode == VisionOSSettings.AppMode.VR)
                 {
                     var sceneManifestDictionary = plist.CreateElement("dict");
-                    var valueElement = plist.CreateElement("true");
-                    sceneManifestDictionary[k_SupportsMultipleScenesKey] = valueElement;
+                    var supportsMultipleScenesValue = plist.CreateElement("true");
+                    sceneManifestDictionary[k_SupportsMultipleScenesKey] = supportsMultipleScenesValue;
+                    var sessionRoleValue = plist.CreateElement("string", k_SessionRoleValue);
+                    sceneManifestDictionary[k_SessionRoleKey] = sessionRoleValue;
                     plist.root[k_SceneManifestKey] = sceneManifestDictionary;
                 }
 
diff --git a/Packages/com.unity.xr.visionos/Runtime/Plugins/visionos/UnityMain.swift b/Packages/com.unity.xr.visionos/Runtime/Plugins/visionos/UnityMain.swift
index f3eb652da..4b600a2bb 100644
--- a/Packages/com.unity.xr.visionos/Runtime/Plugins/visionos/UnityMain.swift
+++ b/Packages/com.unity.xr.visionos/Runtime/Plugins/visionos/UnityMain.swift
@@ -53,22 +53,9 @@ struct MyApp: App {
             }
     }
     
-    @Environment(\.openImmersiveSpace) private var openImmersiveSpace
-    @Environment(\.dismiss) private var dismiss
-    
     @State private var immersionStyle: ImmersionStyle = .full
     
     var body: some Scene {
-        WindowGroup {
-            Text("Loading").onAppear() {
-                Task { @MainActor in
-                    await openImmersiveSpace(id: "CompositorSpace")
-                }
-                // TODO: doesn't work?
-                self.dismiss()
-            }
-        }
-        
         ImmersiveSpace(id: "CompositorSpace") {
             let _ = UnityLibrary.GetInstance()
             let unityClass = NSClassFromString("UnityVisionOS") as? NSObject.Type
-- 

You can also make these changes to the Xcode project manually, but you will need to do so on every build (an append build will overwrite the info.plist). But to make the changes manually, just delete the highlighted code in UnityMain.swift, and add a Preferred Default Scene Session Role to your app’s Info.plist (make sure it’s the Info.plist in the project root for the app, not the one under UnityFramework). There should already be an Application Scene Manifest element, so just click the + on that row and choose the Preferred Default Scene Session Role key. The value you want is Compositor Services Immersive Space Application Session Role.

This will launch your app directly into the immersive space and eliminate the loading window. Audio just works in this configuration, and since there’s no window to close, it won’t get muted! We are close to wrapping up our next release, so I’m not sure if this will make it in. One way or another, we’ll get this change into the package soon.

With that resolved, at least in the repro project you shared, there’s a configuration issue. First of all, the AudioSource’s Spatial Blend property is set to 2D.

Second, the audio source for the music loop is located at (0, 0, 0), in the same location as the camera rig. So you would need to move away from the tracking origin (and likely outside of the fixed 1.5m by 1.5m safety zone) to hear any spatialization. I was able to hear proper spatialization by dragging that slider all the way over to 3D, and placing the audio source at (0, 0, 1), roughly where the cube is located. I also switched the Volume Rolloff to linear, and the Max Distance to 10 meters, since that gave me a reasonable volume at that distance. In that configuration, if I turn my head left and right I hear the audio louder in the ear that is closer to the audio source.

Hope this helps. Please let us know if these workarounds fix the issue on your end. Good luck!

5 Likes

Thanks a lot!

One issue remaining, though: When leaving the app in the background for a while, audio seems to be missing once you activate the app again and cannot be recovered.

That should be fixed in the next release. I noticed it while testing, as well, but we recently made some updates to properly notify Unity about the app going in and out of the background.

2 Likes

Sounds great! Thanks.

I just tested this, and as you say the audio does not resume when you resume the app. However, I’m not sure whether or not this is expected behavior. Testing the same thing in the Editor. If you enter play mode, switch focus to another application, and back to the Editor, the audio does not resume. I haven’t tested in the iOS player but given that it happens in the Editor I wouldn’t be surprised if the same thing happens there.

The good news is that, in this upcoming version with fixed app lifecycle, your MonoBehaviors will properly respond to the OnApplicationFocus event, and you can unpause the audio source there. I just tested this script and it worked for me on device:

using UnityEngine;

[RequireComponent(typeof(AudioSource))]
public class PlayOnResume : MonoBehaviour
{
    AudioSource m_AudioSource;

    void Awake()
    {
        m_AudioSource = GetComponent<AudioSource>();
    }

    void OnApplicationFocus(bool hasFocus)
    {
        m_AudioSource.UnPause();
    }
}

This won’t work on the current live release (0.6.3) because OnApplicationFocus is not called. I’ll ping this thread when our 0.7 release ships and that should unblock you. I’ll get back to you as well regarding whether it is expected on other platforms that you have to explicitly unpause audio sources when the app regains focus.

OK I did a little more testing and I think we should consider this behavior (not unpausing automatically) a bug. I tested an iOS build and the audio resumes automatically when I go to the home screen and switch back to the app. The fact that I could reproduce it in the Editor seems to have been a total fluke.

The script I provided should be a decent workaround in the meantime. It shouldn’t break things if we fix the bug and you leave it around, either. I’ll keep digging on the core issue but we won’t be able to ship a fix until next year.

Thanks @mtschoen for looking into this. It’s greatly appreciated!

I’m afraid the problem lies a step deeper since no sounds can be played from when app gets into this bad state. It only happens in fully immersive mode when you background your app, launch another app with sounds and then return back to your app. From that point no sound whatsoever can be played.

Hm… this isn’t what I found but maybe I didn’t leave the app in the background long enough. Or it may be that the lifecycle fixes that I have on my end fix the issue. Can you please try again once we’ve shipped 0.7 and see if the issue persists?

Yes, of course. Any info on 0.7 release date? :slight_smile:

It happens rather quickly and consistently for us, though. It is a major roadblock for us, obviously, so until 0.7 is out I’ll try experimenting with creating a new basic project and see if I can reproduce.

We are currently doing QA on the 0.7 release. It should be available sometime this week.

2 Likes

I’m happy to report that PolySpatial 0.7.1 is available! Please update your packages and try the script I shared above to see if you are able to resume playing audio. Let us know if you are still having trouble.

Good luck!

2 Likes

@mtschoen Thanks for these fixes! Can confirm the grey window is gone and our spatial audio is working again!! I also don’t have any issues backgrounding our app, going and playing audio elsewhere, and returning to our app. Thanks for all the hard work on this :slight_smile:

Woo hoo! Glad to know things are working for you. I’m still investigating an issue reported by @da_sxr where audio fails to resume after backgrounding the app for “a while.” It seems like there’s a deeper background state that we fail to recover from. I’ve also observed VR apps crashing after being backgrounded for too long. Please keep an eye out for this and let us know if you encounter the same issues.

1 Like

Good news! I found a quick and dirty workaround for the audio mute issue. I still need to dig into why this code isn’t behaving as expected, but there are 3 calls to UnityUpdateMuteState in UnityAppController.mm that are supposed to mute and unmute the audio device, but for whatever reason it always mutes regardless of what value you give it. I found for my test project (a looping audio source, play on awake, with the PlayOnResume script from above on the same object) I was able to just delete/comment those lines and the audio would resume when I put the HMD back on.

You can find the code you need to modify in the generated Xcode project at Classes/UnityAppController.mm, and the 3 lines in particular are 142, 351, 400. If you want to avoid having to make this change for every new build, you can also safely modify this file in the Unity Editor installation itself, at /Applications/Unity/Hub/Editor/2022.3.16f1/PlaybackEngines/VisionOSPlayer/Trampoline/Classes/UnityAppController.mm.

I’m going to keep digging for a proper fix, but I thought I’d share the workaround in case it helps unblock anybody in the meantime.

3 Likes

Sounds good, @mtschoen! Thanks for looking into this.