[IN-72226] UGUI buttons are not working correctly if Unity is rendered into a visionOS Window instead of Volume

If you render Unity’s content into a visionOS WindowGroup with windowStyle(.plain) instead of windowStyle(.volumetric)Unity UI Button’s click-area is not consistent with rendering, e.g. if you look-and-tap on one button another one gets clicked.
We filed a bug for this (IN-68441) that also includes a repro-project and would love to get some help fixing or working around this. Maybe @joejo (who has always been very helpful with out UI problems) or someone else from Unity has any idea?

More information:
We render our game into a RenderTexture and show that on a plane inside a bounded VolumeCamera. What we do different then, is that in Swift we use a .windowStyle(.plain) instead of .windowStyle(.volumetric) to render our game inside a visionOS Window. This has mostly worked (our arcade game Gibbon: Beyond the Trees releases tomorrow on the Vision Pro), but until now we haven’t used our menu we built in Unity UI. For the next update we’re now trying to get our Unity UI to work but are our menu’s buttons don’t work. Probably because of our changes in Swift. We know rendering into a Window is officially not supported, but maybe this can be easily fixed and at some time in the future you could allow Unity users to choose to render into a Volume or Window.
We also have reported a related bug with rendering into Window that is still under review.

If it’s easier to access here’s the repro project and a step-by-step guide on how to reproduce the problem:

If you render Unity’s content in a visionOS WindowGroup with windowStyle(.plain) instead of windowStyle(.volumetric) interactability of buttons is not consistent with rendering. The inconsistent behaviour is amplified when moving the Window around.

Steps to Reproduce:

  1. Create a build and start in visionOS Simulator from Xcode

  2. Interact with buttons, everything works as expected

  3. Move Volume around (also holding shift to move along z-axis), and repeat step 2. Note that everything works as expected.

  4. In the built Xcode project go to UnityVisionOSSettings.swift and change .windowStyle of bounded volume camera from .volumetric to .plain at line 39.
    Line 39 should then read:
    } defaultValue: { UUID() }.windowStyle(.plain).defaultSize(width: 1.000, height: 1.000, depth: 1.000, in: meters)

  5. In the built Xcode project go to Info.plist and change Preferred Default Scene Session Role from Immersive Space Application Session Role to Window Application Role

  6. Run in Simulator again and interact with buttons. You’ll notice that clicking on the right bottom edge of the left top button (Hide) will trigger the left bottom button (Show). Clicking on the left bottom edge of the right top button will trigger right bottom button.

  7. Move Window away from you (by holding shift while moving).

  8. Once you moved the window it works as expected again and you can’t reproduce behaviour of step 6.

Expected behaviour:

  • Changing in what mode Unity content will be rendered should not change the button’s collision.
  • Scaling the window view (e.g. by moving the window away from you) should not change the button’s collision.

Thank you for the report and the detailed repro. That will be very helpful in trying to determine what is going on here.

I can’t really detail what is going wrong here as what you are doing is not something that I’ve done or seen done here internally. This will take some investigation in order to determine root cause fix. I’m really confused by why moving your viewpoint would make this all work again as that doesn’t play any role in how we determine clipping or hit boxing. I would have expected that, if there was an issue, the issue would be independent of viewer movement.