Prevent Hand from occluding the UI in Mixed Reality Mode

Unity Engine
, ,
1

How do you have the hand not occlude the UI - ie UI is always rendered in front even if reality objects occlude

1 Like
2

If this is referring to upper limb occlusion, then we are working on a solution that we hope to have out soon though I have no ETA for that.

This discussion post is talking about the same thing and may have info you can use in your project: Upper Limb Visibility Setting - visionOS / Virtual Reality (Fully Immersive) Apps - Unity Discussions

3

wrist watch UI where the hand should not occlude UI

4

is there not some sort of shader depth mask thing that could work here?

5

Or to just have the UI render as the mixed reality version of an overlay

6

Hi @ina–

While the rendering tricks you’re describing would be viable in traditional rendering environments, we don’t have access to many of these features on visionOS – for example, we don’t have access to the depth buffer nor any ability to specify overlays or full-screen post processes. By design and for the sake of user privacy, passthrough and hands are composed entirely by the OS.

–Tim

7

Hi there! I did a little bit of testing today to nail down the specifics of this hand visibility stuff. The upperLimbVisibility(_: ) API can be used on any ImmersiveSpace. In terms of Unity components, this means that it will take effect if your VolumeCamera’s OutputConfiguration is set to Unbounded. You can read more about this in the VolumeCamera section of the docs.

If your VolumeCamera is set to Bounded, this API will have no effect, so you will always see your hands on top of content in a volume.

Our next package release will expose this as a setting within Unity in VisionOSSettings. If you want to try using it in the meantime, make sure you are testing a scene with an Unbounded VolumeCamera, and modify MainApp/UnityVisionOSSettings.swift in the generated Xcode project for your app. Depending on your project, these may be on different lines within that file, but for me on line 45, there is an ImmersiveSpace defined with the id: “Unbounded”. If you modify the line with the closing bracket for this immersive space (and any others) to look like this:
} defaultValue: { UUID() } .upperLimbVisibility(.hidden)
you should no longer see your hands on top of content. When the hands are set to .visible, the system does some amount of depth testing when you reach into a large object, but it is not sufficient for the types of effects you describe. I’m curious to know if using .upperLimbVisibility(.hidden) works for your use case.