Correct setup for Unity UI

Hi there,

I’m trying to use the default Unity UI in a MR-App but cannot get simple button clicks to work. My current setup:
image

There are two buttons controlling the active state of the image. I can see the highlight on the buttons, but when clicking them nothing happens.

Is this a bug or am I doing something wrong here? Thanks for any help.

1 Like

Have you looked at the visionOS Template or imported the samples to see what is being done there for that? You may need to add a Poly Spatial Hover Effect component to the thing that you want to accept click input on device.

I assume that your canvas is in world space?

In the both the SpatialUI scene, as well as the Hub scene, there is no Canvas UI. There are Input Manager scripts that detect various components, and manually call .Press() on them. Is there something else we should be looking at in these sample scenes to understand how to get Canvas Input working?

I have tried manually detecting a IPointerHandler implementing monobehavior, but there are a few problems with that. One of which is children blocking the touch. For example if you have a placeholder text in an input field, it will block interacting with the input field. Another is that it seems to be inacurrate somehow? I have a vertical layout group of various elements, and depending on whether I simulate a tap more left or right on the top item, it may erroneously select the bottom item. If you would like me to make a more formal post to ask about this instead of continuing this thread, or provide the code I am using to trigger selections, let me know.

Yes my Canvas is in World Space and I don’t remember any canvas in the visionOS template project, but I will look into it again.

I’ll try to add the Hover component to the button. However, highlighting is working out of the box with the various interactive elements on the dev kit.

Ok so after some research I was able to get this to work correctly locally, though I’m on a pre-release version of the package.

That said, it should work OOB. I’m going to try to put together a sample that has this working and then I’ll share it as possible.

1 Like

That would be great! Thank you!

Here is a sample project. This is based on current 0.4.3 and Unity 2022.3.11f1.

NOTE: There is a bug in 0.4.3 that breaks this working in Play Mode. We are looking into this and hope to have a fix out in a later package release.

For anyone trying the project from @joejo you need to edit manifest.json before opening the project, just replace it with this:
(You may need to re-import packages after opening the project the first time as well)

{
  "dependencies": {
    "com.unity.collab-proxy": "2.1.0",
    "com.unity.ide.rider": "3.0.25",
    "com.unity.ide.visualstudio": "2.0.21",
    "com.unity.ide.vscode": "1.2.5",
    "com.unity.polyspatial.visionos": "0.4.3",
    "com.unity.polyspatial.xr": "0.4.3",
    "com.unity.render-pipelines.universal": "14.0.9",
    "com.unity.test-framework": "1.1.33",
    "com.unity.textmeshpro": "3.0.6",
    "com.unity.timeline": "1.7.5",
    "com.unity.ugui": "1.0.0",
    "com.unity.visualscripting": "1.9.1",
    "com.unity.xr.visionos": "0.4.3",
    "com.unity.modules.ai": "1.0.0",
    "com.unity.modules.androidjni": "1.0.0",
    "com.unity.modules.animation": "1.0.0",
    "com.unity.modules.assetbundle": "1.0.0",
    "com.unity.modules.audio": "1.0.0",
    "com.unity.modules.cloth": "1.0.0",
    "com.unity.modules.director": "1.0.0",
    "com.unity.modules.imageconversion": "1.0.0",
    "com.unity.modules.imgui": "1.0.0",
    "com.unity.modules.jsonserialize": "1.0.0",
    "com.unity.modules.particlesystem": "1.0.0",
    "com.unity.modules.physics": "1.0.0",
    "com.unity.modules.physics2d": "1.0.0",
    "com.unity.modules.screencapture": "1.0.0",
    "com.unity.modules.terrain": "1.0.0",
    "com.unity.modules.terrainphysics": "1.0.0",
    "com.unity.modules.tilemap": "1.0.0",
    "com.unity.modules.ui": "1.0.0",
    "com.unity.modules.uielements": "1.0.0",
    "com.unity.modules.umbra": "1.0.0",
    "com.unity.modules.unityanalytics": "1.0.0",
    "com.unity.modules.unitywebrequest": "1.0.0",
    "com.unity.modules.unitywebrequestassetbundle": "1.0.0",
    "com.unity.modules.unitywebrequestaudio": "1.0.0",
    "com.unity.modules.unitywebrequesttexture": "1.0.0",
    "com.unity.modules.unitywebrequestwww": "1.0.0",
    "com.unity.modules.vehicles": "1.0.0",
    "com.unity.modules.video": "1.0.0",
    "com.unity.modules.vr": "1.0.0",
    "com.unity.modules.wind": "1.0.0",
    "com.unity.modules.xr": "1.0.0"
  }
}

@joejo for me the button is highlighting when hovering, but it does not generate any events, and also does not show the pressed state in the Simulator. Is that to be expected with 0.4.3?

If using the same version of Unity and matching XCode, this project should work just fine in simulator. That means, other than the issue mentioned in Play Mode in Unity, you should be getting events just fine.

If using the same version of Unity you also should NOT have required any manual changes to the manifest file in any way to get this to open.

All said, though, I did not test this in earlier versions of Unity at all.

Can you confirm what version of Unity you are using?

I’m using PolySpatial 0.4.3 and Unity 2022.3.11f1. The project you shared uses 0.5.0 from an internal development server according to the manifest.

Oh, snap. I tested it with both (we’re testing next release right now) and must have compressed the wrong version. Let me get that updated/fixed.

Ok, project at link is fixed for 0.4.3, and tested to work in simulator. Sorry about the mix up.

2 Likes

So I was able to build and run the sample project in the simulator, and I saw the “<<< CLICKED >>>” log. I noticed the sample project was in Bounded mode, so I added an XR Origin via Right Click in Hierarchy → XR → XROrigin (AR), Swapped the Volume Camera to Unbounded, childed the Canvas to the camera at a distance of (0,0,1) for convenience, and rebuild/reran. This time there was no clicked log.

I thought maybe it was because of the hierarchy, so I made a script to move it in front of the camera instead. Still no log.

Is it possible this has not been tested in Unbounded mode or in a project build around using AR functionality?

That’s entirely possible and there may be an issue here I didn’t test against. Can you file a bug with this repro project you created and post the id here?

1 Like

So I ended up getting it to work in the various XR samples. I just needed to add similar EventSystem and InputSystems to the relevant scenes. The input system module is also off by default in the Project Launcher scene in the Sample project by default. Thanks for the help :smiley:

1 Like

I’m having a nightmare trying to get this to work in our own project. I’ve tried copying all the settings from the sample @joejo provided here, but I can’t get button clicks working. Are there any tips for debugging this stuff? I feel like I’m missing something tiny somewhere.

For what it’s worth, I’ve tested the sample using Unity 2022.3.12 and 0.5.0 poly spatial packages and the sample works with those upgrades, both via simulator and on device. As stated it doesn’t work in editor, but toggling the InputSystemUIInputModule component off and on at runtime fixes it.

I’ve figured out our problem and can reproduce the issue in the sample project. Basically, we have a canvas and volume camera that is much larger than the sample, and buttons are not clickable unless they are at (0, 0) on the canvas.

I’ve proven this by breaking the sample:

  • Change the volume camera to have dimensions of (650, 650, 650)
  • Adjust the canvas size to match (Scale 0.25, Width/Height: 2600)
  • Adjust the button size to width: 600, height: 150
  • Move the button to the bottom of the canvas

Run this in the simulator (or on device) - looking at the button will highlight it, but clicking it does not invoke the OnClick event handler.

I’ve submitted a bug report but haven’t got confirmation on it yet so I’ve uploaded the modified sample here

1 Like

I traced the issue to GraphicRaycaster.Raycast, when looping through foundGraphics, we hit this test:

if (!RectTransformUtility.RectangleContainsScreenPoint(graphic.rectTransform, pointerPosition, eventCamera, graphic.raycastPadding))
    continue;

and RectTransformUtility.RectangleContainsScreenPoint fails, causing us to skip over each graphic. pointerPosition is always (0,0) and eventCamera is always null.

Is there any update here @joejo I’ve made a super simple modification to the sample you provided in this thread where there is a button in the middle of the screen that is clickable, and a button at the bottom of the screen that is not clickable. Please find the project here

1 Like