(IN-62026) InteractionPosition scaling?

[mikkel_lbs]

We’re seeing input being scaled with some factor in our project.

Basically using

startInteractionPosition + (devicePosition - startDevicePosition)

Gives 1-to-1 input while using

interactionPosition

Is scaled and causes drifting. Is this a known issue, or should I create a repro project?

[mikkel_lbs]

It looks like it is scaled depending on the distance of the start interaction point from the viewpoint. Not sure if this is something visionOS does or it happens on the Unity side?

[mikkel_lbs]

Very poor quality GIF showing the scaling … Red dot is interactionPosition, Green is corrected position, Blue is devicePosition

Reported here: IN-62026 - visionOS - SpatialPointerState.interactionPosition is scaled and doesn’t match hand movement

[orbosphere]

I’m struggling with this a bit. Where are you finding ‘startDevicePosition’ that you use to calculate the corrected 1:1 input?

I’m attempting to detect a tap, identify the interaction position on a plane (the plane has a box collider), and then spawn a click effect prefab right at the interaction point. However, I’m finding I can’t accurately identify that interaction point. When I tap (in visionOS simulator) at a point such as (1, 0, 1), the interaction position seems to be slightly offset from the touch point.

I’m using the following code to get the touch data:

EnhancedSpatialPointerSupport.GetPointerState(activeTouches[0]);

When I tap at (1, 0, 1) and then check the interactionPosition from that touchData, I get the following:

(44.84, 15.03, 44.46)

At first I was really confused over this, but then I considered that maybe the scale of my volume camera is a factor. My Volume Camera dimensions are 30, 30, 30. So I tried dividing the interaction position x, y and z values by that scale factor. After doing that, I get:

(1.49, .50, 1.48)

This is much closer to the anticipated touch point of (1, 0, 1), but it appears to be offset by about 0.5 in all directions, and I’m not sure why.

I’m also finding that the further I look away on the plane and perform a tap, the greater the offset is. So there seems to be some link between my eye position, or possibly the “device position”? So I can’t simply subtract (0.5, 0.5, 0.5) from the interaction position to fix the issue… the offset seems to scale based on something, but I haven’t figured out what.

This post is the closest thing I have come across on these forums to this issue that I’m having. And I think what I’m chasing is the green interaction point from the gif that was posted. But I’m struggling to figure out the proper way to calculate that point.

Can anyone offer any insight on what I’m running into here? Or how to properly identify an interaction point such as (1, 0 , 1) when I’m confident I’m tapping at that location on a plane?

Attached is a photo that shows my mouse cursor at (1, 0, 1) performing a tap in the simulator along with the green tap effect that I’m spawning at the interaction position, after scaling it down by my volume camera’s dimension size. As you can see, the result is a click effect that is slightly offset from my actual click location. The second photo shows stopping at a breakpoint to display the contents of the touch data.

Thanks

Edit: I’ll also note that I’m using PolySpatial 1.1.4

[mikkel_lbs]

Hi,

We simply store the startDevicePosition when the interaction starts, ie. on TouchPhase.Begin… So something like:

var state = EnhancedSpatialPointerSupport.GetPointerState(activeTouches[0]);
if (state.phase == TouchPhase.Begin)
{
  startDevicePosition = state.devicePosition;
}

Regarding your other issues, this is not something we have seen. The initial hit position will be in startInteractionPosition. If this position does not match your expectations, you most likely have a collider that is being hit. You can enable the collider debug overlay in XCode to check on device/simulator. See: Diagnosing issues in the appearance of a running app | Apple Developer Documentation

It’s also a good idea to have a few debug spheres always tracking the interactionPosition and devicePosition, so you can see where the hit is registered.

Best of luck finding the issue

[orbosphere]

Thanks! After your suggestion to check collider debug overlay, I generated a build from Unity for visionOS simulator. After opening that in Xcode, and building to the simulator from there, I found that I no longer had any scaling issue, and the interaction point was aligning exactly as expected. So I went back to Unity, generated a build for the real visionOS, and built that from Xcode to the AVP. Success! Interactions are working normally.

I guess the issue I was running into had something to do with the Play To Device(Simulator). When I run my project from Xcode, interaction position is working as expected.

[vcheung-unity]

Hi,

Sorry for the trouble! There was a regression in PlayToDevice in regards to input - an offset was being applied when a connecting app had a VolumeCamera with dimensions that differed from 1:1:1. A point release to fix it will be released soon.

[orbosphere]

Understood, sounds like that aligns with what I observed. Thanks for the info!