(IN-58042) visionOS - PolySpatial Volume To World shader incorrect if Volume Camera Output Dimensions are not 1,1,1

Continuing the discussion from visionOS Beta Release Notes (0.4.1):

It took a lot of digging, but I finally narrowed it down to being caused by output dimensions not being 1,1,1. The extra weird thing is that it seems to only affect the y-coordinate.

I have posted a bug report with this number: IN-58042

Left is correct | Right is incorrect

Yes, I see what you mean. We were already adjusting the output of RealityKit’s world position node (it was offset vertically by half the size of the output volume). Sounds like we have to adjust further now; I’ll try to find the right mapping.

Hello! I too have observed a peculiar phenomenon regarding the world position.

First, prepare two shaders as follows:

In the editor, both appear identical. However, in the simulator, objects passing through the split node appear more vivid. Is this issue encompassed in IN-58042?

It seems like there is some offset with output dimensions = 1,1,1 as well in our project. Not entirely sure why, but it seems like offsetting the result from PolySpatial Volume To World, by 0.125 * dimensions (not output dimensions) on the y-axis is close to correct.

In our case dimensions are set to 50,50,50 so offsetting the y by 6.25 is close to correct (but not 100% correct). It’s probably just random chance that this number works…

As mentioned previously the result from PolySpatial Volume To World was correct in version 0.3.3 as far as I have tested.

@Yugo_mydearest no that’s not covered by this bug. You probably need to add a “PolySpatial Volume To World” node after the “Position (World)” node, to get correct(ish) results on visionOS.

Thank you! I overlooked PolySpatialVolumeToWorld!
I reverted back to 0.3.3 for testing and indeed, I was able to confirm the difference in the behavior of WorldPosition between 0.3.3 and 0.4.3.
However, the behavior of WorldPosition still seems unnatural when passing through several nodes. I will investigate this issue further.
Continuing the discussion in this thread doesn’t seem appropriate, so if the investigation reveals any issues, I’ll start a new thread!
Thank you very much again!

1 Like

As far as I can tell, one issue here is that the offset we’re applying to compensate for visionOS’s shift of the world position is wrong if the volume is taller than 2. It appears that the offset is min(height/2, 1) rather than height/2. I’m not seeing any other offset, particularly not with an output size of (1, 1, 1).

If I update the example from IN-58042 by adding 0.5 (the difference between height/2 and 1) to the world position y before it enters PolySpatial Volume to World, I get the correct result (no difference between the two states) in visionOS (of course, then I get the wrong result in Unity play mode).

At any rate, I will make the change to the offset. Since you’re seeing additional issues, maybe we can reproduce them better if you submit a further repro case.

Further details:

It looks like the issue with the additional offset with Output Dimensions = 1,1,1 only appears on device, in the simulator it is not present.

To test yourself:

  • Open the repro project.
  • Set Volume Camera Output Dimensions to 1,1,1
  • Set Target SDK to Device in Player Settings.
  • Build and run on Device.
  • Observe the difference between World Space position and Object Space position.

The plot thickens! I just tested our project on a different device, that did not exhibit the additional offset. I will check device SDKs tomorrow.

1 Like

Both use the same SDK: visionOS 1.0 (21N5259k)

Figured it out! If you modify “Settings > Display > Appearance” to anything other than “Large” you will get an offset on the y-axis. This applies in Simulator as well.

You can test easily with the original repro project, and setting output dimensions to 1,1,1. Open “Home > Settings > Display > Appearance” and modify the setting while the app is running in the simulator.

I guess you (or Apple) are taking the “close / move bar” into account when computing the matrices.

It ain’t us! Thanks for tracking this down; I’ll make a note to look into it.

Unfortunately, I wasn’t able to find a way to detect this setting in order to compensate for it. I submitted a bug report to Apple about it (to accompany our report about the y being shifted up in general).

FYI: The offsets have changed in visionOS beta 6. It is no longer vertically offset by half the volume size, and the z-axis is now also offset compared to previous versions.

In our setup adding VolumeCamera.transform.rotation * (0,25,25) to the value output by “PolySpatial Volume To World” produces correct results in beta 6. Note: Our volume dimensions are set to 50,50,50 and we only rotate the volume around the y-axis, so it might not be correct for arbitrary rotations.

(Additional offset when Appearance != Large is still present)

Thanks for the heads-up. It looks like it’s the same calculation, just using z instead of y (so, instead of subtracting min(size.y * 0.5, 1.0) from y, we have to add min(size.z * 0.5, 1.0) to z). We’ll add that fix to an upcoming release.

Yeah; we raised an issue for this with Apple, but haven’t received any updates or suggestions.

A new error has been introduced on 0.7.1. We now have to adjust the output from Volume To World by adding something close to:

offset = VolumeCamera.rotation * new Vector3(0,0,1.5f / 0.9411765f);

to get correct(ish) results in our shaders.

The 0.9411765f is taken from the volume size as reported here:

Volume 13292: window assigned, uuid: 8CD225DF-21A4-4C4C-ADAC-533230F45D57 outputDimensions: SIMD3<Float>(1.0, 1.0, 1.0) actualDimensions: SIMD3<Float>(0.9411765, 0.9411765, 0.9411765)

It would be greatly appreciated if the Volume To World node was tested properly before updates, as we very much rely upon it generating correct results.

(As a side note this also causes the PolySpatial Lighting Node to generate incorrect results.)

Sorry, I haven’t been able to replicate this in the simulator so far. It would help if you could submit a bug report with a repro case and let me know the incident number (IN-#####).

I’ll put something together next week. It might be dependent on the Dimensions of the Volume Camera. And it looks like it could potentially be an OS/layout issue, as the offset can change if you run the eye calibration while having the app open.

1 Like