Summary
On D3D11 / Built-in Render Pipeline / Unity 2022.3.22f1, I wrote a shader that reconstructs world position from the depth texture. It works correctly on Desktop, but in VR (Single Pass Stereo / SPS, running on VRChat), the reconstructed worldPos becomes distorted/twisted depending on distance, and the distortion is mirrored (opposite direction) between the left and right eyes.
Environment
- Unity 2022.3.22f1
- Graphics API: D3D11
- Render Pipeline: BRP
v2f vert (appdata v)
{
v2f o;
UNITY_SETUP_INSTANCE_ID(v);
UNITY_INITIALIZE_OUTPUT(v2f, o);
UNITY_INITIALIZE_VERTEX_OUTPUT_STEREO(o);
// Quad mesh: vertex.xy [-0.5,0.5] -> NDC [-1,1] directly
o.vertex = float4(v.vertex.x * 2.0, v.vertex.y * -2.0, 0.0, 1.0);
o.uv = v.uv;
o.ndc = o.uv * 2.0 - 1.0;
return o;
}
float4 frag (v2f i) : SV_Target
{
UNITY_SETUP_STEREO_EYE_INDEX_POST_VERTEX(i);
float2 uv = UnityStereoTransformScreenSpaceTex(i.uv);
float depth = SAMPLE_DEPTH_TEXTURE(_CameraDepthTexture, uv);
float linearDepth = LinearEyeDepth(depth);
float2 ndc = -i.ndc; // without this negation, the result is wrong even on Desktop
float4 clip = float4(ndc, 1.0, 1.0);
#if defined(USING_STEREO_MATRICES)
float4 view = mul(unity_StereoCameraInvProjection[unity_StereoEyeIndex], clip);
#else
float4 view = mul(unity_CameraInvProjection, clip);
#endif
float3 ray = normalize(view.xyz / view.w);
float3 viewPos = ray * (linearDepth / ray.z);
#if defined(USING_STEREO_MATRICES)
float3 worldPos = mul(unity_StereoCameraToWorld[unity_StereoEyeIndex], float4(viewPos, 1.0)).xyz;
#else
float3 worldPos = mul(unity_CameraToWorld, float4(viewPos, 1.0)).xyz;
#endif
// Visualize as a 1m grid
col = fixed4(step(0.97, frac(worldPos.x - 0.015)),
step(0.97, frac(worldPos.y - 0.015)),
step(0.97, frac(worldPos.z - 0.015)), 1.0);
}
Expected behavior
Using worldPos, draw a 1m grid (X axis → red, Y axis → green, Z axis → blue). On Desktop, the grid stays fixed in world space and renders correctly regardless of camera position/rotation.
Actual behavior
Desktop: Works as expected. The grid stays fixed in world space and is correct from any viewpoint (attached images 1, 2).
VR (SPS): The grid lines become radially distorted, with the distortion increasing with distance. The distortion is mirrored between the left and right eyes (attached images 3, 4).
Investigation so far
- Checked the compiled d3d11 disassembly:
-i.ndc, the stereo matrix array access viaunity_StereoEyeIndex, etc. all compile exactly as written and are consistent across keyword variants. This does not appear to be a compiler issue. - Regarding
-i.ndc(sign flip of clip.xy): Removing this flip breaks the result even on Desktop, so it appears to be required there. - Replacing
unity_StereoCameraToWorld[eye]withunity_StereoMatrixInvV[eye]: Comparing the compiled output, both handleviewPos.zthe same way (negating Z before applying the 3rd column), so this substitution doesn’t seem to make a meaningful difference. - Visualized
ray.xy(the output of InvProjection, after normalize): the left eye shows a skewed/diagonal gradient pattern, while the right eye shows a more axis-aligned (orthogonal) pattern — i.e. the two eyes produce asymmetric results.
