Having problems reconstructing position from depth buffer

Unity Engine
1

Hi everyone, I’m trying to implement screen based reflections, but I can’t get past the initial requirement of going from screen space to view space. Here’s what I’m actually doing now, keep in mind that I’m using this as a post processing effect using Blit:

v2f vert (appdata v)
{
                v2f o;
                o.vertex = UnityObjectToClipPos(v.vertex);
                o.uv = v.uv;
                //camera ray is a vector3 that points from  the camera to the far plane
                o.cameraRay = mul(_CameraInverseProjectionMatrix, float3((float2(v.uv.x, 1-v.uv.y) - 0.5) * 2, 1)).xyz;
                return o;
}

fixed3 frag (v2f i) : SV_Target
{
                float3 decodedNormal;
                float decodedDepth;
                DecodeDepthNormal( tex2D( _CameraDepthNormalsTexture, float2(i.uv.x, 1 - i.uv.y)), decodedDepth, decodedNormal);

                // Here I should now have the position that I'm looking for in view space
                float3 view_pos = i.cameraRay * decodedDepth;
               // and if that is the case, then projecting it again should display the same image than the camera without this transformations...
                float4 projected_view_pos = mul(UNITY_MATRIX_P, float4(view_pos ,1));
                float3 screen_point = projected_view_pos.xyz / projected_view_pos.w;
              return tex2D(_MainTex, float2(screen_point.x,1- screen_point.y) * 0.5 + 0.5);
}

And the result is…almost correct:

2865177--209816--Screenshot_1.png
2865177--209816--Screenshot_1.png858×486 45.7 KB

And if I show the screen positions like this:

return fixed3(float2(screen_point.x,1- screen_point.y) * 0.5 + 0.5, 0);

This is the result:

2865177--209817--Screenshot_2.png
2865177--209817--Screenshot_2.png827×478 37.8 KB

Now that doesn’t look correct at all.

I’ve read everything I could find but I can’t understand what I’m doing wrong. Also here I leave the normals and depth if that may be helpful:
2865177--209818--Screenshot_3.png 2865177--209819--Screenshot_4.png

Also I forgot to tell you that I’m setting the inverse projection matrix with:

mat.SetMatrix(“_CameraInverseProjectionMatrix”,Camera.main.projectionMatrix.inverse);

2

There are several things wrong.
1.

Camera.main.projectionMatrix is not the final projection matrix used by the GPU. Actual projection matrix vary between platforms. You could use GL.GetGPUProjectionMatrix but there is actually no need for it. There’s a built-in shader variable “unity_CameraInvProjection”.

o.cameraRay = mul(_CameraInverseProjectionMatrix, float3((float2(v.uv.x, 1-v.uv.y) - 0.5) * 2, 1)).xyz;

You are trying to transform a point from 4D homogeneous clip space into 3D view space. You can’t just ignore the fourth dimension. Input point and cameraRay must be 4D vectors:

o.cameraRay = mul(unity_CameraInvProjection, float4((float2(v.uv.x, v.uv.y) - 0.5) * 2, 1, 1));

float3 view_pos = i.cameraRay * decodedDepth;

I think decodedDepth is a raw depth. You need the depth in 0…1 range. Use Linear01Depth to convert it to correct range. cameraRay is now in homogeneous space, so you have to convert it to 3d space.

decodedDepth = Linear01Depth(decodedDepth);
float3 view_pos = i.cameraRay.xyz / i.cameraRay.w * decodedDepth;

float4 projected_view_pos = mul(UNITY_MATRIX_P, float4(view_pos ,1));

I’m not sure what UNITY_MATRIX_P is, but it has different value than “unity_CameraProjection”. So, I would just use unity_CameraProjection.

And finally.

return tex2D(_MainTex, float2(screen_point.x,1- screen_point.y) * 0.5 + 0.5);

This makes little sense. screen_point.xy is in clip space. In range from -1 to 1. So, 1 - screen_point.y brings it to range from 0 to 2 and the final range of screen_point.y is from 0.5 to 1.5. That’s hardly what you want. If you want to invert the y axis you need to bring screen_space.y to 0…1 range first.

3

Thanks a lot for your time! now it’s working correctly, and I have a question:

What use does the Camera.main.projectionMatrix have?

4

You’re welcome.

You can use it for CPU only calculations. You just have to know that Unity is using OpenGL convetions. For example, you can transform screen position (say, mouse position) into world space ray to pick 3d objects.
If you want to use it in shader, you have to use Camera.projectionMatrix together with GL.GetGPUProjectionMatrix to get correct matrix for given platform and render target (different matrix for back buffer and different for render texture on some platforms).
You can also use Camera.projectionMatrix to set your own custom projection.

5

this guy:
https://forum.unity3d.com/threads/screen-space-local-reflection.205549/#post-1394571
found out what is going on with UNITY_MATRIX_P

I hope it helps someone :slight_smile: