Render depth to RenderTexture and calculate world position

I have setup a simple script which renders a camera to a RenderTexture though I am having difficulty creating a replacement shader which renders depth.

Essentially I need to:

  • Render depth data into a texture.
  • Given a pixel in screen space (x, y) determine the pixel position in world space (C#).

How might I go about doing this?

Ok you can make a script that creates a copy of your desired camera, and renders with a replacement shader
This particular script is attached to the camera, else you would need to get the camera by other means, here it can be accessed directly.

public class Testing : MonoBehaviour {

    public Texture2D someTex;
    public RenderTexture myDepth;
    GameObject tempCam;

    void Start()
    {
        someTex = new Texture2D(Screen.width, Screen.height);
        myDepth = new RenderTexture(Screen.width,Screen.height,8);
        tempCam = new GameObject("tempCam");
        tempCam.AddComponent<Camera>();
        tempCam.camera.CopyFrom(camera);
        tempCam.camera.enabled = false;
    }

    void Update()
    {
        tempCam.camera.SetReplacementShader(Shader.Find("Custom/ReplacementShader"), "");
        tempCam.camera.targetTexture = myDepth;
        tempCam.camera.Render();

        RenderTexture.active = myDepth;
        someTex.ReadPixels(new Rect(0, 0, Screen.width, Screen.height), 0, 0);
        someTex.Apply();
    }
}

Somthing like that, Leaving the replacement tag renders everything with the replacement shader.

A very simple shader to render depth can be as simple as forwarding your position down from the vertex shader into the fragment shader via TEXCOORD. and the color is the z value of the position divided by the far plane value of the camera which you pass in with the script.

Shader "Custom/ReplacementShader" {

	SubShader {
		Tags { "RenderType"="Opaque" }
		

		CGINCLUDE

		struct VIN {
			float4 pos : POSITION0;
		};

		struct V2F {
			float4 pos  : POSITION0;
			float4 p	:TEXCOORD0;
		};

		V2F myVert( VIN i ) {
			V2F o;
			o.pos = mul( UNITY_MATRIX_MVP, i.pos );
			o.p = mul( UNITY_MATRIX_MVP , i.pos );
			return o;
		}

		fixed4 myFrag(V2F i) : COLOR {
			// Here my farplane is 10, hardcoded, you can pass it like any other parameter from the unity script.
			return fixed4(i.p.z / 10 , i.p.z / 10 , i.p.z / 10 ,1);
		} 

		ENDCG

		pass {
			CGPROGRAM
			#pragma fragment myFrag
			#pragma vertex myVert
			#pragma fragmentoption ARB_precision_hint_fastest
			#pragma target 2.0
			ENDCG
		}
	} 
}

Then you should be able to use your imagination to use the data, and or optimize this code somehow to your liking.

Hope this helps somehow.