Writing depth value in fragment program

tomaszek · November 5, 2010, 4:35pm

Hi,

I’ve got problem with writing depth in fragment program. All fragment programs I saw for ShaderLab write only color, but it seems to be possible to output depth too (as I see in HLSL/Cg forums). I’ve been digging the forum for hours but I still got no idea how to do it. Was trying shader from attachment, but compiler throws such error:

Shader error in ‘TestShader’: D3D shader assembly failed with: (8): error X2022: scalar registers cannot be masked
Shader Assembly: ps_2_0
; 4 ALU
def c0, 0.00000000, 0, 0, 0
dcl t0
frc r1, t0
mov r0.z, c0.x

Any idea how to make it working ?

422416–14672–$TestShader.shader (893 Bytes)

Dreamora · November 5, 2010, 4:49pm

The texture you write to in the fragment program is only a color buffer so writing depth will not work I guess.
Unity 3 renders the depth and normal buffer in a different pass (given you enable the camera to render a depth texture) than the color, but you can potentially access this one.

tomaszek · November 7, 2010, 5:28pm

Thank you, fast and competent reply as usual by dreamora :).

I’ve just analysed compiled version of simple surface shader. Indeed - as far as I see there is only color buffer written. I will have to sort things another way aquiring information from depth texture. The problem is - I want this shader (grass shader) to work on terrains which are usually queued as “geometry-100”. So I’d need to write my material later as it will be alpha masked “transparent”. Are there any known drawbacks when shifting terrain material later in render queue ? Dangerous side effects to be aware of ?

Dreamora · November 7, 2010, 5:38pm

likely totally non working depth sorting actually.
It already runs at its edge of usability as you might find out with some PostFX actually.

duke_1 · August 16, 2011, 5:15am

Has this been addressed in 3.4?

Noisecrime · April 16, 2012, 1:16am

Just a note for those wanting to write to depth in D3D9 shader and getting the ‘D3D shader assembly failed with: (36): error X2022: scalar registers cannot be masked’ error.

Although this bug still remains, the problem is with the cg compiler, something that has been known since 2008 (according to this old cg thread on nvidia forums). So either its never been fixed or Unity is using a pretty old version of cg?

Thankfully it appears from initial testing this issue can be solved by changing the compiled shader code from

mov oDepth.z, r0.z

to

mov oDepth, r0.z

I.e. removing the .z mask, which since its on a scalar doesn’t work and hence the error.

To change the compiled code simply create your shader normally, ignoring the error. Once finished select the shader and click ‘open compiled shader’ button in the inspector. Now copy the entire contents of the compiled shader and paste it into a new shader file. Look through the code and replace the lines causing the bug as above, then save the shader.

It does mean that your original shaderlab/cg shader will never compile or be usable in d3d9 environment so don’t use it or include it in your builds! However your fixed ‘compiled’ version should work fine. At least it does for me in the editor and as a window build.

tomaszek · April 16, 2012, 9:32am

Good to know it’s possible. Works in OpenGL/GLSL and D3D9 after NoiseCrime’s fix, but still there are suspicious messages in compiled code:

for mobile profiles:

/* NOTE: GLSL optimization failed
0:32(15): error: `gl_FragDepth’ undeclared
0:32(15): error: type mismatch
*/

so can’t say if works for mobile

error about flash export incompatibility

Shader warning in ‘TestShader’: register type 9 not supported in AGAL (compiling for flash) at line 5

Anyway - it’s big step further and could be useful, however I dropped the idea using this feature completelt for performance reasons (writing into depth buffer is told to be slow, but honestly I’ve never tested it since I couldn’t do it well in Unity).

Noisecrime · April 16, 2012, 3:04pm

Good point about Mobile ( where writing to depth buffer is not supported as far as I understand) and Flash.

As for performance, seems fine to me (Windows). I guess it could be slower than normal rendering since the gpu has no idea what depth you are going to set, so normal optimisations (e.g. early outs) can’t be done. However its not likely that you’d use this frequently for many objects, more likely as in my case, you’d want to use it once per frame to populate the depth buffer with specific values.

Aras · May 8, 2012, 6:19am

We can’t do much about mobile here. In OpenGL ES 2.0, writing to the fragment depth is not possible. There is just no way to do it.

Again, not much we can do. In Stage3D’s AGAL shader language, writing to fragment depth is not possible.

To not produce any warnings, add “#pragma exclude_renderers gles flash” to the shader code.

Elecman · July 23, 2013, 4:36am

The "X2022: scalar registers cannot be masked’ bug is fixed in Unity 4.2. The shader should be like this by the way:

Shader "TestShader" {
SubShader {
    Pass {
        Fog { Mode Off }
		CGPROGRAM        
			#pragma exclude_renderers gles flash
			#pragma vertex vert
			#pragma fragment frag
			
			// vertex input: position, UV
			struct appdata {
			    float4 vertex : POSITION;
			    float4 texcoord : TEXCOORD0;
			};
			
			struct v2f {
			    float4 pos : POSITION;
			    float4 uv : TEXCOORD0;
			};
            
			v2f vert (appdata v) {
			    v2f o;
			    o.pos = mul( UNITY_MATRIX_MVP, v.vertex );
			    o.uv = float4( v.texcoord.xy, 0, 0 );
			    return o;
			}
            
			struct C2E2f_Output {
				float4 col:COLOR;
				float dep:smile:EPTH;
			};		
            
			C2E2f_Output frag( v2f i ) {
            
				C2E2f_Output o;
			    half4 c = frac( i.uv );
			    o.col=c; // this is not iteresting
			    o.dep=0; // this is what I want to output into Z-buffer (0 value is just an example)
			    return o;
			}
		ENDCG
		}
	}
}

BenC · July 24, 2013, 11:40am

Has anyone used this successfully in DX11?

Depth output from a pixel shader works fine for me in DX9 mode (after upgrading to 4.2 to fix the mask issue described above), but if I put the renderer into DX11 mode the depth output seems to be ignored - I just get the regular geometry depth…

KingOfColly · December 8, 2013, 11:54am

Anyone made any progress with this?

I’m having the same problem with my shader when trying to implement a logarithmic depth buffer (‘Use Direct3D 11’ enabled in Player settings). Using the log calc gives the same output as depth = 0 and depth = 1. Like it ignores the value I set (or more likely I’m doing it wrong!). This is on PC by the way (Win7 x64)

Shader "Custom/LogDepth"
{
	Properties
	{
		_MainTex ("Base (RGB)", 2D) = "white" {}
	}
	
	SubShader
	{
		Pass
		{
			Tags { "RenderType"="Opaque" }
			
			CGPROGRAM
	        #pragma target 4.0
	        #pragma vertex vert
	        #pragma fragment frag
	        #include "UnityCG.cginc"
	            
			sampler2D _MainTex;
			float4 _MainTex_ST;
			float c = 1.0;
			float far = 1000000.0;
			
			struct v2f
			{
				float4 position : POSITION;
				float2 uv : TEXCOORD0;
			};
			
			struct fragOut
			{
				half4 color : COLOR;
				float depth : DEPTH;
			};
					
			v2f vert(appdata_base v)
			{
				v2f o;
				
				o.position = mul(UNITY_MATRIX_MVP, v.vertex);
				o.uv = v.texcoord;
				
				return o;
			}
	
			fragOut frag(in v2f i)
			{	
				fragOut o;
			
				o.color = tex2D(_MainTex, i.uv);
				o.depth = (log(c * i.position.z + 1) / log(c * far + 1) * i.position.w);
				//o.depth = 0;
				//o.depth = 1;
				
				return o;
			}
			ENDCG
		}
	} 
	
	FallBack "Diffuse"
}

tomaszek · May 4, 2015, 7:36pm

Finally I found the solution - simply we can’t use old semantics with DX11 (DEPTH) - we need to use (SV_Depth as pixel function output and SV_Target instead of COLOR, in case of new deferred we’ve got indexed SV_Target0 to SV_Target3 semantics for MRT output).

Tom

antislash · May 28, 2015, 11:52am

Hi Tom,
could you quickly give an example please ? new to shaders and a bit confused.
tried this and couldn’t make it work, nothing in zbuffer.
thanks a lot
Victor

         float4 _MainTex_ST;
        float4 _Color;
        sampler2D _MainTex;
        half _Dist;
        float c = 1.0;
        float far = 1000000.0;

    
         struct v2f {
              float4 pos : SV_POSITION;
              fixed4 color : COLOR;
              float3 norm : NORMAL;
              float2 uv : TEXCOORD0;
           };
  
        struct fragOut
        {
            half4 color : SV_Target ;
            float depth : SV_Depth ;
        };

          v2f vert (appdata_full v)
          {
              v2f o;
            o.norm = v.normal.xyz;
            o.pos = mul (UNITY_MATRIX_MVP, v.vertex );
            o.uv = TRANSFORM_TEX(v.texcoord.xy,_MainTex);
            //assign color and alpha
            o.color.xyz = _Color.xyz;
            o.color.w = 1;
            return o;
        }
    
        fragOut frag (in v2f i)
        {
            fragOut o;
            fixed4 tex = _Color * tex2D (_MainTex, i.uv);
            o.color = tex;
            o.depth = (log(c * i.pos.z + 1) / log(c * far + 1) * i.pos.w);
            o.color.rgb=  o.depth; //
            return o;
          }

tomaszek · May 28, 2015, 11:55am

Something like this:

struct fragOut
{
   half4 color : SV_Target;
   float depth : SV_Depth;
};

Tom

antislash · May 28, 2015, 12:12pm

still have nothing…well…thanks anyway

tomaszek · May 28, 2015, 12:38pm

I don’t know if above example still works, anyway, as a hint:

    float depthWithOffset = i.eyeDepth*(1+rayLength/distance(i.posWorld.xyz, _WorldSpaceCameraPos)); // Z-DEPTH perspective correction
     o.depth = (1.0 - depthWithOffset * _ZBufferParams.w) / (depthWithOffset * _ZBufferParams.z);

in vertex function use COMPUTE_EYEDEPTH(vertexFunctionOutputStruct.eyeDepth);

the eyeDepth value is passed to fragment function as i.eyeDepth above. You also need to pass world position (posWorld) to frag function. With rayLength variable you can offset resultant depth back and forth in world units. When rayDepth=0 we’ve got not modified depth. Such formula works fine in my shaders, although determining rayDepth (depth offset in world units) may be very tricky when you try to make computations in, for example, tangent space.

Tom

UnityGuillaume · May 28, 2015, 2:47pm

Just a note, if I’m not mistaken SV_Depth exist ONLY in D3D10 and higher.

Also, there is some limitation with multisampling, see Semantics - Win32 apps | Microsoft Learn

I don’t know how Unity handle it for platform don’t handling it, wouldn’t be surprise it discard the value entirely.

antislash · May 28, 2015, 2:48pm

hi Tom
your hint looks fine for perspective correction( wich i was also looking for, i’m thankful)
i’m really noobish, but what i don’t get is where and how do you write to zbuffer.
i will use your code as i am trying fo get a very simple fog working (thus persp correction);
what i would really love is write a damn depth to the z buffer so i could write my semi-transparent shader pixels to zbuffer.
i think i must have a look at unity foliage code or so.
thanks very much
victor

antislash · May 28, 2015, 2:50pm

i wish a unity god of code could provide a simple example of a shader that whrites what i want to zbuffer

Topic		Replies	Views
Writing to depth buffer with custom shader Unity Engine Shaders	31	33434	May 26, 2020
Write to Depth from Fragment Unity Engine Shaders	6	6227	January 20, 2014
Rendering depths into render textures in both OpenGL and Direct3D Unity Engine Shaders	3	4299	September 5, 2017
Accessing depth buffer from a surface shader. Unity Engine Shaders	19	31544	October 16, 2019
modifying Z-buffer depth Unity Engine Shaders	10	26893	October 17, 2013

Writing depth value in fragment program

Related topics