Hello !
We are making a custom terrain engine using Unity for a school project, and have started using geometry shaders to display trees and grass as flat billboards.
After some research we found out how to make Unity cast shadows on a custom shader without a surface shader (since we have a vertex / geometry / fragment shader setup for both trees and grass), thanks to this post : link text
This post was also really helpful to get the shadow caster to work with a geometry shader : link text
So what we have is a normal pass to render the tree or grass billboard, and then a second pass (ShadowCaster pass) that does the same thing but with V2F_SHADOW_CASTER; in our structs and SHADOW_CASTER_FRAGMENT(input) in the fragment shader instead of returning a color. This works : it makes the trees cast a shadow that the ground can receive.
However what we can’t figure out is how to get a Shadow Collector pass to work with our geometry shaders, and after a LOT of research it seems that no one has ever tried this or posted about it. What we’d like to do is have the grass receive shadows from trees and the terrain (to have great-looking woods !)
Simply adding a third pass exactly like what the code in the link does doesn’t work, and we can’t get a third pass to work doing the same thing as the normal pass (with geometry shader and everything) combined with the V2F_SHADOW_COLLECTOR; in structs and TRANSFER_SHADOW_COLLECTOR() in the shader programs.
This is the shadow collector pass we are trying to get to work :
// Pass to render grass as a shadow collector
Pass {
Name "ShadowCollector"
Tags { "Queue" = "Geometry" "RenderType"= "Overlay" "LightMode" = "ShadowCollector"}
Fog {Mode Off}
LOD 200
ZWrite On ZTest LEqual Cull Off
Offset 1, 1
CGPROGRAM
#pragma target 5.0
#pragma vertex COLLECTOR_VS_Main
#pragma fragment COLLECTOR_FS_Main
#pragma geometry COLLECTOR_GS_Main
#pragma multi_compile_shadowcollector
#pragma fragmentoption ARB_precision_hint_fastest
#define SHADOW_COLLECTOR_PASS
#include "UnityCG.cginc"
// **************************************************************
// Vars *
// **************************************************************
float _SizeH;
float _SizeL;
float growUp;
float4x4 _VP;
sampler2D _SpriteTex1;
uniform float maxDist;
uniform float3 camPos;
uniform float4 _LightColor0;
// **************************************************************
// Data structures *
// **************************************************************
struct COLLECTOR_GS_INPUT
{
V2F_SHADOW_COLLECTOR;
...other stuff related to size and rotation of the grass
};
struct COLLECTOR_FS_INPUT
{
V2F_SHADOW_COLLECTOR;
float2 tex1 : TEXCOORD9;
};
// **************************************************************
// Shader Programs *
// **************************************************************
// Vertex Shader ------------------------------------------------
COLLECTOR_GS_INPUT COLLECTOR_VS_Main(appdata_base v)
{
COLLECTOR_GS_INPUT output = (COLLECTOR_GS_INPUT)0;
TRANSFER_SHADOW_COLLECTOR(output)
...other stuff
return output;
}
// Geometry Shader -----------------------------------------------------
[maxvertexcount(4)]
void COLLECTOR_GS_Main(point COLLECTOR_GS_INPUT p[1], inout TriangleStream<COLLECTOR_FS_INPUT> triStream)
{
stuff to create the four vertices of the quad for the billboard :
float3 up = float3(0, 1, 0);
float3 rot = float3(p[0].rotX, 0, p[0].rotY);
rot = normalize(rot);
float3 right = cross(up, rot);
_SizeH = p[0].size;
_SizeL = p[0].size;
float halfSy = 0.5f * _SizeH;
float halfSx = 0.5f * _SizeL;
float4 v[4];
p[0].pos.y -= growUp;
v[0] = float4(p[0].pos + halfSx * right, 1.0f);
v[1] = float4(p[0].pos + halfSx * right + _SizeH * up, 1.0f);
v[2] = float4(p[0].pos - halfSx * right, 1.0f);
v[3] = float4(p[0].pos - halfSx * right + _SizeH * up, 1.0f);
float4x4 vp = mul(UNITY_MATRIX_MVP, _World2Object);
...the interesting bit :
COLLECTOR_FS_INPUT pIn;
UNITY_INITIALIZE_OUTPUT(COLLECTOR_FS_INPUT, pIn);
TRANSFER_SHADOW_COLLECTOR(pIn)
pIn.pos = mul(vp, v[0]);
pIn.tex1 = float2(1.0f, 0.0f);
triStream.Append(pIn);
pIn.pos = mul(vp, v[1]);
pIn.tex1 = float2(1.0f, 1.0f);
triStream.Append(pIn);
pIn.pos = mul(vp, v[2]);
pIn.tex1 = float2(0.0f, 0.0f);
triStream.Append(pIn);
pIn.pos = mul(vp, v[3]);
pIn.tex1 = float2(0.0f, 1.0f);
triStream.Append(pIn);
}
// Fragment Shader -----------------------------------------------
fixed4 COLLECTOR_FS_Main(COLLECTOR_FS_INPUT input) : COLOR
{
SHADOW_COLLECTOR_FRAGMENT(input)
}
ENDCG
}
With this, we don’t get any immediate error when launching the scene but when grass starts to appear (it seems that weirdly the shader only compiles at runtime) we get this error on the TRANSFER_SHADOW_COLLECTOR(pIn) (at line 117 in the code sample) in the geometry shader :
Shader error in 'Grass/GrassRendering ': invalid subscript 'vertex' 'mul': no matching 2 parameter intrinsic function; Possible intrinsic functions are: mul(float|half|double|min10float|min16float|int|uint|min12int|min16int|min16uint, float|half|double|min10float|min16float|int|uint|min12int|min16int|min16uint) mul(float|half|double|min10float|min16float|int|uint|min12int|min16int|min16uint, floatK|halfK|doubleK|min10floatK|min16floatK|intK|uintK|min12intK|min16intK|min16uintK) mul(float|half|double|min10float|min16float|int|uint|min12int|min16int|min16uint, floatLxK|halfLxK|doubleLxK|min10floatLxK|min16floatLxK|intLxK|uintLxK|min12intLxK|min16intLxK|min16uintLxK) mul(floatM|halfM|doubleM|min10floatM|min16floatM|intM|uintM|min12intM|min16intM|min16uintM, float|half|double|min10float|min16float|int|uint|min12int|min16int|min16uint) mul(floatM|halfM|doubleM|min10floatM|min16floatM|intM|uintM|min12intM|min16intM|min16uintM, floatM|halfM|doubleM|min10floatM|min16floatM|intM|uintM|min12intM|min16intM|min16uintM)
Being still quite new to shaders, we can’t understand what the error means and what can be done about it… Also these are Unity functions that don’t seem to be documented anywhere, did we miss it maybe ?
So our question is, how would one do to receive shadows on geometry shaded billboards ? And if this is the correct way to do it, can someone help us understandind why it doesn’t work ?
Thanks a lot guys, I know this is a long post and I’d very grateful towards anyone trying to help 