What I’m trying to do should be pretty simple: a decal shader that overlays it’s own normals on top of normals already existing in the deferred GBuffer, instead of overwriting everything on pixels covered by decal faces. It was said to me that what I want to accomplish can be done through use of finalgbuffer modifier.
Full disclosure: I’m pretty bad at writing shaders and I only attempt to use a finalgbuffer because a senior shader programmer with Unity 5.2 beta access recommended it to me as a perfect tool to create the shader I describe. 
First, some visual examples, for context.
Let’s say you have two models like these:
The one on the right is textured with this normal map:
The end result, with the decal adding it’s normals on top, looks like this:
Now, from what I understand, the shader should work like this:
-
Add finalgbuffer modifier to the surface shader and create a function for it
-
Declare sampler2D for _CameraGBufferTexture2
-
Calculate screenUV in the finalgbuffer function
-
Use tex2D with screenUV on _CameraGBufferTexture2 in the finalgbuffer function
-
Transform your tangent space normals to world space (as GBuffer is storing world space normals)
-
Overlay your transformed normal onto sampled _CameraGBufferTexture2
-
Output the result to inout half4 normal in the finalgbuffer function
Except I have no idea how to make it work, because the surface part of the shader ruins the result by overwriting everything, and I can’t discard the result of a surface shader using e.g. Blend Zero One because that will also discard anything I will do in the finalgbuffer function.
One of the ideas I have is using screenUV and sampling all four GBuffer textures in the surface function to output that to SurfaceOutputStandard struct, but that is not working either, unfortunately. I am pretty sure I am completely misunderstanding the point of finalcolor/finalgbuffer if I do that too, I think, because if it was possible to read form GBuffer and write to it at the same time, separate functions wouldn’t have been necessary at all.
The only reference I have, and pretty much the only instance of finalgbuffer use in existence so far is Terrain splat shaders in Unity. If someone can take a look at them, that would be nice, because I’m not sure I’m understanding how finalgbuffer is used there correctly.
Maybe one missing piece of the puzzle is use of decal:add or decal:blend modifiers, but I’m not sure.
Can someone help? If you have no experience with finalgbuffer, that’s alright - some of you are probably familiar with finalcolor modifier, which, according to Unity, works almost exactly the same. Maybe you can at least show me how to make a shader with a similar idea but using finalcolor (sans normal operations, just e.g. overlay of a red fill).
P.S.: I have encountered these questions a dozen of times now, so just in case:
Q: Those models look awful, why not just use a single unwrapped object with baked normals?
A: Few reasons:
- Makes it possible to detail objects with huge surface area. Not even insanely high-res unwraps will save you if you want to put crisp detail on a wall or a vehicle with surface area of, say, 30 meters.
- Absolutely perfectly consistent texel density - every single detail in every single object in every single environment will have exactly the same texel density. Great for the look.
- Ease of authoring art - slapping quads onto models is the definition of a fast workflow. An artist can detail ten objects with that approach in a time it will take him to set up highpoly and bake maps for one traditional object.
- Ease of updating art - modify the normal map decal to, say, alter a shape of a rivet, get every single rivet in existence updated on thousands of objects. Good luck doing that with traditional floater baking workflow.
- Free memory - huge per-object normal maps are no longer needed if you are using med-poly base geometry with normal decals on top. Just one small decal atlas per all objects and you’re done. Hell, you might not even need other per-object atlases - just jump to tiled materials and add all your detail with decals.
This workflow is not some obscure fantasy, it was used for most of the modular objects in Alien Isolation and is being used in most ship and level models for Star Citizen. There are existing shaders accomplishing what I’m asking about on UE4 and CE.
Q: What’s wrong with alpha blended shaders? Just make a texture where normal mapped parts are opaque.
A: That won’t give the desired result. I want to overlay my normals over existing normals.
If a surface has grain or bumps in the normal, decal has to preserve that instead of putting decal face normal with tangent space perturbations applied onto it. More than that, even if the underlying surface lacks any normal mapping - overlaying onto GBuffer is still desirable, because geometry and normals of decal faces might not precisely follow the geometry and normals of the underlying surface (this is especially true in med-poly modeling workflow with custom object normals that’s becoming popular among artists now). Same deal with other components - I don’t want to overwrite albedo, smoothness and other stuff.
Q: Why aren’t you using the Command Buffers showcase from Unity 5.1, it has an example doing exactly this?
A: Nope, it has an example for box projected decals which is not at all what I want to do - I need a surface shader for pre-authored decal geometry and UVs). In addition, command buffers seemed to be quite slow performance-wise and required inconvenient setup (additional components etc.) I’d like to avoid if possible.
Q: Why not use GrabPass?
A: GrabPass will give a lit frame, which makes it’s contents unusable for “invisible” surface output.
