I’m a veteran programmer new to the shader world and I need some functionality but don’t know for sure whether it’s possible to do it within a shader. Here is something I would like to implement and would appreciate advice as to whether a) it’s possible and b) which direction to take :
A shader that acts somewhat like glass, such that when looking through the “glass” everything behind it is dimmer, and I can specify the level of dimness (ie. pitch black to absolutely transparent) via some byte value. However, I would like to be able to have the shader ignore the dimness on certain meshes via a transparency mask.
For example, let’s say that there’s a big window and I’m looking through it. Outside it’s bright and sunny and I adjust the “dim” switch for the window from absolutely transparent (ie. I see everything through the window clearly as nature intended) to absolutely nothing at all (ie. I can’t see anything through the window at all, it’s completely black, gray, or whatever color I define as nothing). I toggle the switch to a medium setting so that I can still see through the window but everything is 50% dimmer than before. Now I have a friend outside pick up another plane of glass and he holds it up in the air and waves it around as if holding a picketting sign, running back and forth. I notice that everything behind his plane of glass is back to normal; in other words, they’re not dim. The glass he is waving around negates the dimming effect of my window, so that even through I may have my window at absolute dimness and pitch black and I can’t see anything (not even my friend running around), I can still see his glass moving around as clear as day, as well as everything behind it.
So as you can see from the example I gave, my friends “glass” is acting like a mask/filter to my window, but I want to give the mask an additional level of flexibility. I want to make it so that this mask uses a transparancy texture so that based on the alpha level it will go from 0 (completely negate the dimness of my window) to 255 (does nothing at all to negate the dimness of my window).
Writing it all out makes it seem a lot more complex than it really is, so I hope you get the idea. I think I should be able to get away with making only one custom shader for the window; I don’t think I need to create a custom shader for the “filter” since I should be able to use a regular old transparent shader for it, right? Somehow I think I’m wrong…
Any thoughts? If it’s not possible exactly as I describe, would you be able to suggest alternatives?
Here is an idea (untested) which assumes that your whole scene has only dimming windows of one particular opaqueness A0 (A0 = 1: opaque window, A0 = 0: transparent window).
Render all opaque stuff without blending but make sure that they write the value A0 into the alpha component of the framebuffer. (Alternatively you can also render a screen-filling rectangle after rendering all opaque geometry, set the color mask to alpha only and set the alpha component of all pixels to A0.)
Render the window of your friend in the transparent render queue (i.e. after all opaque geometry has been rendered but before the dimming window is rendered). When rendering the window of your friend set a color mask to alpha only and write a alpha value between A0 and 0 (where A0 means that the window of your friend has no effect, and 0 means that the window will avoid any dimming).
In a queue after the transparent render queue, render the dimming window with a blending equation that uses the alpha of the framebuffer instead of the alpha of the fragment as a blending factor of the fragment color (and 1 minus this factor for the framebuffer color).
I assume this should work, but I’m probably missing something.
The way I would do this would be using render textures and some custom rendering:
Use shader replace to render all the ‘anti-windows’ into their own buffer. You would end up with a texture that contained only the areas that cancel out the window. You have 4 channels to work with here so you could do a lot of cool stuff!
Render the window, use this texture as a mask for where the glass should be cancelled out or not.
It will require a few things: custom shaders and some shader replace. Check out the shader replace example projects on the unity website they should get you off to a good start!
