Unity 4.2 was just announced with support for “stencil buffers”. I only know a little bit about these, so could somebody provide some clarification on what that practically means?
Will the API be making it in to Unity Free? I’m assuming yes, because stencil buffers are commonly used to create hard shadows, which are also in Unity 4.2 Free.
What kinds of effects can I make with it? If I understand right, it could allow me to do things like reflections, portals, etc. without RenderTextures. A quick Googling revealed a sort of glow effect done with stencil buffers, but the article was a little bit over my head.
For Pro users, what does it mean? What can you do with stencil buffers that you couldn’t do or was inefficient or ugly with RenderTextures?
If it’s what I’m thinking it is, a stencil buffer has been around for many years. OpenGL supported them back in GL 1.1 or whatever. It’s basically an extra per-pixel buffer and typically it used to be 1 byte per pixel, ie 8-bit per pixel. There are various stencil operations that can be performed such as whether or not pixels will be written to the stencil buffer, whether the pixels in the stencil buffer will be used to discard pixels that you’re about to draw, etc… and one of various different modes for comparing the values in the buffer to decide whether to pass or block a pixel from being output. You can draw to the stencil buffer without drawing to the color/backbuffer, and vice versa, or do both at once etc. So basically there are various uses for this such as masking out areas of the screen, implementing a special split-screen mode or a cutout, you can as you say use them to make stencil shadows, and also you can use it to fill concave polygons. I’m not sure though what kind of exposure Unity will be giving and how many options you’ll have for how to use it.
Only viable solution for realistic convincing good reflection, is in pro. No matter what you do,it is the only solution, performance-wise. I tried it in several ways. On all,it require rendertexture which is pro only. Even if you do it by capturing what is on the screen(an script on wiki page does the job,search as “free version of rendertexture”),it will be damn slow as it eats up your cpu. You try to move that workload to gpu through compute shader,again rendertexture is necessary for writing data back. So do not hope realistic reflection(preferably mirror type) in free version. And no way for mobile device too.
As far as I know(which is not so much,any informative help direction is welcomed :p),
pixel shader affects all the fragment,object has. With stencil buffer,you can limit it however you want. I presume with some sort of texture maps(alpha?).
I guess stencil buffer access will not be available in Unity Free. Unity uses shadow mapping for shadows so no stencil buffer required here.
It’s interesting that stencil buffer access is made available now after all those years.
I’m also keen to hear some possible usages. Maybe some image effects could rely on the stencil buffer instead of alpha channel masks which would give some more flexibility.
Are stencil buffers supported by mobile devices at all?
One use that may help sometimes is to e.g. draw a circle to the stencil buffer then render a rectangular camera, so that only the pixels within the circle get rendered… though there are other ways to do this with shaders now, which is why I said rendertexture would be far more useful than stencils given what you can do with them in shaders.
Only if you have a solid UI, which very few games do these days.
Stencils have a wide range of applications even though they act essentially as a simple 1 bit mask (though generally they are 8 bit buffers, enabling storing (or incrementing/decrementing) a value between 0-255 ). Stencil shadows can still be useful as masking is always useful, but generally yeah, you’ll either have a direct need for them or not.
Well actually you can do this sort of thing also by modifying some shaders to only draw within a given area either based on a procedurally calculated shape or some additional texture layer acting as a stencil.
Very true, though isn’t one of the advantages of stencils is that it happens much early in the render pipeline and thus acts as an early-out to avoid more process intensive parts such as shaders?
Don’t know if things have changed with modern gpu’s as i’ve never really had access to stencils in any of the 3D engines i’ve used in the past, but pretty sure it was one of the main features when initially conceived.
Actually you’re right, it prevents the processing of the fragment further… I forget what the exact order is but it’s fairly early in the pipe. That is a possible performance benefit. However, it’s also something of a fixed-function pipeline kind of deal, and once shaders came along to replace a huge chunk of the pipeline it makes you wonder if it would be just as fast doing a fragment shader.
i wish that someone will make an asset that use stencil for point light shadows… i will buy it for 100$ for sure
the current point light shadow is useless, i paid 1500$ twice for that and every day i hope they improve it as they did with the directional light shadow.
I think you might have the wrong impression of stencil shadows. They are not a good generalised solution, they require all models be ‘closed’ (i.e each edge is shared by exactly only two triangles), use more storage for model geometry, increase processing on the cpu every frame and they don’t scale well with increasing numbers of casters and light sources and suffer from fill-rate issues.
As such they are really a solution for very special cases or effects and would require considerable additional investment in time to prepare assets and attention to scene set-up.
i am aware of their requirement and limitation, but for some reason i like them a lot.(Thief Deadly Shadows The Chronicles of Riddick : Escape from Butcher Bay was my favorite games) and for my needs they will serve me will.
stencil shadows was on machines form 7-8 years ago, it will be easy for a CPU from this year to run them with no problems.
Stencil access will be pro-only feature. Sorry, we have to make the split somewhere and I actually like the fact that keeping features like this from free doesn’t hinder the ability to make games, but just reserves some of the more funky effects for pro.
And about what it does - basically what imaginaryhuman said.
You can think of it as a mask. Your framebuffer or render texture will have additional 8 bits per pixel, which you can write into or increment/decrement their value by rendering something. Writing color or depth at the same time is optional, as usual, so you can only render into the “mask”, if you like. Subsequent objects can have their rendering dependent on the contents of the “mask”, e.g. cheaply discard pixels or further modify those 8 bits.
Stencil can be very useful to help with a bunch of different effects, which might otherwise be more cumbersome to implement or slower, like cutting holes in objects with other objects or making characters behind obstacles render in a special way without screwing up depth test of their own overlapping geometry.
It will render only those pixels, where stencil was 0 or 1 (2 has to be greater). At those pixels the value in the stencil buffer will be replaced with 2. If the z-test failed, the value in the stencil will be decremented (and the pixel will be discarded). Since the block above didn’t define what to do if the z-test passes, but stencil test fails, the value in the stencil buffer won’t be modified (and the pixel will be discarded).
