CommandBuffer.Blit() isn't stencil buffer friendly

Unity Engine
1

EDIT: I’m consolidating this thread to keep it short and simple.

THE ISSUE:
Creating a black and white image of a stencil buffer, for later use in a post-processing effect, requires too many blits with CommandBuffer.Blit().

Here’s the summary:

  1. If you blit from the screen to a rendertexture, you can’t use a custom shader… or else the resulting render texture is blank. Also, it never copies the stencil to any render texture. I believe both these issues are bugs. (Case 834634)

  2. If you blit from a render texture, to another render texture, the stencil can’t be access in the shader. Probably because as I said in #1, stencils aren’t copied to render textures. I believe this is a bug.

  3. If you blit from a render texture to the screen, everything works including the stencil (weee!!), but now you just over-written the picture on the screen. You’d then have to fix that, I suppose with another blit to put the picture back again.

  4. You can’t blit from screen to screen, or from one render texture to itself, unless your shader specifically uses a global texture (not _MainTex). Otherwise result is blank picture.

  5. Any attempt to render the camera to a render texuture will result in no ability to access or use the stencil buffer, due to issue #1 above.

In the end it’s possible but it requires too many blits.

→ first blit is from screen to render texture, and you can’t use a shader, as I said in #1 above.
→ second blit is from that render texture to screen, to make the stencil image, since as I said in #2, you can’t blit from a RT to another RT, or else there is no stencil… but anyway now you just put the black and white picture on your screen so screen is screwed up.
→ third blit is from screen to render texture, again with no shader allowed (see #1 above), just to “save” the stencil image you made. Or else, if you skip this step, how are you gonna use it later?
→ fourth blit is from the original render texture, back to the screen again, with a shader to do some post processing effect. You’d have to set the stencil image to a global property, so that this shader would have it also to work with. This step is required for another reason also: to put the picture back on the screen, since you screwed it up on the second blit.

That’s a minimum of 4 blits just to make (and use) this stencil texture. I’d happily do it if it could be done with just one blit from screen to screen, but can’t (see #4 above).

Acceptable solutions:

A) It should be possible to blit from screen to render texture, and use a custom shader. Currently, this isn’t possible (see point #1 above)

B) Any blits from the screen to a render texture, should copy the stencil buffer to that render texture. Currently, it does not copy it, which probably is what results in point no. “C” below.

C) Any blits from render texture to another render texture should be able to access the stencil. Currently, it is not accessible.

D) It would be nice if it were possible to blit from the screen, to the screen, with a custom shader. This would cut down on the number of blits needed to do many post processing effects.

2

See my threads for more details:

And my bug report, which includes a well-made example scene:

(Case 834634): https://fogbugz.unity3d.com/default.asp?834634_utcuq9n1gbgpbit3

3

The issue here seems to be working in the version of unity I am using (some 5.5 based version). I’ve pinged QA to investigate what is happening in 5.4 here.

This is intentional. Only the color target is copied when blitting RT’s and blitting depth / stencil is not normally needed and has a performance cost. If you want to do this you can’t use blit and have to do a custom render. It’s not too hard to do and we do this for many of the command buffer image effects. One thing to note is that we DO NOT currently have a way to nicely copy the depth and stencil buffer, but you can use one that has already been created with a different color buffer. In command buffers you can do it like so:

            cmdBuffer = new CommandBuffer();
            cmdBuffer.name = "cmdBuffer";
            int texID = Shader.PropertyToID("_OlderTexture");
            cmdBuffer.GetTemporaryRT(texID, -1, -1, 0);

            // set the target to be the temp created texture + the current depth / stencil buffer
            // when we draw now we will be using the old depth stencil but a NEW color target
            cmdBuffer.SetRenderTarget(texID, BuiltinRenderTextureType.Depth);
            cmdBuffer.DrawMesh(MyMesh, MeshMatrix, material);
            Camera.main.AddCommandBuffer(CameraEvent.AfterForwardAlpha, cmdBuffer);

We are working on API to make this cleaner and nicer right now.

Well it will as you are using the stencil from the target.

This is how GPU’s work. You can’t read and write from the same target unless the API supports framebuffer fetch. https://www.khronos.org/registry/gles/extensions/EXT/EXT_shader_framebuffer_fetch.txt

4

Thanks bunches for the reply.

Did you test this? I assume this also is working in your version of Unity (5.5)? This doesn’t work in 5.4.1f1. Gives this error:

709×59 2.83 KB

I tested this at least a dozen times in the last week, and it never did work although I didn’t add it to my list of errors. I’m curious, if this is working in 5.5, are you able to use this to Blit from one RT to another RT and use the stencil?

Understand, I’m not “trying to copy the stencil to a RT”. I don’t care if the stencil is copied. Only reason I ever was trying to copy the stencil to the RT, is because you COULDN’T use something like BuiltinRenderTextureType.Depth in your SetRenderTarget(), so I was trying to find a work-around.

Also if this is possible in 5.5, then no need to use .DrawMesh() as you suggested… I should be able to just set the depth buffer and blit()!

Sounds like good times await me in 5.5.

5

I’ll upload the project when I’m at work tomorrow (I was using the one from you bug report as a basis). And yes I did test this.

Some things are a bit broken / unintuitive with command buffers currently (mostly depth / stencil related). We have a developer scheduled to fix a bunch of issues in this area in a few months. This won’t be just a ‘oh lets fix some bugs’ pass but a real sold ‘how should they work, whats broken’. So please raise bugs / feedback about CB’s as you encounter it.

When I was putting together the project on my PC I was using renderdoc just to see what was being done on the GPU. If you don’t use it currently I recommend looking at some captures as it is a massive time saver.

6

Okay will do. How do I make sure this future developer will get / see my reports?

I mean right now, off the top of my head, even though I think we’ve established that a few issues I reported may be already fixed in 5.5, I would still say the following issues exist / should be looked at:

  1. Stencils really should copy to RT’s, maybe not by default but at least as an option.
  2. BuiltinRenderTextureType.Depth should work (it doesn’t in 5.4.1).

And then, there are functions that don’t seem to work with stencils:

  1. cmdBuffer.SetRenderTarget() doesn’t seem to do anything.
  2. Camera.main.depthTextureMode = DepthTextureMode.Depth doesn’t seem to do anything.
  3. Camera.main.SetTargetBuffers(color, depth) doesn’t seem to do anything.
  4. cmdBuffer.GetTemporaryRT(, , , 24)… the “24” on this call doesn’t seem to do anything.

If line #1 on all these functions/options is simply “return;”, I wouldn’t be surprised, because they seem to do nothing.

None of the documentation talks about which functions work with stencils, and which don’t. So, since stencil’s are part of the depth, I had to include all these functions in my tests to make sure that my problems weren’t being caused by a wrong setting on one of these. Documentation on these and other depth functions need to mention their interactions if any with stencil buffers.

Another issue:

If all cameras are set to render to RT’s, there is no way to blit to the screen! It seems messy that I should have to set up another camera, and tell it to cull everything, just so that I can have a dummy camera to give me a means to put a picture on the screen (by blitting to that dummy camera’s CameraTarget). There should be a BuiltinRenderTextureType.Screen, which will always point to the screen, and allow a blit to the screen even if there are NO cameras rendering to the screen at the time.

And then, there are these errors:

720×99 4.78 KB

7

Some of these things I’m mentioning here isn’t necessarily bugs. Obviously I’m not doing things right. But then that begs the question, why am I doing so many things wrong? Several reasons:

  1. The bugs we mentioned above (some of which might be fixed in 5.5 already).
  2. The fact that stencil is mixed with depth, but almost all depth-related functions aren’t working with stencil.
  3. The fact that documentation doesn’t mention what functions interact with stencils.
8

Also, in the documentation for Graphics.Blit, it says:

This is all wrong, and the documentation shouldn’t say this. You can do a blit with the stencil, with a render texture as the source. I donno why this statement is in the documentation but it threw me off for some time.

Also, it seems to indicate that the stencil is part of the render texture. That also threw me off. And furthermore, this whole statement is a nightmare. A scary, evil nightmare. Sends you off on a wild goose chase trying to learn how to do all of this, and in the end, nothing about any of this will help a single bit with any problem you are having with stencil buffers.

9

But that statement IS correct. You can use SetRenderTarget with a color buffer and a separate depth/stencil from a different target. And it does work as expected.

It is part of the render texture. It’s part of the depth/stencil target.

Historically unity has treated render textures as a color + depth target, this is due to a bunch of legacy GL FBO fun from when unity was first written and it has leaked through into the API layer. Modern API’s treat targets as individual / separable textures, which is why there is now api on render texture to get the associated RenderBuffer. Internally though there is still some coupling between the RT and the Depth target.

These don’t seem to do what you think they do, and it is documented.

It does, we have tests and post processing that use this functionality. If it didn’t work a large number of features would not work. See here: https://bitbucket.org/Unity-Technologies/cinematic-image-effects/src/52a673841e7070266c062760ddd8a9f2320c4329/UnityProject/Assets/Standard%20Assets/Effects/AmbientOcclusion/AmbientOcclusion.cs?at=trunk&fileviewer=file-view-default

Unity - Manual: Cameras and depth textures This is well documented. You get access to a resolved depth texture.

I haven’t seen this is any of my testing.

Can you describe exactly what you are trying to do and show me the code you have for what you are trying to achieve? You are saying a lot of things are broken, but I can’t actually see what you are doing or what your expectations are in these cases.

10

No. 1:
The statement in question says it (the stencil) is part of the source, not target. But you just said it (I assume you mean stencil) is part of the target. Not sure if this makes a difference, but just pointing this out for the sake of exactness.

No. 2:
The statement says that the stencil is part of a (Render)texture. But earlier, you said:

So, it says “…stencil buffer that is part of the source RT”, and you say stencil buffers are never copied to RT’s, so how can a stencil buffer be “part of the RT”?

No. 3:
As you requested, I’m attaching an example package. It includes two scenes. One scene is called test_GraphicsBlit.scene, and it uses Graphics.Blit, and it works as expected. The next scene is called test_CommandBufferBlit.scene, and it attempts to re-create the same functionality line for line. And it doesn’t work, and generates errors.

Please also observe that in my attached scene, I use a Graphics.Blit(), and it works with the stencil just fine. So the documentation seems to indicate that to get the stencil to work, you must do a complex manual drawing of a quad.

So the doc is misleading because it:

  1. Suggests that a stencil can be part of a RT. Which sends you on a quest to learn how to read/write stencils to RT’s.
  2. Suggests you can’t use stencil without drawing a manual quad. Which sends you on a quest to learn how all that.

I loaded it AmbientOcclusion.cs. And I commented out the line in question:

//cb.SetRenderTarget(mrt, BuiltinRenderTextureType.CameraTarget);

It didn’t make any visible difference.

900×209 203 KB

Full-size screenshots: Imgur: The magic of the Internet

Alright I concede this one. I commented out this line from the AmbientOcclusion.cs, and it did make a difference. Maybe no difference for stencil (which has been my focus). But a difference for the depth.

2803821–203602–GraphicsBlit_vs_CommandBufferBlit.unitypackage (51.2 KB)

11

In case it’s any help, I submitted this package also as a bug report, case # 836103.

Note that I’m still using 5.4.1f1.

12

A render texture has a color target, and a depth/stencil target. I was referring to this. Specifically the depth/stencil target of the source texture. Yeah, this wasn’t super precise language.

Yeah I was imprecise here. Like I said in 1; a RT is a combination of a color + depth/stencil target. When a blit happens we only blit the color portion of the RT from one RT to the other. The source and target may have a depth / stencil, but we do not read and write between the two. This isn’t to say that it doesn’t have one, just that we don’t do anything with it during a blit operation.

There a few reasons for this:

  • It’s slower and (generally) not needed
  • Very difficult to match between MSAA RT’s (blits happen with resolved RT’s that have NO associated depth /stencil)
  • Manual reuse of depth targets is generally better even if the api right now sucks and is easy to mess up

Stencils are part of the RT. But right now there is no way to read / write them between RT’s. We have ways of doing this for depth. But not for stencil. This is an API limitation currently. I’m not sure if it’s in unity or at the gfx level (i know that stencil is a weird thing that is not always well supported).

Well a blit IS a quad, it’s a helper function that render with a texture and sets up some things like texture and render targets. But it does TOO much for the use case you have.

That path is only used for deferred + ambient + HDR only I believe, make sure you have that turned on.

Depth and stencil are ONE buffer on the hardware. It’s a 32 bit buffer with 24 bits for depth, and 8 bits for stencil.

13

Anyway, enough text.

So I just spent some taking a look into this and managed to make a working command buffer version. That is pretty simple in the end. It’s attached here (note: I did this on 5.5, i believe it will also work on 5.4, but if it does not let me know because that IS a bug).

So in your original code you do this (added some comments inline):

if (cmdBuffer == null)
{
    // code also requires a custom target render texture
    // and then blit to screen :(
    cmdBuffer = new CommandBuffer();
    cmdBuffer.name = "cmdBuffer";

    // this is fine, set up the target and clear
    cmdBuffer.SetRenderTarget(StencilTempTextureID);
    cmdBuffer.ClearRenderTarget(true, true, Color.black);

    // setting a render target then blitting does nothing,
    // unity will set the target (inclding depth / stencil target)
    // to whatever the second argument is here. In this case this will be
    // either the FINAL camera target or the backbuffer
    cmdBuffer.SetRenderTarget(StencilTempTextureID, CameraRenderTextureID);
    cmdBuffer.Blit(CameraRenderTextureID, BuiltinRenderTextureType.None, PostprocessMaterial);

    // once again same issue as above.
    cmdBuffer.SetGlobalTexture("_StencilTempTexture", StencilTempTexture);
    cmdBuffer.SetRenderTarget(BufferID, CameraRenderTextureID);
    cmdBuffer.Blit(CameraRenderTextureID, BuiltinRenderTextureType.None, stencil_is_on);

    GetComponent<Camera>().AddCommandBuffer(CameraEvent.BeforeImageEffects, cmdBuffer);
}

Mine has some changes I will detail inline

// NEED TO ADD THIS
// forces camera to go via an internal RT for rendering
// it's easier than allocating and managing your own.
// Any image effect on the camera will remove the need
// for this.
// In 5.6 there will be an API on camera for forceIntoRenderTexure
void OnRenderImage (RenderTexture src, RenderTexture dst)
{
    Graphics.Blit (src, dst);
}


void OnEnable()
{
    // moved these to here
    // can still be in OnStart, but I prefer to
    // lifescycle manage in OnEnable
    PostprocessMaterial = new Material(Shader.Find("StencilToBlackAndWhite"));
  
    // We need one temp texture here for the loop,
    // allocation + clearning can be moved into the
    // cmd allocation below unless you want to reuse it.
    StencilTempTexture = new RenderTexture(Screen.width, Screen.height, 24);
    StencilTempTexture.name = "Stencil";

    StencilTempTextureID = new RenderTargetIdentifier(StencilTempTexture);

    if (cmdBuffer == null)
    {
        cmdBuffer = new CommandBuffer();
        cmdBuffer.name = "cmdBuffer";

        // now we have a RT from the camera render we can reuse the depth / stencil from that
        cmdBuffer.SetRenderTarget(StencilTempTextureID, BuiltinRenderTextureType.CameraTarget);
        // clear just the color
        cmdBuffer.ClearRenderTarget(false, true, Color.black);

        // We can't use blit here as it will internal set destination,
        // so draw a quad instead. Vertex shader in the shader has been changed
        // see the code attached.
        cmdBuffer.DrawMesh(quad, Matrix4x4.identity, PostprocessMaterial, 0, 0);

        // set the global texture
        cmdBuffer.SetGlobalTexture("_StencilTempTexture", StencilTempTexture);
        // for now blit to camera texture, you could blit this
        // somewhere else if you want. Right now it will overright the
        // camera texture.
        cmdBuffer.Blit(null, BuiltinRenderTextureType.CameraTarget, stencil_is_on);

        GetComponent<Camera>().AddCommandBuffer(CameraEvent.BeforeImageEffects, cmdBuffer);

    }
}
This does the change in a clear + 2 blits.

ONE VERY IMPORTANT NOTE! This will not work properly with MSAA as it’s written(require additional work). When using MSAA rendering rendering happens to a special texture type. If you want to use this as a source texture for blits this needs to be resolved into a normal texture for the hardware to use. When a resolve happens only the color surface is resolved from the GPU side to the usable render texture side. This means there will NOT be a valid depth stencil present for doing the stencil pass on.

2804560–203651–workingcmdblit.unitypackage (43.4 KB)

2 Likes
14

Wow I didn’t expect this.

Okay so, trying to get your example to work with 5.4…

  1. You can’t open a 5.5 scene in 5.4, ha ha. But I think I can guess from your code that your scene had just one camera, so I’m recreating it like that. If you had two cameras (like my scenes did), let me know.

  2. You didn’t seem to include the StencilToBlackandWhite.shader file in your package. Did you make any changes to this shader? You said you updated the vertex shader, so I assume you did? If you could post that shader, that’s the only file missing.

  3. I tried it with my current StencilToBlackandWhite.shader file and it continually spams this error:

This could be because I’m missing your modified StencilToBlackandWhite.shader?

  1. Using 5.4, I had to change the line: Blit(null, to Blit((Texture)null, to avoid an ambiguous call error.

  2. Besides the above errors, it simply displays a solid black screen at the moment.

Send me that shader if you can, and I’ll give it another spin.

PS. I have some comments on some of the other things you said, but I’d like to get your example working for now. :slight_smile:

15

Oops. :frowning: Yeah there are changes to that shader.

Attached a 5.4 zip version. Has both the 5.4 project AND the .package.

2805732–203754–5.4CmdBliT.zip (105 KB)

16

This is getting weirder.

So, I ran it in the project you included (thanks for that by the way), and it worked.

But, I created a brand new project, and copied your Assets folder in, and it still was spamming my console with this same error:

But when I loaded your project folder (at zip/5.4CmdBliT ) into 5.4.1f1, it worked great.

So after trying this and that, I copied your ProjectSettings folder (from zip/5.4CmdBliT/ProjectSettings) into my new project, and the error went away.

1599×746 56.2 KB

Looks like your ProjectSettings files are all about 5 times smaller than mine. Yet, my project is brand new, I didn’t touch a thing. Any idea what’s going on here? scratches head. You didn’t happen to send me a projectsettings folder that was created in 5.5?

I’m attaching my project, this time the whole project folder structure in a zip.
It contains an exact copy of your assets folder, but the rest of the project (including the ProjectSettings folder) is simply a brand new project created by Unity 5.4.1f1. I deleted the Library folder just to keep the zip small.
I also included your ProjectSettings folder, it’s named “ProjectSettings – From Tim-C”.
Right now, it doesn’t work, it spams that error.
But if you replace the ProjectSettings folder with “ProjectSettings – From Tim-C”, the error goes away.

2805960–203765–ProjectSettings difference from TimC.zip (61.8 KB)

17

my project has MSAA disabled as i mentioned in my original post. its also using text serialization thats why the files are different sizes,

18

Oh, right, duh. I read that, but it didn’t click that I actually needed to check it. :slight_smile: Turning off AA fixed it. Although if I comment out your mysterious Graphics.Blit(), the error returns. Interesting.

Thanks Tim for all your help with all of this.

I have some feedback.

You seem to have done some kind of wizardry in your solution that only a dev who knows the internal workings of the engine would know. Your comments say that Graphics.Blit(src, dst) is needed because of some internal issue.

This looks like features written by the devs, for the devs. :stuck_out_tongue:

CommandBuffer Feedback:

  1. Need documentation. Example code in the docs. A mile long list of examples codes.
  2. depthSurface == NULL || rcolorZero->backBuffer == depthSurface->backBuffer isn’t comprehendable to non-devs.
  3. BuiltinRenderTextureType.Screen would be nice to have, to always be able to blit to the screen.
  4. Making Blit’s “set destination” feature to be optional would be nice.
  5. AA being ON, causing Blits to not work in this example, is probably not documented.
  6. OnRenderImage being needed to Blit in this example, is probably not documented.
  7. SetRenderTarget before a Blit doing “nothing”, should probably be mentioned in Blit’s documentation, even if it seems obvious to you, it wasn’t obvious to me.
  8. CommandBuffer should not ever have to rely on Graphics.Blit(), like in this example.
  9. “Temporary render texture _CameraDepthTexture not found” needs a lot more explanation in the error message, or a link to a documentation page that explains possible causes and solutions.
  10. “Built-in render texture type X not found while executing” needs a lot more explanation, and a help page, same as above.
  11. The documentation on Camera.main.depthTextureMode = DepthTextureMode.Depth should give a brief summery of where this texture is, and how its accessed, and a list of which functions and features would depend on it, and if it does or doesn’t contain stencil information. As it is right now, its documentation reads almost as if it thinks you already know all of this.
  12. The statement on Graphics.Blit’s documentation page concerning stencils, and the need to manually draw a quad, is still unsettling to me, and a lot (all?) my previous complaints about it still stand I think. Also it’s vague, and doesn’t give any example code specific to stencils.
  13. There are too many render textures, render targets, and target buffers.
    ----> OnRenderImage creates what you describe as a “Internal RT”.
    ----> Camera.main.targetTexture creates, I suppose, a different RT.
    ----> Camera.main.depthTextureMode = DepthTextureMode.TheThreeStooges.
    ----> _CameraDepthTexture, another RT?
    ----> “permanent” RTs.
    ----> “temporary” RTs.
    ----> BuiltinRenderTextureType.RTs (bunches here)
    ----> color and depth buffers so you can make, even more RT’s.
    ----> Then there is the screen itself, which seems to be something other than all of this, and very mysterious.
    All this is fine, I like to have lots of control. That is, if it all just works. But when these functions give out incomprehensible error messages, and the stencil can’t be accessed, having this many different things to search through to try to find the cause is a nightmare.
  14. The documentation on almost all these CommandBuffer features only describes what they do when they’re working perfectly, on a perfect day.
  15. Need more troubleshooting type of documentation.
  16. Need more relationship type of documentation, detailing which features would affect others, and which wouldn’t, and to what extent, and any potential conflicts between features. This would greatly reduce the research time needed to fix a problem.
  17. Especially need more stencil documentation as it relates to Graphics.Blit and especially CommandBuffer.Blit, since stencil is one of the most problematic features.

You don’t have to answer any the above, unless you want this discussion to go on forever. But maybe you could try to forward the above feedback to whoever will be working on this stuff in the future.

I’ll submit your example scene as a bug report, since… eh… turning on or off AA should not spam an incomprehensible console error, and because commenting out OnRenderImage gives the same error, and plus that’s just too much of a hack you did, anything that requires that much internal knowledge and hacking to work around should probably be looked at as a bug.

For all other people who found this thread, who like me were struggling to get CommandBuffer.Blit to work with or without stencils, I’m attaching all of my example scenes that I made in the last 10 days or so, not including the ones posted above.

2797304–202936–CommandBufferUsePreviousRenderTexture.unitypackage (95.3 KB)
2797158–202920–CommandBufferStencils.unitypackage (64.4 KB)
2806141–203790–CommandBufferBlitWorkaround.unitypackage (25 KB)
2799060–203105–CommandBufferBlitImprovements.unitypackage (23.4 KB)
2806141–203792–CommandBufferBlit_Final.unitypackage (28.1 KB)

3 Likes
19

…and one more example (won’t let me attach more than 5 in a single message lol)

2806146–203793–GraphicsBlit_Working.unitypackage (64.9 KB)

20

+1 to more command buffer documentation (more techy stuff is fine, just something that’s not Aras’s super old cb examples from 5.0 which have to be downloaded as a package).