CustomPassUtils.RenderFromCamera not working as expected.

I’m using Unity 2021.1.2f1 and HDRP version 11.0.0

I’m attempting to use a Custom Pass and CustomPassUtils.RenderFromCamera() to render a bunch of objects from a separate top-down orthographic camera to a render texture.

The objects I need to render in this top-down camera are NOT visible to the Main Camera, they’re in a layer that’s excluded from the Culling Mask of the Main Camera, and only the top down camera should be able to render.

My Custom Pass is pretty simple. I have a second incactive camera I’d like to render from (topDownCamera), a render texture to render to (topDownRenderTexture), and a culling mask containing the objects to render (topDownLayerMask).

Then the Execute function is just

    protected override void Execute(CustomPassContext ctx)
    {
        CoreUtils.SetRenderTarget(ctx.cmd, topDownRenderTexture, ClearFlag.All);
        CustomPassUtils.RenderFromCamera(ctx, topDownCamera, topDownLayerMask);    
    }

RenderFromCamera() uses the correct position and orthographic projection of my top down camera, but it does not use the correct aspect ratio, which is set to 1 on my top down as I’m rendering to a 512x512 square texture.

Instead it uses the aspect ratio of the Main Camera (or scene camera). That’s annoying, but it’s not a huge problem as I can compensate for an incorrect aspect ratio.

The big problem is that it won’t render anything on the topDownLayerMask if the main camera isn’t also rendering it. The objects to render get culled, either by the layer mask or by frustum culling of the main camera.

Now at first I thought it was working fine, but it was actually only working in the editor because I have a Scene window open, and the scene camera was allowing the objects to avoid culling.

With the scene window closed, or with

        if (ctx.hdCamera.camera.cameraType == CameraType.SceneView)
            return;

included at the start of the Execute function, then nothing renders.

Is there a way I can force the objects in the topDownLayerMask to render duing RenderFromCamera, even though they’re not visible in the main camera?

I assumed because RenderFromCamera takes a LayerMask that this would be a non-issue, but it’s not working for me.

Is there something obvious I’m missing? Is there a better way to achieve this?

More Info:

Custom Passes is my third attempt at this problem.

Previously I had it working simply using a second camera, but in HDRP this adds >1.5ms of additional render time, even if all the Custom Frame Settings > Rendering flags are set to false, which I assume is meant to streamline the camera.

Then I had it (almost) working by using Command Buffers and drawing the objects with commandBuffer.DrawRenderer(). But in HDRP, commandBuffer.SetViewProjectionMatrices() doesn’t work, so I can’t render from the viewpoint of my top-down camera.

If either of these previous methods can work with a slight tweak, (either streamlining the use of a second camera, or a working version of commandBuffer.SetViewProjectionMatrices) please let me know, as they’d be preferable to using Custom Passes.

Hello,

This will be fixed soon when we’ll expose a new way of overriding camera view for rendering objects.

Yes, that’s expected because the custom pass uses the culling result from the current camera to draw the objects, and more specifically, it’s the culling result in CustomPassContext that contains the list of objects to render (obtained from the camera culling).

You can work around this issue by overriding “AggregateCullingParameters” function in the custom pass, it will allow you to add your layer in the culling parameters of the camera. So when the culling step happens, your objects will be included in the culling result of the custom pass and you’ll be able to render objects that are not visible in the camera but are inside the frustum of the camera. For more information on this, you can consult the troubleshooting page of custom passes: https://docs.unity3d.com/Packages/com.unity.render-pipelines.high-definition@latest?subfolder=/manual/Custom-Pass-Troubleshooting.html#culling-issues
Or this example in the custom pass sample repository: https://github.com/alelievr/HDRP-Custom-Passes/blob/master/Assets/CustomPasses/UI_Blur/UI_Blur.cs#L26

Note that if you want to render objects that are outside of the current camera frustum, it’s also possible but you need to do the culling of those objects yourself. There is an example of this here using a disabled camera: https://github.com/alelievr/HDRP-Custom-Passes/blob/master/Assets/Scenes/CameraDepthBaking/CameraDepthBake.cs#L33. The culling result can also be cached if your camera doesn’t move, which decreases a lot the rendering cost.

The LayerMask parameter in RenderFromCamera is to apply the filter on the objects in the culling result in parameter, it doesn’t change how the culling is performed by the camera.

Thanks so much, that solved my problem.

All it needed was

   protected override void AggregateCullingParameters(ref ScriptableCullingParameters cullingParameters, HDCamera camera)
    {
        cullingParameters.cullingMask |= (uint)topDownLayerMask.value;
    }

Regarding the other approaches I attempted, that work in Legacy, do you expect there’s any potential for then in the future in HDRP - that is, streamlining the use of a second camera (stripping all lighting and post processing off it so it can render without a large performance cost), or a working version of commandBuffer.SetViewProjectionMatrices? Or is Custom Passes the only real solution going forward for rendering views that aren’t the main camera in HDRP?

For now, it’s not possible to strip everything like that with an HDRP camera, and even with everything disabled, it’s still costly. There is no plan to fix it that way in HDRP because there is also a lot of CPU cost that we don’t control as a package (for example culling is done on C++ and we have very little control over it + it’s expensive).
The correct solution is a custom pass, in the future, we’ll expose the OverrideCameraRendering API (you can see it here in the code: https://github.com/Unity-Technologies/Graphics/blob/master/com.unity.render-pipelines.high-definition/Runtime/RenderPipeline/HDRenderPipeline.cs#L2530). It’s a more flexible approach to rendering objects from another view but it’ll be mainly for custom passes (and maybe camera custom render).

FYI commandBuffer.SetViewProjectionMatrices doesn’t work anymore because we use constant buffers instead of relying on global matrices, this is for performance reasons and currently, it’s not planned to support it again.

So, yes custom passes are the way to go.

Note that it’s also possible to use “Fullscreen Passthrough” on a camera (it completely disable HDRP rendering on the camera so there is no cost any more) but you have to do the rendering manually which may be complicated, especially if you want lighting on your objects (in that case I guess it’s impossible without changing HDRP and writing a lot of code).
https://docs.unity3d.com/Packages/com.unity.render-pipelines.high-definition@12.0/manual/HDRP-Camera.html

Hi @antoinel_unity , Looks interesting! Can you share any estimation on that feature (or link to track it). And how it can be used today with HDRP 11 (Custom pass that overrides camera projection)?

Looks like CustomPassUtils.RenderFromCamera have a bug with SSS material and Motion blur

SSS material - green “weapon” thats how it looks from regular camera, and black - renders via custom pass.

Same with Motion blur (regardless of SSS material):

I’m going to see if we can integrate it to 12.0 but not sure.

Yeah, it’s “expected” because the RenderFromCamera function only draws the Forward pass of objects and it doesn’t include motion vectors or SSS, currently there is no way to do multiple rendering passes with a single call like this.

For an “FPS camera”, the best remains to do a ShaderGraph that cancels the FOV of the projection matrix in the vertex shader (if you want all the feature support of HDRP). But it’s pretty complicated to do.

Thank you.
Will OverrideCameraRendering API help with SSS, motion blur ect. situation?

Can you explain more detail on how to change the projection matrix for whole material in Shader Graph? As far as I understand, it is not enough to just change clip pos vertex position, so shadows, lighting and other stuff work properly.