How to make a camera render to both screen and texture?

[orreborre_1]

Hello!

Trying to learn more about rendering in Unity. I have already googled and read a lot of threads here and tried a lot of code during many evenings now, but it seems a lot of these threads are quite old, so I decided to try my luck and make a new one.
I’m on Unity 2022.3.

Anyway, I’m trying to have a camera render to a low resolution render texture after it has rendered whatever it sees to the screen. Or before, it doesn’t matter, I just want it to render to both the texture and the screen.

This is how I thought it should work; create a command buffer, blit from camera target to the render texture, and add the command to the camera. Unfortunately, this piece of code seems to do absolutely nothing at all, so I guess I’ve misunderstood how it should work.

using System;
using UnityEngine;
using UnityEngine.Rendering;

public class UiCamera : MonoBehaviour {
    [SerializeField] private Camera cam;
    [SerializeField] private RenderTexture rt;
    private CommandBuffer cmdBuffer;

    private void Update() {
        cmdBuffer = CommandBufferPool.Get();
        RenderTargetIdentifier rti = new RenderTargetIdentifier(rt);
        cmdBuffer.Blit(BuiltinRenderTextureType.CameraTarget, rti);
        cam.AddCommandBuffer(CameraEvent.AfterEverything, cmdBuffer);
    }
}

I’m super thankful for any help. I’m currently going through Catlike Coding’s stuff on Custom SRP, but I was hoping it would be easy to just add another command to a camera and that I wouldn’t need a whole customized render pipeline.

Thanks for reading and have a nice day!

[ElliotB]

I feel the key ingredients you are missing are (i) a ScriptableRenderPass to inject your instructions when rendering and (ii) a ScriptableRenderFeature to add your pas to the pipeline. There’s a good book that was recently released which contains some examples of how to do this for a variety of effects:

New free e-book: 12 Recipes for popular visual effects using the Universal Render Pipeline

Hi everyone, We just wrapped up a new e-book, Recipes for popular visual effects using the Universal Render Pipeline, which contains 12 recipes for popular visual effects that can be applied to a wide range of games, art styles, and platforms. It was created by highly experienced internal and external Unity developers and technical artists, and reflects the latest features available in Unity 2022 LTS. >> DOWNLOAD IT HERE << A few topics we cover: Stencils and instancing Toon and outl…

[orreborre_1]

@ElliotB

I feel the key ingredients you are missing are (i) a ScriptableRenderPass to inject your instructions when rendering and (ii) a ScriptableRenderFeature to add your pas to the pipeline. There’s a good book that was recently released which contains some examples of how to do this for a variety of effects:

https://discussions.unity.com/t/922004

Thank you for pointing me in the right direction, I will for sure read through that book. Incredible that it never showed up in my google results during these last couple of weeks.
Ok so I’ll need to get familiar with ScriptableRenderPass and ScriptableRenderFeature then. I did try following some youtube videos where those were used but never got any results in Unity that they got in the videos. The stuff I’ve read must’ve been sorely out of date since they did this in the Update loop in some occurrences.
I will report back here if I ever get this working.

[ElliotB]

This resource is also useful:
https://docs.unity3d.com/Packages/com.unity.render-pipelines.universal@16.0/manual/renderer-features/how-to-fullscreen-blit.html

[orreborre_1]

Hey again render people, reporting back a few days later. Writing down my findings here in case someone finds it useful.

So I checked that book out, @ElliotB , and though it wasn’t what I was looking for, it made me find another book: Introduction to the Universal Render Pipeline for advanced Unity creators | Unity
This had a section with some stuff about Render Features, which was outdated, but after a bit of messing around and reading this upgrade guide Upgrading to URP 13 (Unity 2022.1) | Universal RP | 16.0.6 I finally managed to get some stuff at least rendering something.

I am aiming to create a 2d shadow trick effect, where I (using an orto camera) copy everything on the screen to a (smaller) render texture that uses a material and shader to just use the alpha from the input render texture, make it black, and then put this render texture on a quad mesh in the background, and then offset this quad some pixels to make it look like everything casts a shadow.

Something like this might give you an idea:

I have managed to make a camera both render what it sees and also render this to a render texture:

As you can see, there are some problems left to solve

Firstly, I need to not render the quad itself to the render texture, as that will produce the smearing effect you can see in the image above. This would imply I need to use layermasks or something, I’m not sure. I did try using a layermask and a replacement material, which is why the sphere is rendered as unlit blue on the render texture. However I can’t seem to make anything not render.

Secondly, working with replacement materials, layermasks seem not to work on UI components. The green text “shadow” here is in the same layer and hierarchy as the sphere, but the attempted replacement material yields no results. The same goes for the Image component, btw. Not sure if it’s the layermask not working, or if replacement materials won’t work with UI components. I understand that replacing the material for texts would mess the text up completely, but it’s worth mentioning that nothing happens. I suppose I could make this work if I just decide that all UI cast shadow. But I would prefer being able to control it.

I think I might need to go about this some other way. I would really like not having multiple cameras though, since this would require me to change the Canvas’ camera all the time during rendering. However, I’m open to suggestions.

I will keep trying things, but if you have any idea on where I could push this further or another way to achieve this effect I would be very thankful for any response

Have a nice day!

[orreborre_1]

Hello again,
I abandoned the shadow thing, and tried making an edge effect instead.
I’m rendering a render texture on top of the camera result, but as you can see, the sorting that happens in the render on the render texture only seems to sort on an object-center basis. Is there a way to have it do proper sorting?
This is using SortingCriteria.BackToFront

You can see the edge layer popping whenever the sphere passes the box center.
Anyone had a similar problem?

Thanks for reading, I hope you’re having a nice day!

[orreborre_1]

So I tried dumbing it down as much as I could to try and see what’s wrong with the sorting. I started almost from scratch stealing a lot from GitHub - MirzaBeig/Anime-Speed-Lines: Post-processing effect to procedurally generate a anime/manga-style vignette of lines typically used to portray speed or surprise. (Thank you!)

I’m still having sorting issues. Any help would be appreciated!
Here you can see my problem:

Note how the sorting is only working on a per-object-basis when rendered to the render texture.
I’m replacing the material with a simple material that just shows the normals. I’m also using that same material on the two objects in the Game window just for me to see that the sorting is working as it should in there.

Here’s the code:
EdgeEffectRenderPassFeature.cs

using UnityEngine;
using UnityEngine.Rendering;
using UnityEngine.Rendering.Universal;
using System.Collections.Generic;

public class EdgeEffectRenderPassFeature : ScriptableRendererFeature
{
    [System.Serializable]
    public class CustomRenderPassSettings
    {
        public Material material;
        public RenderTexture rt;
        public LayerMask mask;
        public RenderPassEvent renderPassEvent = RenderPassEvent.AfterRenderingPostProcessing;
    }

    EdgeEffectRenderPass m_ScriptablePass;
  
    public CustomRenderPassSettings settings = new CustomRenderPassSettings();

    /// <inheritdoc/>
    ///
    public override void Create()
    {
        m_ScriptablePass = new EdgeEffectRenderPass(settings);
    }

    // Here you can inject one or multiple render passes in the renderer.
    // This method is called when setting up the renderer once per-camera.

    public override void AddRenderPasses(ScriptableRenderer renderer, ref RenderingData renderingData)
    {
        // we dont wan't to run the pass if there is no material
        if (settings.material == null)
        {
            Debug.LogError("Render Pass `EdgeEffectRenderPassFeature` missing material");
            return;
        }

        if (renderingData.cameraData.cameraType == CameraType.Game)
            renderer.EnqueuePass(m_ScriptablePass);
    }

    public override void SetupRenderPasses(ScriptableRenderer renderer, in RenderingData renderingData)
    {
        if (renderingData.cameraData.cameraType == CameraType.Game)
        {
            // Calling ConfigureInput with the ScriptableRenderPassInput.Color argument
            // ensures that the opaque texture is available to the Render Pass.
            m_ScriptablePass.ConfigureInput(ScriptableRenderPassInput.Color | ScriptableRenderPassInput.Depth | ScriptableRenderPassInput.Normal | ScriptableRenderPassInput.Motion);
            m_ScriptablePass.SetTarget(renderer.cameraColorTargetHandle);
        }
    }

    class EdgeEffectRenderPass : ScriptableRenderPass
    {
        RTHandle m_CameraColorTarget;
        RTHandle rtHandle;
        LayerMask mask;

        // store settings instead of material itself
        EdgeEffectRenderPassFeature.CustomRenderPassSettings settings;

        // name this what you want, it will be used to name the profile in frame debugger
        const string profilingName = "EdgeEffectRenderPass";

        private readonly List<ShaderTagId> shaderTags = new List<ShaderTagId>
        {
            new ShaderTagId("SRPDefaultUnlit"),
            new ShaderTagId("UniversalForward"),
            new ShaderTagId("UniversalForwardOnly"),
            new ShaderTagId("LightweightForward")
        };

        public EdgeEffectRenderPass(EdgeEffectRenderPassFeature.CustomRenderPassSettings settings)
        {
            // storing the settings allows you to add more features faster without having to boiler plate code,
            // also ensures that any changes made in the render feature reflect in the pass
            this.settings = settings;
            renderPassEvent = settings.renderPassEvent;
            if(settings.rt != null) {
                rtHandle = RTHandles.Alloc(settings.rt);
            } else {
                Debug.Log("RT IS NULL");
            }
            mask = settings.mask;

            // create a new profiling sampler with are chosen name,
            // else you get just a generic "ScriptableRendererPass" name
            this.profilingSampler = new ProfilingSampler(profilingName);
        }

        public void SetTarget(RTHandle colorHandle)
        {
            // get the source target from rendering data every frame
            m_CameraColorTarget = colorHandle;
        }

        public override void OnCameraSetup(CommandBuffer cmd, ref RenderingData renderingData)
        {
            ConfigureTarget(m_CameraColorTarget);
        }

        public override void Execute(ScriptableRenderContext context, ref RenderingData renderingData)
        {
            var cameraData = renderingData.cameraData;
            if (cameraData.camera.cameraType != CameraType.Game)
                return;

            FilteringSettings filteringSettings = new FilteringSettings(RenderQueueRange.opaque, mask);
            DrawingSettings drawingSettings = CreateDrawingSettings(shaderTags, ref renderingData, SortingCriteria.BackToFront);
            if(settings.material != null) {
                drawingSettings.overrideMaterial = settings.material;
            }

            CommandBuffer cmd = CommandBufferPool.Get(nameof(EdgeEffectRenderPass));
            using (new ProfilingScope(cmd, profilingSampler))
            {
                Blitter.BlitCameraTexture(cmd, m_CameraColorTarget, rtHandle, settings.material, 0);
            }
            context.ExecuteCommandBuffer(cmd);
            context.DrawRenderers(renderingData.cullResults, ref drawingSettings, ref filteringSettings);
            context.Submit();
            cmd.Clear();

            CommandBufferPool.Release(cmd);
        }
    }
}

and here’s the test normal shader used (but any shader doesn’t seem to sort properly):

Shader "Custom/normalstest"
{
    Properties
    {
        _MainTex ("Texture", 2D) = "white" {}
    }
    SubShader
    {
        Pass
        {
            CGPROGRAM
            #pragma vertex vert
            #pragma fragment frag
            // make fog work
            #pragma multi_compile_fog

            #include "UnityCG.cginc"

            struct appdata
            {
                float4 vertex : POSITION;
                float3 normal : NORMAL;
                float4 color : COLOR;
                float2 uv : TEXCOORD0;
            };

            struct v2f
            {
                float2 uv : TEXCOORD0;
                float4 vertex : SV_POSITION;
                float3 normal : NORMAL;
                float4 color : COLOR;
            };

            CBUFFER_START(UnityPerMaterial)

            sampler2D _MainTex;
            float4 _MainTex_ST;

            CBUFFER_END

            v2f vert (appdata v)
            {
                v2f o;
                o.vertex = UnityObjectToClipPos(v.vertex);
                o.uv = TRANSFORM_TEX(v.uv, _MainTex);
                o.color = v.color;
                o.normal = mul(unity_ObjectToWorld, v.normal).xyz;
                return o;
            }

            fixed4 frag (v2f i) : SV_Target
            {
                fixed4 col = tex2D(_MainTex, i.uv);
                col.rgb = i.normal.xyz;
                col.a = 1;
            
                return col;
            }
            ENDCG
        }
    }
}

Thanks for reading! Have a nice weekend