Multiple render targets

[iko79]

Hi,

I am trying to render different properties of my scene geometry into different buffers (world positions/normals/ambient colors, etc.) and I cannot find a way to do that. The only thing that attracted my attention was the method Graphics.SetRenderTarget( RenderBuffer[ ] colorBuffers, RenderBuffer depthBuffer ), however I don’t know how to use it in combination with my scene camera and how to write into the the rendertargets from a shader. I stumbled across this example here, but while it helps me getting an idea of how to write my shader, it doesn’t help me much in terms of scene rendering, since it only renders a fullscreen quad by some OpenGL calls. I tried to call camera.Render and camera.RenderWithShader instead, but without success:

		RenderTexture lastActive = RenderTexture.active;
		Graphics.SetRenderTarget( mrtRB, mrtTex[0].depthBuffer );
		GL.Clear( false, true, Color.clear );
		this.cameraCopy.camera.RenderWithShader( this.gBuffersMaterial.shader, "" );
		RenderTexture.active = lastActive;

Does anybody know how to do this?

Thanks a lot!
-iko

[aubergine]

use temporary render texture and camera.renderwithshader to write on this temporary texture and set it as a shader property. You can do this as many times as you want.

Your renderwithshader params are wrong by the way, read the documents.

EDIT: if you need a good example, you can check my glow per object asset in the store.

[Dolkar]

I did a bit of research on this… Of course, you can fill multiple textures in multiple passes if you don’t care about your fps.
Anyways, turns out that to use MRTs on actual meshes, you have to render them using the low level DrawMeshNow. I temporarily solved it by using a single texture, but ARGBFloat. That gives you 24 usable bits per channel you can store your data into. Here are the encoding and decoding functions I used:

		inline float encodeChannel(float3 channel) {
			return dot(floor(channel * 255), float3(1, 1 / 256.0, 1 / 65536.0));
		}
		inline float3 decodeChannel(float channel) {
			float bitcorr = 256.0 / 255.0;
			return float3(channel / 255.0,
						  frac(channel) * bitcorr,
						  frac(channel * 256) * bitcorr * bitcorr);
		}

It’s not perfect… it encodes 255 as 254 for example, but the error is never greater than 1. And Aubergine, the params look alright. renderwithshader takes a shader and a string.

[aubergine]

And Aubergine, the params look alright. renderwithshader takes a shader and a string.

Always follow what unity suggests, if you use an empty string then it will replace all including transparent stuff.
You have to use “RenderType” as replacement tag which is unity standart and you have to prepare your replacement shader according to, if you dont want trouble later on.
Read the documentation.

[iko79]

Thanks aubergine, but if I got you right you’re suggesting to render one target at a time? That’s exactly what I’m doing right now and I am looking for an alternative, so I can generate all gBuffers at one single pass.

@Dolkar: Well, I do care for th FPS I’ve got some expensive geometry in my scene right now and want to avoid multiple passes on it. Thanks for the workaround, but I think I have to try the low-level approach.

[aubergine]

from the documents:

A single RenderTexture object represents both color and depth buffers, but many complex rendering algorithms require using the same depth buffer with multiple color buffers or vice versa.

So,

Graphics.activeColorBuffer

and

Graphics.activeDepthBuffer

hold the current renderbuffer info. As well as;

static function SetRenderTarget(colorBuffers: RenderBuffer, depthBuffer: RenderBuffer): void;

controls which renderbuffers to write on from your shader which shares the same depthbuffer.

Look at the Graphics class in the documents.

[iko79]

Thanks for your reply, aubergine. However I cannot see what the difference is to what I tried to do in the first place (see initial post above). Maybe you can help me figure out what the problem could be. Perhaps I’m doing something wrong in the shader? Here’s the initialization code of the RenderTextures (note that they’re not temporary, but this shouldn’t be an issue…)

	RenderTexture[] mrtTex	=	new RenderTexture[3];
	RenderBuffer[] mrtRB	=	new RenderBuffer[3];
	//...
	void Start()
	{
		Shader shader = Shader.Find( "Custom/GBuffers" );
		if( !shader )
			throw new Exception( "GBuffers shader not found" );
		this.gBuffersMaterial = new Material( shader );

		this.cameraCopy = new GameObject();
		this.cameraCopy.transform.localPosition = Camera.main.transform.localPosition;
		this.cameraCopy.transform.localRotation = Camera.main.transform.localRotation;
		this.cameraCopy.transform.localScale = Camera.main.transform.localScale;
		this.cameraCopy.name = Camera.main.name + " copy";

		Camera cam = this.cameraCopy.AddComponent<Camera>();
		cam.CopyFrom( Camera.main.camera );
		cam.enabled = false;

		this.positionTarget = new RenderTexture( (int)this.cameraCopy.camera.pixelWidth, (int)this.cameraCopy.camera.pixelHeight, 32, RenderTextureFormat.ARGBFloat );
		this.positionTarget.wrapMode = TextureWrapMode.Clamp;
		this.positionTarget.filterMode = FilterMode.Point;

		this.normalTarget = new RenderTexture( (int)this.cameraCopy.camera.pixelWidth, (int)this.cameraCopy.camera.pixelHeight, 32, RenderTextureFormat.ARGBFloat );
		this.normalTarget.wrapMode = TextureWrapMode.Clamp;
		this.normalTarget.filterMode = FilterMode.Point;

		this.ambientTarget = new RenderTexture( (int)this.cameraCopy.camera.pixelWidth, (int)this.cameraCopy.camera.pixelHeight, 32, RenderTextureFormat.ARGBFloat );
		this.ambientTarget.wrapMode = TextureWrapMode.Clamp;
		this.ambientTarget.filterMode = FilterMode.Point;

		this.mrtTex[0] = this.positionTarget;
		this.mrtTex[1] = this.normalTarget;
		this.mrtTex[2] = this.ambientTarget;

		for( int i = 0; i < this.mrtTex.Length; i++ )
		    this.mrtRB[i] = this.mrtTex[i].colorBuffer;
	}

In OnPreRender I do the following:

	void OnPreRender()
	{
		if( !this.enabled )
			return;

		this.cameraCopy.camera.CopyFrom( Camera.main.camera );

		RenderTexture lastActive = RenderTexture.active;
		Graphics.SetRenderTarget( mrtRB, mrtTex[0].depthBuffer );
		GL.Clear( false, true, Color.clear );
		this.cameraCopy.camera.RenderWithShader( this.gBuffersMaterial.shader, "" );
		RenderTexture.active = lastActive;
	}

Again, replacementTag is empty, causing a rendering of all the objects in the scene. My fragment shader looks as follows:

	struct PixelOutput
	{
		float4 pos : COLOR0;
		float4 normal : COLOR1;
		float4 ambient : COLOR2;
	};

	PixelOutput frag(v2f i) : COLOR {

		PixelOutput o;
		o.pos = float4( 1.0, 0.0, 0.0, 1.0 );
		o.normal = float4( 0.0, 1.0, 0.0, 1.0 );
		o.ambient = float4( 0.0, 0.0, 1.0, 1.0 );

		return o;
	}

Do you see any problems with that code? There is no output on the three set rendertargets whatsoever.

Thanks,
iko

[aubergine]

First and foremost, check your system with this:

print(SystemInfo.supportedRenderTargetCount);

Also, as i cant see your whole code, im not sure if you set copycameras target texture with

copycamera.targetTexture = to the rendered texture;

[Dolkar]

That’s it though… You can’t use a camera to render meshes to MRTs. The Graphics.SetRenderTarget works only for immediate rendering using Graphics.DrawMeshNow (possibly Graphics.DrawMesh as well). When you use camera.RenderWithShader, it internally overwrites the active render targets with camera.targetTexture. But you can’t set camera.targetTexture to an array of buffers!

So, as I said, the only solution is to draw the meshes yourself.

[iko79]

Hi again,

now, according to the Unity3D 4.3 release notes MRTs should be possible via Camera.SetTargetBuffers – and indeed it is. Unfortunately though, it seems only to work with up to 4 render targets, although my graphics card (GTX 780) should be able to handle 8 of them, at least according to GPU Caps Viewer, which reports “GL_MAX_DRAW_BUFFERS: 8”. I’m not entirely sure how this translates to DirectX and I don’t know how to doublecheck the max RT number in DirectX Caps Viewer, but I’m pretty sure it should be the same, right? But whenever I try to declare an output likefloat4 shininess : COLOR4Unity reports “Too many interpolators used (maybe you want #pragma glsl?) at line 25”. Inserting the pragma for glsl as suggested doesn’t help. I’m using shader model 3.0 by the way. What could be wrong here?

Thanks!

[Dolkar]

I noticed Unity as a developing tool focuses not on what your graphics card supports, but what the majority of them do. As far as I know, 8xMRT support is quite bleeding-edge and there aren’t even that many cases where you might need it.

[iko79]

Well, as soon as you do something like deferred rendering, you are. Since you have to store the material properties in different buffers, you reach the number of five quite fast. Perhaps there is a different was of doing what I’m trying to, which I’m missing?

[Pyromuffin]

Do you have an example of using setTargetBuffers with camera rendering? I’ve tried to figure it out before, and not had any luck with it.

[Dolkar]

If 4 buffers are not enough for you to use in a deferred shading renderer, you’re doing something wrong. I could fit everything comfortably into just 3 32-bit buffers.

[castor76]

Yes, some example with MRT using Camera.setTargetBuffers would be much appreciated.
Also is it possible to do MRT using surface shader?

[iko79]

I do this exactly as shown above. Just that, instead of Graphics.SetRenderTarget, I use Camera.SetTargetBuffers and Camera.RenderWithShader here.

camera.backgroundColor = Color.black;
camera.SetTargetBuffers( mrtRB, mrtTex[0].depthBuffer );
camera.clearFlags = CameraClearFlags.SolidColor;
camera.RenderWithShader( this.shader, "" );

For the pixel shader, I define multiple color outputs via the COLOR0 - COLOR3 semantics in the output struct:

struct PixelOutput
{
	float4 pos : COLOR0;
	float4 normal : COLOR1;
	float4 albedo : COLOR2;
	float4 spec : COLOR3;
};

Can’t help you with surface shaders, sorry…

[stevesan]

Make sure you switch the camera to be Forward. I haven’t thought it through, but there is probably a good reason why Deferred (which might be your Player Settings default) doesn’t work with MRT.

[PhobicGunner]

@iko79

Well, as soon as you do something like deferred rendering, you are. Since you have to store the material properties in different buffers, you reach the number of five quite fast. Perhaps there is a different was of doing what I’m trying to, which I’m missing?

You forget that three of those four have four different color channels - Red, Green, Blue, and Alpha.
Half the fun of deferred rendering is figuring out how to pack your information into those color channels

Let’s take a typical PBR deferred renderer. We’ll need:
Diffuse color (RGB)
Specular color (RGB)
Normals (RGB)
Surface roughness (monochrome)
Depth

At first glance you might think this needs 5 render textures.
But!
Notice that if we did that, Diffuse, Specular, and Normals would each have one channel left over (the alpha channel). Surface roughness furthermore has 3 whole channels being used for nothing.
So let’s pack Surface Roughness into the unused alpha channel for Specular. That reduces us down to 3 color buffers plus a depth buffer - Albedo(RGB), Normal(RGB), Surface(RGBA), Depth. That fits very comfortably into our 4 buffer limit, and still gives us two monochrome channels at our disposal (for instance, if you wanted to integrate screen space skin shading a la CryEngine you might put the skin diffusion factor into the alpha channel of Albedo).

Actually, you can get even more mileage out of this if you opt for compact normal storage (for instance the CryEngine-style spheremap method produces just two values as far as I can tell, which you can pack into R and G and therefore you end up with a full extra three channels at your disposal - the alpha channel of Albedo, and B+A of normals texture)

Keep in mind this isn’t a Unity limitation - ALL engines do tricky G-buffer packing stuff like this.