Hey guys,
Unity 5.5 got this new orange outline around selected game objects in the Editor
Does anybody know if we can use this for drawing outline around objects in games?
Thank you
If you add the Effects package from the engine, the Post FX “Edge detection” have changed a lot. I suggest you to give it a shot.
nice, thank youI’ll check it out
Did you get it working?
3 Likes
Hi,
I’ve made an implementation of the Unity editor outline to be used in our game. I’m basing it on this post https://forum.unity3d.com/threads/selection-outline.429292/#post-2776318.
I’ve managed to get the outlines rendering but cant’ get the part right where the outlines should render with a desaturated color when the outline object is behind something.
I’ve found the problem that the first pass that renders the color mask does not obey the ZTest LEqual command and always renders all fragments to the red channel, even if the object is behind something.
// #0: things that are visible (pass depth). 1 in alpha, 1 in red (SM2.0)
Pass
{
Blend One Zero
ZTest LEqual
Cull Off
ZWrite Off
Offset -0.02, 0
CGPROGRAM
#pragma vertex vert
#pragma fragment frag
#pragma target 2.0
float _ObjectId = 1;
#define DRAW_COLOR float4(1,1,1, 1)
#include "SceneViewSelected.cginc"
ENDCG
}
So, the red channel is always filled with the value 1 for all outline objects, even if it is behind something, thus i do not have the information i need in the postprocess shader to desaturate the outline.
If i understand it right it should not have a value at all (red=0) if id does not pass the ZTest, right?
In the image below you can see the output from the red channel. As you can see all fragments pass the ZTest, which should not be the case.
In the image below you can see the final postprocess pass that renders the outlines. If i had the right information in the red channel i would be able to desaturate the outline color for the pixels that are ocluded.
Below is the Outline script that is attached to the main camera. It creates a temporary camera to render the maskbuffer and the applies the postprocess passes i.e blurring and then combining the result.
using UnityEngine;
using System.Collections;
public class OutlinePosteffect : MonoBehaviour
{
Camera AttachedCamera;
public Shader Post_Outline;
public Shader DrawSimple;
Camera TempCam;
Material Post_Mat;
void Start ()
{
AttachedCamera = GetComponent<Camera>();
TempCam = new GameObject().AddComponent<Camera>();
TempCam.enabled = false;
Post_Mat = new Material(Post_Outline);
}
//Subtract the original image from the blurred image
void OnRenderImage(RenderTexture source, RenderTexture destination)
{
//set up a temporary camera
TempCam.CopyFrom(AttachedCamera);
TempCam.clearFlags = CameraClearFlags.Color;
TempCam.backgroundColor = Color.black;
//cull any layer that isn't the outline
TempCam.cullingMask = 1 << LayerMask.NameToLayer("Outline");
RenderTextureFormat rtFormat = RenderTextureFormat.ARGBHalf;
if (!SystemInfo.SupportsRenderTextureFormat(RenderTextureFormat.ARGBHalf))
rtFormat = RenderTextureFormat.ARGB32;
//make the temporary rendertexture
RenderTexture TempRT = new RenderTexture(source.width, source.height, 0, rtFormat);
//put it to video memory
TempRT.Create();
//set the camera's target texture when rendering
TempCam.targetTexture = TempRT;
// Outline process:
// 1. render selected objects into a mask buffer, with different colors for visible vs occluded ones (using existing Z buffer for testing)
TempCam.RenderWithShader(DrawSimple,"");
// 1. End
// 2. blur the mask information in two separable passes, keeping the mask channels
RenderTexture horizontalBlur = new RenderTexture(source.width, source.height, 0, rtFormat);
horizontalBlur.Create ();
Post_Mat.SetVector ("_BlurDirection", new Vector2 (1, 0));
Graphics.Blit(TempRT, horizontalBlur,Post_Mat,0);
RenderTexture verticalBlur = new RenderTexture(source.width, source.height, 0, rtFormat);
verticalBlur.Create ();
Post_Mat.SetVector ("_BlurDirection", new Vector2 (0, 1));
Graphics.Blit(horizontalBlur, verticalBlur,Post_Mat,0);
// 2. End
// 3. blend outline over existing scene image. blurred information & mask channels allow computing distance to selected
// object edges, from which we create the outline pixels
Graphics.Blit(verticalBlur, destination, Post_Mat, 1);
// 3. End
//release the temporary RT
TempRT.Release();
horizontalBlur.Release ();
verticalBlur.Release ();
}
}
After some reading in other post’s it seems that if you are using a secondary camera(like i do) it will not share the depthbuffer with the first camera, and thats why the ZTest does not work. But it would be nice if someone from Unity could confirm this.
Every camera have his own depthbuffer and it will not be shared indeed.
Hmm, ok. So could anyone maybe shed some light on how i could make the mask camera ZTesting against the first camera depth?
Or explain in more depth how they do it in the Editor explained in short here https://forum.unity3d.com/threads/selection-outline.429292/#post-2776318.
// 1. render selected objects into a mask buffer, with different colors for visible vs occluded ones (using existing Z buffer for testing)
I have still not been able to solve this. Posting my code in the hopes that someone will be able to help
OutlinePosteffect.cs - Attach this to the main camera.
using UnityEngine;
using System.Collections;
using System.Collections.Generic;
public class OutlinePosteffect : MonoBehaviour
{
Camera _mainCamera;
Camera _maskCamera;
RenderTexture _maskRT;
RenderTexture _blurRT;
public Shader Post_Outline;
public Shader DrawSimple;
public Material _postMaterial;
void Awake()
{
_blurRT = new RenderTexture(Screen.width, Screen.height, 0, RenderTextureFormat.Default);
_maskRT = new RenderTexture(Screen.width, Screen.height, 0, RenderTextureFormat.Default);
_mainCamera = GetComponent<Camera>();
_maskCamera = new GameObject("MaskCamera").AddComponent<Camera>();
_maskCamera.enabled = false;
_postMaterial = new Material(Post_Outline);
}
void OnRenderImage(RenderTexture source, RenderTexture destination)
{
_maskCamera.CopyFrom(_mainCamera);
_maskCamera.backgroundColor = Color.black;
_maskCamera.clearFlags = CameraClearFlags.Nothing;
_maskCamera.cullingMask = 1 << LayerMask.NameToLayer("Outline");
RenderTexture activeRT = RenderTexture.active;
RenderTexture.active = _maskRT;
GL.Clear(true, true, Color.clear);
RenderTexture.active = activeRT;
// 1. render selected objects into a mask buffer, with different colors for visible vs occluded ones (using existing Z buffer for testing)
// Passing the source.depthBuffer does not work. I.e it does not use it in the ZTest like i think it should?
_maskCamera.SetTargetBuffers(_maskRT.colorBuffer, source.depthBuffer);
_maskCamera.RenderWithShader(DrawSimple, "");
// 1. End
// 2. blur the mask information in two separable passes, keeping the mask channels
_postMaterial.SetVector("_BlurDirection", new Vector2(0, 1));//Vertical
Graphics.Blit(_maskRT, _blurRT, _postMaterial, 0);
_postMaterial.SetVector("_BlurDirection", new Vector2(1, 0));//Vertical
Graphics.Blit(_blurRT, _maskRT, _postMaterial, 0);
// 2. End
// 3.blend outline over existing scene image.blurred information &mask channels allow computing distance to selected
// This is the #1: final postprocessing pass in PostOutline.shader. Right now i just substract mask channel(col.r) from blurred channel(col.b) to get the outline
Graphics.Blit(_maskRT, destination, _postMaterial, 1);
_maskRT.DiscardContents();
_blurRT.DiscardContents();
}
}
DrawSimple.shader - Drag it to the “Draw Simple” slot on the camera that has the OutlonePosteffect.cs
Shader "Custom/DrawSimple"
{
SubShader
{
// #0: things that are visible (pass depth). 1 in alpha, 1 in red (SM2.0)
Pass
{
//One = The value of one - use this to let either the source or the destination color come through fully.
//Zero = The value zero - use this to remove either the source or the destination values.
Blend One Zero
//Only render pixels whose reference value is less than or equal to the value in the buffer.
ZTest LEqual
//Off = Disables culling - all faces are drawn. Used for special effects.
Cull Off
//Controls whether pixels from this object are written to the depth buffer (default is On). If you’re drawng solid objects, leave this on.
//If you’re drawing semitransparent effects, switch to ZWrite Off. For more details read below.
ZWrite Off
// push towards camera a bit, so that coord mismatch due to dynamic batching is not affecting us
Offset -0.02, 0
CGPROGRAM
#pragma vertex vert
#pragma fragment frag
#pragma target 2.0
float _ObjectId = 1;
#define DRAW_COLOR float4(1,1,1, 1)
#include "SceneViewSelected.cginc"
ENDCG
}
// #2: all the things, including the ones that fail the depth test. Additive blend, 1 in green, 1 in alpha (SM2.0)
Pass
{
//Additive Blending
Blend One One
//Use the larger of source and destination.
BlendOp Max
//Always passes
ZTest Always
//Controls whether pixels from this object are written to the depth buffer (default is On). If you’re drawng solid objects, leave this on.
//If you’re drawing semitransparent effects, switch to ZWrite Off. For more details read below.
ZWrite Off
//Off = Disables culling - all faces are drawn. Used for special effects.
Cull Off
//Set color channel writing mask. Writing ColorMask 0 turns off rendering to all color channels.
//Default mode is writing to all channels (RGBA), but for some special effects you might want to leave certain channels unmodified, or disable color writes completely.
ColorMask GBA
// push towards camera a bit, so that coord mismatch due to dynamic batching is not affecting us
Offset -0.02, 0
CGPROGRAM
#pragma vertex vert
#pragma fragment frag
#pragma target 2.0
float _ObjectId;
#define DRAW_COLOR float4(0, 0, 1, 1)
#include "SceneViewSelected.cginc"
ENDCG
}
}
}
PostOutline.shader - Drag it to the “Post_Outline” slot on the camera that has the OutlinePosteffect.cs
Shader "Hidden/SceneViewSelected"
{
Properties
{
_MainTex ("Main Texture", 2D) = "white" {}
_Cutoff ("Alpha cutoff", Range(0,1)) = 0.01
}
SubShader
{
CGINCLUDE
#include "UnityCG.cginc"
struct Input
{
float4 position : POSITION;
float2 uv : TEXCOORD0;
float4 projPos : TEXCOORD1;
};
struct Varying
{
float4 position : SV_POSITION;
float2 uv : TEXCOORD0;
float4 projPos : TEXCOORD1;
};
Varying vertex(Input input)
{
Varying output;
output.position = mul(UNITY_MATRIX_MVP, input.position);
output.uv = input.uv;
output.projPos = ComputeScreenPos(output.position);
return output;
}
ENDCG
Tags { "RenderType"="Opaque" }
// #0: separable blur pass, either horizontal or vertical
Pass
{
ZTest Always
Cull Off
ZWrite Off
CGPROGRAM
#pragma vertex vertex
#pragma fragment fragment
#pragma target 2.0
#include "UnityCG.cginc"
float2 _BlurDirection = float2(1,0);
sampler2D _MainTex;
float4 _MainTex_TexelSize;
// 9-tap Gaussian kernel, that blurs green & blue channels,
// keeps red & alpha intact.
static const half4 kCurveWeights[9] = {
half4(0,0.0204001988,0.0204001988,0),
half4(0,0.0577929595,0.0577929595,0),
half4(0,0.1215916882,0.1215916882,0),
half4(0,0.1899858519,0.1899858519,0),
half4(1,0.2204586031,0.2204586031,1),
half4(0,0.1899858519,0.1899858519,0),
half4(0,0.1215916882,0.1215916882,0),
half4(0,0.0577929595,0.0577929595,0),
half4(0,0.0204001988,0.0204001988,0)
};
half4 fragment(Varying i) : SV_Target
{
float2 step = _MainTex_TexelSize.xy * _BlurDirection;
float2 uv = i.uv - step * 4;
half4 col = 0;
for (int tap = 0; tap < 9; ++tap)
{
col += tex2D(_MainTex, uv) * kCurveWeights[tap];
uv += step;
}
return col;
}
ENDCG
}
// #1: final postprocessing pass
Pass
{
ZTest Always
Cull Off
ZWrite Off
Blend SrcAlpha OneMinusSrcAlpha
CGPROGRAM
#pragma vertex vertex
#pragma fragment fragment
#pragma target 2.0
#include "UnityCG.cginc"
sampler2D _MainTex;
float4 _MainTex_TexelSize;
half4 fragment(Varying i) : SV_Target
{
half4 col = tex2D(_MainTex, i.uv.xy);
float saturateFac = 10;
float alpha = saturate((col.b - col.r) * saturateFac);
half4 outline = half4(1, 0, 0, alpha);
return outline;
}
ENDCG
}
}
}
SceneViewSelected.cginc - Paste this code in a text editor and save the file in your project as “SceneViewSelected.cginc”
#ifndef DRAW_COLOR
#define DRAW_COLOR 1
#endif
#include "UnityCG.cginc"
sampler2D _MainTex;
float4 _MainTex_ST;
float _DoClip = 0;
fixed _Cutoff = 1.0;
struct appdata_t
{
float4 vertex : POSITION;
float2 texcoord : TEXCOORD0;
};
struct v2f
{
float4 vertex : SV_POSITION;
float2 texcoord : TEXCOORD0;
};
v2f vert (appdata_t IN)
{
v2f OUT;
OUT.vertex = UnityObjectToClipPos(IN.vertex);
OUT.texcoord = TRANSFORM_TEX(IN.texcoord, _MainTex);
return OUT;
}
fixed4 frag (v2f IN) : SV_Target
{
if (_DoClip)
{
fixed4 col = tex2D( _MainTex, IN.texcoord);
clip(col.a - _Cutoff);
}
return DRAW_COLOR;
}
I’m very near to a deadline at work, I don’t have much time to help you but you should create an UnityPackage, it will be much more easy for me or someone else to have a fast check of what you have done.
As far as I understand, you want to have an Outline rendering when your mesh is behind another mesh.
I did something similar yesterday on a personal project but it was not an ouline (just changing color):
Please ignore the minecraft textures used for my test…
The walls are not sprites but meshes generated via a Geometry Shader with ZWrite on
In this usage the character is a sprite shader with a second pass doing only this:
Pass
{
Name "FORWARDOCC"
Tags
{
"Queue" = "Transparent"
"IgnoreProjector" = "True"
"RenderType" = "TransparentCutout"
"PreviewType" = "Plane"
"CanUseSpriteAtlas" = "True"
}
Cull Off
Lighting Off
ZWrite Off
ZTest Greater
Blend One OneMinusSrcAlpha
CGPROGRAM
#pragma vertex vert
#pragma fragment frag
#pragma multi_compile _ PIXELSNAP_ON
#include "UnityCG.cginc"
struct appdata_t
{
float4 vertex : POSITION;
float4 color : COLOR;
float2 texcoord : TEXCOORD0;
};
struct v2f
{
float4 vertex : SV_POSITION;
fixed4 color : COLOR;
half2 texcoord : TEXCOORD0;
};
fixed4 _Color;
fixed4 _OccludedColor;
sampler2D _MainTex;
v2f vert( appdata_t IN )
{
v2f OUT;
OUT.vertex = mul( UNITY_MATRIX_MVP, IN.vertex );
OUT.texcoord = IN.texcoord;
OUT.color = IN.color * _Color;
#ifdef PIXELSNAP_ON
OUT.vertex = UnityPixelSnap( OUT.vertex );
#endif
return OUT;
}
fixed4 frag( v2f IN ) : SV_Target
{
fixed4 c = tex2D( _MainTex, IN.texcoord );
clip( c.a - .5 );
return _OccludedColor;
}
ENDCG
}
Basically what did the trick was this:
ZWrite Off
ZTest Greater
Blend One OneMinusSrcAlpha
[EDIT]
Your problem here is the fact that the outline is a PostEffect and not the shader of the object itself.
I don’t know if there is a reasonable trick to achieve this though since your depth buffer only give you the depth in screen-space.
An outline per object may be a better solution ?
The outline is rendering when the mesh is behind another mesh, just as i want it. The problem is that i want to change the saturation or alpha on the outline when that happends (Just like in the editor). In order for me to do that this part needs to work
// 1. render selected objects into a mask buffer, with different colors for visible vs occluded ones (using existing Z buffer for testing)
But the second camera that is rendering the mask does not use the “main” cameras depthbuffer for the ZTest, thats why it fails, it always passes the ZTest so no different colors in the mask. I’ve read numerous other posts about similair problems and found that this should force the mask camera to use the same depth
_maskCamera.SetTargetBuffers(_maskRT.colorBuffer, source.depthBuffer);
But i cant get that to work either 
I will prepare a unitypackage so that anyone can easaly test it.
Thanks
You did not send any package. Were you finally able to recreate the editor effect ?
1 Like
I don’t get what you guys are doing. I’m trying to get a red outline around a selected object just like you see in the Unity editor. I want it to work regardless of how many materials (or which shaders) an object uses (just like the editor successfully does). I have looked at Kristopher’s code. I don’t care about saturation or alpha at all. I just want to get an outline. I see nowhere in the script where you give it the object(s) you want outlined. Is it supposed to just do things that are hidden behind something? What if I want the outline around just one object in my MMO? And I do want the outline to be culled by walls, etc.
I actually thought I got it working at one point when I had the player paused but I had selected my object by accident and Unity had displayed the outline. DANG IT… I WANT that! 
Ye, I tried the above 4 scripts too but there seems to be some more setup required.
You need to add a Outline layer to the layers cause it gets called in OutlinePosteffect. I don’t know where to assign it too tho, the meshes I presume?
The maskcamera is disabled when I run playmode and I just see a black screen.
In the editor the meshes that contain the shader are just not visible to the camera either.
@KristoferBoman I just tried this out too and can’t seem to get it working… Similar issues as others have said.
To answer the last 3 posts you guys must place the objects you want to outline into a Layer named “Outline”, you will have to create it via the editor.
1 Like
Yeah, I had done that but it still didn’t work for me…
Yeah actually it does’nt work out of the box: you have to change the OutlinePostEffect.cs this way:
using UnityEngine;
using System.Collections;
public class OutlinePosteffect : MonoBehaviour
{
Camera _mainCamera;
Camera _maskCamera;
RenderTexture _maskRT;
RenderTexture _blurRT;
public Shader Post_Outline;
public Shader DrawSimple;
public Material _postMaterial;
void Awake()
{
_blurRT = new RenderTexture(Screen.width, Screen.height, 0, RenderTextureFormat.Default);
_maskRT = new RenderTexture(Screen.width, Screen.height, 0, RenderTextureFormat.Default);
_mainCamera = GetComponent<Camera>();
_maskCamera = new GameObject("MaskCamera").AddComponent<Camera>();
_maskCamera.enabled = false;
_postMaterial = new Material(Post_Outline);
}
void OnRenderImage(RenderTexture source, RenderTexture destination)
{
_maskCamera.CopyFrom(_mainCamera);
_maskCamera.backgroundColor = Color.black;
_maskCamera.clearFlags = CameraClearFlags.Nothing;
_maskCamera.cullingMask = 1 << LayerMask.NameToLayer("Outline");
RenderTexture activeRT = RenderTexture.active;
RenderTexture.active = _maskRT;
GL.Clear(true, true, Color.clear);
RenderTexture.active = activeRT;
// 1. render selected objects into a mask buffer, with different colors for visible vs occluded ones (using existing Z buffer for testing)
_maskCamera.targetTexture = _maskRT;
//_maskCamera.SetTargetBuffers(_maskRT.colorBuffer, source.depthBuffer);
_maskCamera.RenderWithShader(DrawSimple, "");
// 1. End
// 2. blur the mask information in two separable passes, keeping the mask channels
_postMaterial.SetVector("_BlurDirection", new Vector2(0, 1));//Vertical
Graphics.Blit(_maskRT, _blurRT, _postMaterial, 0);
_postMaterial.SetVector("_BlurDirection", new Vector2(1, 0));//Vertical
Graphics.Blit(_blurRT, _maskRT, _postMaterial, 0);
// 2. End
// 3.blend outline over existing scene image.blurred information &mask channels allow computing distance to selected
// This is the #1: final postprocessing pass in PostOutline.shader. Right now i just substract mask channel(col.r) from blurred channel(col.b) to get the outline
Graphics.Blit(source, destination);
Graphics.Blit(_maskRT, destination, _postMaterial, 0);
_maskRT.DiscardContents();
_blurRT.DiscardContents();
}
}
Kristofer forgot to Blit the source image to the destination so it did not render correctly
Hmm… I’m guessing his code is changed in some way to not work exactly like the editor outline? With your code I am actually able to see the objects but, the background is pure black (no skybox or such), and I can only see the slightest hint of an outline color on the edges, not a bold outline like the editor outline.
I have attached the bare-minimum of a test project to show what I’m now getting. So, it seems it’s getting there, but not quite fully there yet.
I am even trying to use the unmodified version from this post , but it seems that one just makes the entire screen pink. 
Thanks for your help!
3348501–261807–OutlineTest.zip (21.3 KB)