Is stochastic transparency possible within unity? Making every second or so MSAA sample transparent. Can I identify which msaa sample fragment is drawing to? Could I use atomic counter to calculate which sample MSAA is writing to? Or is there better way to do it?
Would maybe UV coordinates help? splitting a pixel into a grid of 4x for 2x multisampling for example could determine which MSAA sample it is?
You’re overthinking the MSAA part of it. MSAA’s advantage is it only renders the fragment shader of an object once per pixel, but determines the coverage of that single fragment invocation with multiple coverage samples. You want to modify the coverage samples without forcing super sampling. You can do that with SV_Coverage.
To use that in any Unity shader requires using custom vertex fragment shaders. You can’t use Surface Shaders or Shader Graph, or any built-in shader as is. SV_Coverage is a uint where each bit sets if it should be covered or not. A few years ago I asked a few people how they set the values in SV_Coverage (or, more specifically, the OpenGL equivalent, gl_SampleMask).
I’ve used Inigo Quiles’ example code with success, though for my use case it created other artifacts on triangle borders. So instead of using the primitive ID I’ve used a 3D noise texture or the technique in this paper:
https://casual-effects.com/research/Wyman2017Improved/index.html
Indeed I want to avoid super sampling. Will take a look at SV_coverage, maybe that will do the trick. I want to mix OIT based on deep opacity map with stohastic transparency, really interesting topic to study 
thanks for the info.
If I understand correctly I can simply set alpha to 0.5 and it will automatically turn half of msaa samples off hence giving a look of 0.5 alpha? Cannot find any Sv_covarage examples.
If you’re using AlphaToMask On in your shader (which is Unity’s flag for enabling alpha to coverage), yes. But modern GPUs it’s the same 50% of coverage samples for every pixel, so you don’t get the benefits of stochastic transparency. Overlapping 50% alpha objects will completely cover each other. Older GPUs, like 8+ year old ones, used to do some amount of dithering or noise automatically with alpha to coverage, but not anymore.
If that’s all you want, then yes, you don’t need anything extra. Otherwise that twitter thread I linked to has two glsl examples that can be translated into hlsl.
Something like this:
Shader "Unlit/SV_Coverage"
{
Properties
{
_Color ("Color", Color) = (1,1,1,1)
_MainTex ("Texture", 2D) = "white" {}
[KeywordEnum(SV_Coverage, AlphaToMask)] _MSAA_ALPHA ("MSAA Transparency Mode", Float) = 0.0
}
SubShader
{
Tags { "Queue"="AlphaTest" "RenderType"="Opaque" }
LOD 100
Pass
{
AlphaToMask [_MSAA_ALPHA]
CGPROGRAM
#pragma vertex vert
#pragma fragment frag
#pragma target 5.0
#pragma shader_feature _ _MSAA_ALPHA_ALPHATOMASK
#include "UnityCG.cginc"
struct appdata
{
float4 vertex : POSITION;
float2 uv : TEXCOORD0;
};
struct v2f
{
float4 vertex : SV_POSITION;
float2 uv : TEXCOORD0;
};
fixed4 _Color;
sampler2D _MainTex;
float4 _MainTex_ST;
v2f vert (appdata v)
{
v2f o;
o.vertex = UnityObjectToClipPos(v.vertex);
o.uv = TRANSFORM_TEX(v.uv, _MainTex);
return o;
}
// adapted from https://www.shadertoy.com/view/4djSRW
float hash13(float3 p3)
{
p3 = frac(p3 * .1031);
p3 += dot(p3, p3.yzx + 33.33);
return frac((p3.x + p3.y) * p3.z);
}
fixed4 frag (v2f i
#if !defined(_MSAA_ALPHA_ALPHATOMASK)
, out uint coverage : SV_Coverage
#endif
) : SV_Target
{
fixed4 col = tex2D(_MainTex, i.uv) * _Color;
float noise = hash13(i.vertex.xyz - frac(_Time.y * float3(0.5, 1.0, 2.0)));
int MSAA = 4;
col.a = saturate(col.a * ((float)(MSAA + 1) / (float)(MSAA)) - (noise / (float)(MSAA)));
#if !defined(_MSAA_ALPHA_ALPHATOMASK)
int mask = (240 >> int(col.a * float(MSAA) + 0.5)) & 15;
int shift = (int(noise * float(MSAA-1))) & (MSAA-1);
coverage = (((mask<<MSAA)|mask)>>shift) & 15;
col.a = 1.0;
#endif
return col;
}
ENDCG
}
}
}
SV_Coverage
AlphaToMask
Note I’m using 3D noise here to dither the alpha, so you’re getting a little bit of the stochastic transparency even with the alpha to mask method.
2 Likes
Thanks! That’s exactly what I needed 
You are golden @bgolus !
Could you please explain this part a bit? What would be required to make it work with MSAA8 or MSAA2? What is the range of values for coverage?
I need to use this custom coverage to make my col.a available for cheep custom depth and it works nice with MSAA4, but I would like to understand it more.
And what about col.a = (col.a - _Cutoff) / max(fwidth(col.a), 0.0001) + 0.5, that you’ve recommend to use in one of yours replies elsewhere, is it needed here?
I’ll start with this one, since it’s a little easier to explain.
This sharpens a soft alpha gradient to the width of a single pixel which results in a similar anti-aliased edge to what MSAA produces on an actual geometry edge. This also helps avoid the issue Alpha to Coverage usually has with overlapping A2C geometry appearing to become “more transparent” as only the closest transparent surface of similar opacity will be visible.
See this comparison between traditional alpha test, a basic alpha to coverage implementation that just outputs the texture’s alpha unmodified, and an alpha to coverage shader that uses that sharpening code.
It comes from my article on Alpha to Coverage here:
https://bgolus.medium.com/anti-aliased-alpha-test-the-esoteric-alpha-to-coverage-8b177335ae4f
I take some pride in knowing that after I released this article, this technique has been adopted by several engines, including Unreal and Unity, as well as others.
The value of the coverage variable, or more specifically the SV_Coverage output, isn’t so much “a value” as it is a bit mask. By that I mean if you think of that uint as the bit values used to construct it, each bit is a “should this MSAA coverage sample be visible”. For MSAA 4x the allowed “values” are 0000, 0001, 0010, 0011, 0100, 0101, 0111, 1000, 1001, 1010, 1011, 1100, 1101, 1110, and 1111, or the binary for the uint values between 0 and 15. For MSAA 2x it’s 00, 01, 10, and 11 (0 through 3), MSAA 8x it’s 00000000 through 11111111, or 0 through 255 (which I won’t iterate all of the binary for here).
That coverage mask then gets "and"ed with the actual geometry coverage, so you still get normal geometry edge MSAA.
To make it work with an arbitrary MSAA level, you should be able to replace the two uses of 15 with uint(pow(2, MSAA) - 1).
3 Likes
Thank you Ben!
Just to clearify for others, the 240 should be replaced as well, so it can look like this:
uint maskRange = uint(pow(2, _MSAA) - 1);
int mask = ((maskRange * maskRange + maskRange) >> int(col.a * float(_MSAA) + 0.5)) & maskRange;
The number for MSAA8 can be found in previously attached twitter post.
This is very nice and clean answer, I’ve seen your webside with those images before, and it clearly makes difference when using AlphaToMask On. With this custom coverage approach though, I get this results (before=>after).
Do you have an idea please? Any modifications with _Cutout (and in our case even ted2Dlod sample (mipmap offset) and col.a = (col.a - _Cutoff) / max(fwidth(col.a), 0.0001) + 0.5) leads to transparency issues like this:
1 Like
@bgolus appologies for pinging you again, have you ever met with previously stated issue?
Are you using that alpha sharpening line with the dithering / stochastic transparency? First, the sharpening needs to be done to the texture alpha before any noise is applied, and second, generally you either use the sharpening or the stochastic transparency, not both!
I dont use stochastic transparency. Reason I use this custom coverage it to be able to use alpha channel of the SV_Target for custom depth (VR + no depth prepass => saving Opaque depth into alpha and using it in other passes). Something like this (Most basic shader without any VR stuff, but shows the problem):
Shader "Unlit/NewUnlitShader"
{
Properties
{
_MainTex ("Texture", 2D) = "white" {}
_Cutoff ("Cutoff", Range(0,1)) = 0.5
}
SubShader
{
Tags { "Queue" = "AlphaTest" "RenderType" = "TransparentCutout" }
Pass
{
CGPROGRAM
#pragma target 4.5
#pragma vertex vert
#pragma fragment frag
#include "UnityCG.cginc"
sampler2D _MainTex;
float4 _MainTex_ST;
float _Cutoff;
struct appdata
{
float4 vertex : POSITION;
float2 uv : TEXCOORD0;
};
struct v2f
{
float4 pos : SV_POSITION;
float2 uv : TEXCOORD0;
};
v2f vert (appdata v)
{
v2f o;
o.pos = UnityObjectToClipPos(v.vertex);
o.uv = TRANSFORM_TEX(v.uv, _MainTex);
return o;
}
fixed4 frag (v2f i, out uint coverage : SV_Coverage) : SV_Target
{
fixed4 col = tex2D(_MainTex, i.uv);
//fixed4 col = tex2Dlod(_MainTex, float4(i.uv, 0.0, clamp(mipLevel + _AlphaMip, 0, _MaxAlphaMip)));
col.a = (col.a - _Cutoff) / max(fwidth(col.a), 0.0001) + 0.5;
uint _MSAA = 8;
uint maskRange = uint(pow(2, _MSAA) - 1);
int mask = ((maskRange * maskRange + maskRange) >> int(col.a * float(_MSAA) + 0.5)) & maskRange;
int shift = (int(float(_MSAA-1))) & (_MSAA-1);
coverage = (((mask<<_MSAA)|mask)>>shift) & maskRange;
col.a = i.pos.z;
return col;
}
ENDCG
}
}
}
Are you just trying to avoid writing into the alpha channel? If so you should be able to just use a blend mode that still uses transparency, but doesn’t write into the alpha channel.
I haven’t tested with MSAA, but for non-MSAA, this works (alpha blending, but doesn’t write into alpha channel) :
Blend SrcAlpha OneMinusSrcAlpha, Zero One
Use
col.a = saturate((col.a - _Cutoff) / max(fwidth(col.a), 0.0001) + 0.5);
Though the rest of that code is actually for stochastic transparency, not just alpha to coverage. If that’s all you want you can simplify it down to this:
Shader "Unlit/sewyA2C"
{
Properties
{
_MainTex ("Texture", 2D) = "white" {}
_Cutoff ("Cutoff", Range(0,1)) = 0.5
}
SubShader
{
Tags { "Queue" = "AlphaTest" "RenderType" = "TransparentCutout" }
Pass
{
CGPROGRAM
#pragma target 4.5
#pragma vertex vert
#pragma fragment frag
#include "UnityCG.cginc"
sampler2D _MainTex;
float4 _MainTex_ST;
float _Cutoff;
struct appdata
{
float4 vertex : POSITION;
float2 uv : TEXCOORD0;
};
struct v2f
{
float4 pos : SV_POSITION;
float2 uv : TEXCOORD0;
};
v2f vert (appdata v)
{
v2f o;
o.pos = UnityObjectToClipPos(v.vertex);
o.uv = TRANSFORM_TEX(v.uv, _MainTex);
return o;
}
fixed4 frag (v2f i, out uint coverage : SV_Coverage) : SV_Target
{
fixed4 col = tex2D(_MainTex, i.uv);
col.a = saturate((col.a - _Cutoff) / max(fwidth(col.a), 0.0001) + 0.5);
uint _MSAA = 8;
coverage = (1u << uint(col.a * (half)_MSAA + 0.5)) - 1u;
col.a = i.pos.z;
return col;
}
ENDCG
}
}
}
1 Like
Noice, works like charm!
What does 1u mean?
Unsigned integer literal
https://learn.microsoft.com/en-us/windows/win32/direct3dhlsl/dx-graphics-hlsl-appendix-grammar
1 Like