Alpha blending in a shader different to readPixels

Graphics Shaders & Materials
1

I have a semi transparent shader on some objects which looks great in a render but when I encode it to a png via ReadPixels I get a different result and therefore its impossible to re-create the same image outside of Unity for example in photoshop. Is there a trick to fixing this?

So in a Unity render it looks like the second image and after saving and comping it looks like the top one. How do I make the top one look like the bottom one?

I can almost get what I want using “Screen” mode in photoshop but that messes up the non-transparent bits. Is there a way to modify the image alpha/colours before encoding it to png so it will comp the transparent bits like in screen mode but leave the rest untouched when just using a normal composite?

The shader code looks like this

Shader "My Shaders/XRay" {
    Properties {
        _Color ("Tint (RGB)", Color) = (1,1,1,1)
        _RampTex ("Facing Ratio Ramp (RGB)", 2D) = "white" {}
        _Tint ("Texture tint", Color) = (1,1,1,1)
        _Rle ("Rle color", Color) = (0.5,0.5,0.5,1)
    }
    SubShader {
    	Cull Off //turn this on if using Blend One One
        ZWrite Off
        Tags { "RenderType"="Opaque" "Queue" = "Transparent" "VisibleInDepth"="On" }
        //Blend One One
        Blend One OneMinusSrcColor
        Pass {
            
            CGPROGRAM 
            #pragma vertex vert
            #include "UnityCG.cginc" 
 
            struct v2f {
                V2F_POS_FOG;
                float4 uv : TEXCOORD0;
            };
            
            v2f vert (appdata_base v) {
                v2f o;
                PositionFog( v.vertex, o.pos, o.fog );
                float3 viewDir = normalize(ObjSpaceViewDir(v.vertex));
                o.uv = float4( abs(dot(viewDir,v.normal)), 0.5, 0.0, 1.0 );
                return o;
            }
            ENDCG 
 
            SetTexture [_RampTex] {constantColor[_Color] combine texture * constant} 
        }
    }
    // Fallback to Alpha Vertex Lit
	Fallback "Alpha/VertexLit", 2
}

196977--7161--$picture3_420.png
196977--7162--$picture_2_203.png

2

This is basically a limitation of Photoshop’s blending modes. If you can’t do it with the built-in modes, you’ll have to do your blending elsewhere. To help you discern whether you can use the built-in modes, take a look here:

(copy and paste due to forum limitations)

or here for a more programmatic approach:

http://www.nathanm.com/photoshop-blending-math/

3

I don’t think its a limitation of the blending modes I think its more a limition with how alpha and colour are stored when you grab the pixels from a render.
The issue is that I want normal users to be able to save images and then put them on any background without having to have extensive photoshop expertise, our users are doctors and surgeons not digital compositors :cry:

One solution I’ve toyed with but haven’t had time to try is to write my own post process that somehow takes the image without alpha and then creates the correct alpha and colour values for these objects, that could be a lot of work so I was hoping someone might have a quick trick that would do it.

4

I think I see what you mean now. If you want to save images with transparency, you need to have your shader record its alpha value without using any blending: Blend Off
Then make sure the clear colour for your camera has an alpha of zero, and when you use ReadPixels you’ll get the alpha channel straight from the fragment shader rather than from the blending function.

Of course, it will look wrong in Unity because you can’t see the alpha channel in the render target. The resulting PNG will look right, though. If you need it to look right both in Unity and in saved files, you’ll have to render with a different shader in each case.

5

cheers I’ll try that, that sounds like exactly what I want. I guess there is no way to dynamically switch the blend mode to off is there, you have to do it with a separate shader?

6

rats, that doesn’t work at all :cry: in fact it’s worse…

197587--7232--$picture_1_768.png

7

I’ve found a solution but its a horrible hack.
I basically do a screen overlay before encoding to a png and saving but only for pixels that have a hue that matches my transparent shader, this obviously only works because all the transparent bits are using the same shader and would instantly break if I use other colours. Still it will have to do for now.

8

What did you do to get the second image? It’s not clear to me what you’re showing. Shader code would be great, too.

9

Shader code is in the first post.

The key is

Blend One OneMinusSrcColor

The default for this Xray shader when downloaded is

Blend One One

But the issue with that blending mode is 2 fold, first I have the same issue when saving image i.e it’s not WYSIWYG but also inside the Unity player it doesn’t handle multiple layered surfaces overlapping very well as you get total burn out where the opaque bits overlay each other.

Using Blend One OneMinusSrcColor I was able to get a much more subtle xray effect where there isn’t any burnout. The issue then became how to let users save and image in a way that they could then comp them onto any colour or any other image and have it look just as nice. I.e same colours, and no burnout or darking as in the first example.

10

Ok, but when you use Blend Off on a transparent background, and use ReadPixels and save the result as a PNG, you should be getting a decent export that can then be composited in Photoshop using whatever you want (regular complementary alpha blending being the default). Is that not working?

Of course, it won’t look right in Unity because it won’t be blended properly, but it should look right in Photoshop, etc.

11

When I do that I don’t get any transparency on the xrayed objects so although the very front edges look right the internals don’t feature any overlapping surfaces. See the pic above that is on black. That image is a comp not what I get in Unity but what I get after saving.

12

Ok, I’d forgotten about overlapping. I’ve been thinking about this for a while and it’s confusing me. There must be a solution, but I’m not sure how to set up a shader so that the colour channel comes out properly. The alpha channel can be layered properly by starting with a transparent clear colour, and having the shader use:

Blend OneMinusSrcAlpha One

as the blending function.

13

Ok, I think I’ve got it!

This only works for alpha blending, but I think it works pretty well. That said, I haven’t tried it.

First, set your camera’s clear colour to (0,0,0,0).

Second, use a shader that uses two passes. The first pass uses:

ColorMask RGB
Blend SrcAlpha OneMinusSrcAlpha

The second pass uses:

ColorMask A
Blend One OneMinusSrcAlpha

Both passes perform the same texture operations and produce the same RGBA output.

Finally, you need to divide the colour channel of your render target by its alpha channel. This will remove the black component introduced by alpha blending onto a black background. You can do this on a pixel-by-pixel basis after using ReadPixels, but remember that a lot of pixels will still have an alpha value of zero, so just ignore those ones.

The result should have the correct values in all four channels, and will behave properly as a PNG. Even better, because the shader writes the colour channels normally, you don’t have to use a separate shader for displaying to the screen and for saving to PNG.

14

Close but no banana!

I’ve run out of time at the moment to play with this but hopefully there is a solution in there somewhere, I’ll try again hopefully next week. I can see your logic and it has given me some good ideas on what to try.

15

Did you try it? It worked as expected for me. Here is the modified shader:

Shader "FX/XRay Special Alpha" { 
	Properties { 
		_Color ("Tint (RGB)", Color) = (1,1,1,1) 
		_RampTex ("Facing Ratio Ramp (RGB)", 2D) = "white" {} 
	} 
	SubShader { 
		Cull Off //turn this on if using Blend One One 
		ZWrite Off 
		Tags { "RenderType"="Opaque" "Queue" = "Transparent" "VisibleInDepth"="On" } 
		Pass { 
			ColorMask RGB
			Blend SrcAlpha OneMinusSrcAlpha 
			
			CGPROGRAM 
			#pragma vertex vert 
			#include "UnityCG.cginc" 
  
			struct v2f { 
				V2F_POS_FOG; 
				float4 uv : TEXCOORD0; 
			}; 
			
			v2f vert (appdata_base v) { 
				v2f o; 
				PositionFog( v.vertex, o.pos, o.fog ); 
				float3 viewDir = normalize(ObjSpaceViewDir(v.vertex)); 
				o.uv = float4( abs(dot(viewDir,v.normal)), 0.5, 0.0, 1.0 );
				return o; 
			} 
			ENDCG 
  
			SetTexture [_RampTex] {constantColor[_Color] combine texture * constant} 
		} 
		Pass { 
			ColorMask A
			Blend One OneMinusSrcAlpha 
			
			CGPROGRAM 
			#pragma vertex vert 
			#include "UnityCG.cginc" 
  
			struct v2f { 
				V2F_POS_FOG; 
				float4 uv : TEXCOORD0; 
			}; 
			
			v2f vert (appdata_base v) { 
				v2f o; 
				PositionFog( v.vertex, o.pos, o.fog ); 
				float3 viewDir = normalize(ObjSpaceViewDir(v.vertex)); 
				o.uv = float4( abs(dot(viewDir,v.normal)), 0.5, 0.0, 1.0 );
				return o; 
			} 
			ENDCG 
  
			SetTexture [_RampTex] {constantColor[_Color] combine texture * constant} 
		} 
	}
}

And here is the script I used to capture the image and divide colour by alpha:

using UnityEngine;
using System.Collections;
using System.IO;

[RequireComponent(typeof(Camera))]

public class DivideAlpha : MonoBehaviour {
	
	public RenderTexture renderTexture;
	
	void TakePicture () {
		Texture2D texture = new Texture2D(renderTexture.width, renderTexture.height, TextureFormat.ARGB32, false);
		
		RenderTexture.active = renderTexture;
		texture.ReadPixels(new Rect(0, 0, renderTexture.width, renderTexture.height), 0, 0, false);
		texture.Apply();
		for (int x = 0; x < texture.width; x++) {
			for (int y = 0; y < texture.height; y++) {
				Color color = texture.GetPixel(x, y);
				float alpha = color.a;
				if (alpha != 0f) {
					color /= alpha;
					color.a = alpha;
					texture.SetPixel(x, y, color);
				}
			}
		}
		RenderTexture.active = null;
		
		byte[] bytes = texture.EncodeToPNG();
		File.WriteAllBytes(Application.dataPath + "/../SavedScreen.png", bytes);
		DestroyImmediate(texture);
	}
	
	protected IEnumerator Start() {
		yield return 0;
		TakePicture();
	}
}

Here is a picture of two spheres using the shader. As you can see, they composite properly even here on this forum.

16

That looks great, I did try it but it didn’t come out like that, the read and blue would have been black. I probably missed something in the code somewhere, I’m still working my way through shaderLab haven’t figured it all out yet. Just looking at the code quickly it looks the same so I’ll try out your version and see where I must have something wrong, at least I know it should work which is great.
Many thanks for taking the time out to get a correct solution this is really helpful.

17

Do you find it works in unity too? I think that assumption maybe wrong and I will need to return to the idea of swapping the shaders when outputting?

18

There’s something odd going on here, your posted image looks exactly like what I’m after (more or less) but when I use your code in my file it comes out much less like what I want, which was why i thought it wasn’t working.

This is a screen shot straight from a test file where I’ve tried to recreate your test. To me there is a black tint to the transparent parts which you don’t get. I can only think this is somehow something to do with the ramp image I use? I’m at a loss.

Below is a comparison of what I’m currently getting with your shader versus my current one and hacky screen overly fix.

198224--7282--$compare_431.jpg
198224--7282--$compare_431.jpg995×836 175 KB

198224--7283--$picture_1_643.png

19

The shader produces the same result in Unity. If you need a different coloured background for real-time display, you can just switch it to black while taking the picture. This can be done invisibly by using Camera.Render after setting RenderTexture.active to your target texture.

20

Are you using a ramp with black in it? Try this one instead:

(there’s a PNG there in case it’s not obvious)

Your ramp looks like it goes from opaque white to transparent black, whereas mine goes from opaque white to transparent white. Using your old blending method, you didn’t see the black because it was additive. With pure alpha blending, you’ll see any dark colours in your ramp as well.