RenderTexture Painting - color blending issues

I am trying to implement brush painting, very similar to what photoshop has. I have been “mostly” successful, except that I am having some issues with blending and I’m not really sure how this needs to be addressed.


Painting technique (C#)

RenderTexture.active = paintRenderTexture;

GL.PushMatrix();
GL.LoadPixelMatrix(0, paintRenderTexture.width, 0, paintRenderTexture.height);
Graphics.DrawTexture(rect, brushTexture, paintMaterial);
GL.PopMatrix();

RenderTexture.active = null;

Paint Shader

  • Shader used to render the brush in the render texture

    Shader “Unlit/TexturePainter/Blit”
    {
    Properties
    {
    _MainTex(“MainTex”, 2D) = “white” {}
    _Color(“Color”, Color) = (1,1,1,1)
    }
    SubShader
    {
    Tags { “RenderType” = “Opaque” }

          Pass
          {
              Blend One OneMinusSrcAlpha
              Lighting Off
              Cull Off
              ZTest Always
              ZWrite Off
    
              ...
    
              fixed4 frag(v2f i) : SV_Target
              {
                  fixed4 brush = tex2D(_MainTex, i.texcoord0) * _Color;
                  return fixed4(_Color.rgb * brush.a, brush.a);
              }
              ENDCG
          }
      }
    

    }


Preview shader

  • This shader is used to render the final image on the quad.

    Shader “Unlit/TexturePainter/Preview”
    {
    Properties
    {
    _MainTex( “Texture”, 2D ) = “white” {}
    }
    SubShader
    {
    Tags { “RenderType” = “Transparent” }

          Pass
          {
              Blend One OneMinusSrcAlpha
              ZWrite Off
    
              ...            
    
              fixed4 frag(v2f i) : SV_Target
              {
                  fixed4 col = tex2D(_MainTex, i.uv);
                  return fixed4(col.rgb, col.a);
              }
              ENDCG
          }
      }
    

    }


Blending issue GIF

  • If you focus on the area where I am overpainting on one spot, you can see the artifacts start forming around the edges of the brush texture (areas where alpha is not 100%)
    172835-fgi6yzhuxa.gif

Conclusion

I’ve also tried using separate blending modes for RGB and for A channels, but I was mostly unsuccessful. While I got rid of the issue I mentioned above, I had a different issue where alpha values weren’t blended correctly.
Is it even possible to achieve what I want using this technique, or is there another way this needs to be aproached?

Any help on this issue will be greatly appreciated :slight_smile:


Back before Unity 5 (and, therefore, before the “free” version supported many features like RenderTextures), I self-taught myself color blending by making a texture-painting script.

Why the preface?

It meant I was blending all the colors without using shaders.


The problem you're running into is that you're pre-multiplying the alpha into the color you're drawing onto your surface, so the edges (as you mentioned) of the brush wind up the wrong color. Pre-multiplying color is only important in the context of having no alpha channel support. Rather, pre-multiplying is how you get the resulting color to display on screen in certain context, but it's not universally necessary and definitely isn't necessary in Unity, where textures are (generally speaking) 32-bit, containing alpha data.

In regard to my aforementioned “old project”, the main function I used for complex color blending was:

// Transcribed from Unityscript, so apologies ahead of time for any typos
public static Color Blend(Color current, Color new)
{
	Color output;
	// Blended alpha values with a maximum of 1
	// Example: 0.5 background + 0.5 brush = 0.75 output alpha
	output.a = current.a + (new.a * (1.0f - current.a));
	if(output.a == 0.0f)
	{
		return new Color(current.r, current.g, current.b, 0f);
	}
	else
	{
		// Example: White to Black, 0.5 alpha on background (current) and brush (new)
		// ((0.0 * 0.5) / 0.75) + ((1.0 * 0.5 * (1-0.75)) / 0.75)
		// ((0) + ((0.5 * 0.25) / 0.75)
		// ~0.1666 final pixel channel color
		output.r = ((new.r * new.a) / output.a) + ((current.r * current.a * (1 - new.a)) / output.a);
		output.g = ((new.g * new.a) / output.a) + ((current.g * current.a * (1 - new.a)) / output.a);
		output.b = ((new.b * new.a) / output.a) + ((current.b * current.a * (1 - new.a)) / output.a);
	}
	return output;
}

This is made on the basis of blending a non-zero opacity new pixel color with any existing pixel, where the opacity of the existing pixel influences how much priority the color of the new pixel will have. This is also weighted with an additional step over a shader’s Blend SrcAlpha OneMinusSrcAlpha approach to transparency blending by dividing by the output color to determine the new resulting color with high accuracy, but again, this is just part of the process of doing all the blending manually.

Now, having said all that, when you’re drawing a texture on top of another one through a shader, you would generally be able to just rely on that blending. At a glance, and by my own personal impression, it looks like you have your role backwards for the brush, in that it’s your opaque shader. Furthermore, you’re also double-multiplying its values, by combining both tex2D(_MainTex, i.texcoord0) * _Color and fixed4(_Color.rgb * brush.a (factoring in the _Color value in both scenarios).

If you make your first shader Unlit/TexturePainter/Blit transparent (“RenderType” and “Queue”, most importantly), blend using Blend SrcAlpha OneMinusSrcAlpha and remove the rgb pre-multiplication (i.e. _Color), I imagine that would solve most of the problems in this situation.