Traditional complementary alpha blending (like what you get from the factors you mentioned above) is just a linear interpolation between foreground and background by foreground alpha:
output = lerp(foreground.rgb, background.rgb, foreground.a)
So at a high level, what you want is:
output = lerp(texture1.rgb, background.rgb, texture1.a)
output = lerp(texture2.rgb, output.rgb, texture2.a)
output = lerp(texture3.rgb, output.rgb, texture3.a)
Unfortunately, you can’t apply the math so straightforwardly in a shader without use a GrabPass. The issue is that all your calculations depend on background.rgb, which you don’t have access to until the blending stage.
Luckily, there is a compositing trick called premultiplied alpha which allows us to do everything we want without knowing the background colour until blending time. Premultiplied alpha textures have their RGB channels already multiplied by their alpha before being sent to the shader. With premultiplied alpha, your standard blending equation becomes:
output = background.rgb*(1 - foreground.a) + foreground.rgb
Note that in an actual single-texture shader, you would use Blend One OneMinusSrcAlpha to accomplish both the multiplication and addition.
Composing this with three textures results in:
output = ((background.rgb*(1 - texture1.a) + texture1.rgb)*(1 - texture2.a) + texture2.rgb)*(1 - texture3.a) + texture3.rgb
We can expand this to:
output = background.rgb*(1 - texture1.a)(1 - texture2.a)(1 - texture3.a)
- texture1.rgb*(1 - texture2.a)*(1 - texture3.a)
- texture2.rgb*(1 - texture3.a)
- texture3.rgb
This neatly isolates the background colour and its factors, allowing you to add them afterward by putting the destination factors into your fragment shader’s alpha output and using the inverted premultiplied alpha blend coefficients of One SrcAlpha.
If you look back at the definition of premultiplied alpha, you might also notice that it isn’t strictly necessary in order to achieve the correct result. You can expand each of the textureN.rgb terms and do the alpha multiplication in the shader.