Alternate solutions to color mask without relying on the individual RGB channels?

So I just wanted to share a thought and see what you guys think. I was thinking of making a simple shader that replaces blocks of solid color on a texture, with another solid color. The reason is that I think this could be useful for iterating palette swaps of sprites more easily. Thing is, I’m having issues coming up with a solution for identifying specific color values. If I could settle for 4 colors, it’d be easy to control with RGBA.

To further illustrate what I want to do, take a look at this sprite color breakdown from skullgirls:

With that mask to the left, they can generate all of the alternate costumes for that character (Then I assume the shadows and highlights are overlayed with another sprite sheet or something similar).

Oh yeah, and to make things easier, let’s assume our mask values are hardcoded into our pipeline, so I already know I wanna use Red, Green, Blue, Cyan, Magenta, Yellow or something.

The closest solution I’ve thought of yet is just making an if statement to check the color - could work, but maybe one of you guys can think of a more cost effective way of doing it.

My advice is to never use something like a hard separation based on the color (as in an if statement), but always leave room to flow from one color into the next. Besides being able to use gradients in this way, it also offers a more stable color selection when the mipmaps are used or compression is enabled. You can go for a 8 color interpolation based on the 8 extreme points on the RGB cube. (And 16 if you add the alpha).

float3 mask = tex2D().rgb; // Input rgb mask colors
float2 mask_r = float2(mask.r, 1.0 - mask.r);
float2 mask_g = float2(mask.g, 1.0 - mask.g);
float2 mask_b = float2(mask.b, 1.0 - mask.b);
float4 mul1 = mask_r.yxyy * mask_g.yyxy * mask_b.yyyx; // Black, red, green, blue
float4 mul2 = mask_r.xyxx * mask_g.xxyx * mask_b.xxxy; // White, cyan, magenta, yellow
float3 result = mul1.x * color1;
result += mul1.y * color2;
...
result += mul2.w * color8;