HLSL bit shifting

Questions & Answers
1

I have a sprite that I am trying to enable palette swapping on. I’ve written a tool to iterate over the color values in the sprite and encode the colors palette index within the first(?) two bits in each color value:

ex: pixel index is 21 and the original color value was #a22633 (162, 38, 51)

1. convert 21 to 6 bits: 010101
2. separate those bits into three 2 bit segments: 01  01  01
3. drop the last two bits off each color value and replace them with the bits from each segment
            R: 162 => 161               10100010 => 10100001
            G: 38 => 37                   100110 =>   100101
            B: 51 => 49                   110011 =>   110001

The next part is where I’m struggling. I want to be able to read these values in and reconstruct the index inside my fragment shader, but I’m having some difficulty with reading the bits.

I’ve added the following function to my shader

int CheckBit(float f, float bit, int setValue) 
{
    float sample = f % bit;
    if (sample == 0) return 0;
    if (bit == 2 && sample == 1.0) return setValue;
    if (sample >= bit/2.0) return setValue;
    return 0;
}

and I’m using it like this:

float4 UnlitFragment(Varyings i) : SV_Target
{
    float4 mainTex = SAMPLE_TEXTURE2D(_MainTex, sampler_MainTex, i.uv);
    float4 pack = mainTex * 255.0f;
    int index = 0;
    index += CheckBit(pack.r, 2, 16);
    index += CheckBit(pack.r, 4, 32);
    index += CheckBit(pack.g, 2, 4);
    // etc ...

    // use the index value to sample color value from our palette
    float2 puv = float2(index / 64.0, 0); // palette is a 64x1 sprite
    float4 palette = SAMPLE_TEXTURE2D(_PaletteTex, sampler_PaletteTex, puv);
    palette.a = mainTex.a;
    palette.rgb *= mainTex.a;
    return palette;
}

This doesn’t appear to be generating the correct index values. I think my bit reading code is off, but I’ve tried a lot of other methods and none seem to work correctly. I feel like I’m missing something basic.

2

So if I read correctly, you are just trying to reconstruct the index from the packed texture? Assuming you are targeting SM4.0+ and have already correctly packed the index the way you say (2 bits into each colour channel), then you can use bitwise ops to mask and shift like so to reconstruct;

// Assuming the 6 bits are packed evenly and in segment order of BGR;
uint4 p = (uint4)pack;
// Mask the first 2 bits (3 = 0011) of each channel and shift/combine to get the index
int index = (p.r & 3u) | ((p.g & 3u) << 2u) | ((p.b & 3u) << 4u);
// Invert the mask to remove the packed index from the normal colour info
p.rgb &= ~3u;