I’m using the alpha channel as a color lookup index and just found out that I need a hair more than 256 values, which means I need to get the extra bit from somewhere else. I figure since the human eye is least sensitive to red I can steal the high-order bit from it by writing a quick script to map all red values from 0-255 to 0-127 and save the resulting image. When it comes time to render I simply extract the low 7 bits, double the value, and render it, piece of cake. The problem is, how do I a) extract the low 7 bits and b) extract the high-order bit so I can append it to the alpha to create a value from 0-511? I can easily do bitwise manipulations in C#, but I don’t see any way to do it in a shader.
Will solution a actually work? I’ve worked it out in my head and it doesn’t compute, so maybe I’m missing something.
For example, for simplicity sake let’s say that instead of the usual 8 bits we have 3 bits (bits 1, 2, and 4 for a total of 8 values) represent the range from 0-1 :
First, I assume 0.5 is half of 1.0, and therefore 3/7 is half of 7/7, so 0.5 == 011 = 3 = 3/7.
Let’s take 111 as input (I expect to get 011 as a result) : 7/7 mod 3/7 = 1/7 = 001, which is not the 011 we expected. Already we are wrong (correct me if I’m wrong, lol).
However, you put me on the right track. All I need to do is check to see if the input is > 0.5. If true, I subtract 0.5 from it to get the low 7 bits; if false, I use the input as is.
I don’t know whether it will actually work. What exactly are you doing? Reading the alpha channel from a 24-bits framebuffer?
I’m not sure but the mapping of the framebuffer value A from 0 to 255 to a floating-point number a from 0 to 1.0 might be: a = A/255 or a = A/256 (or even a = A/255.5). I assumed it is the second variant, but that’s probably not correct.
Yes, I believe that it’s 0-255 (or 8 bits) but it was simpler to demo the issue in 3 bits instead. Also, although Cg theoretically supports bitwise operators, none of the profiles available implement them. In any case, I believe I have the answer, which I already submitted at the end of my previous post.
