I have 2 sets of pixel color arrays that represent images. I want to merge them like you might in a pixel shader using something similar to the following logic:
for (int x = 0; x < width; x++)
{
for (int y = 0; y < height; y++)
{
color[x+y*width] *= SampleImageAtPercent(x/(float)width,y/(float)height);
}
}
I am having trouble coming up clean logic to sample a group of pixels correctly. There are a lot of cases to account for like blending portions of neighbor pixels together when the images are unpleasant fractions of each other, direct copying if they are multiples of each other, and doing all this when say one is bigger on width but smaller on height then the other.
I tried googling around for how does tex2D (_MainTex, IN.uv_MainTex) in a shader work because thats essentially what I want but I wasn’t able to find anything
Are the two images changing often or can you do prepasses on them to speed up your lookup sampling?
If you can do some one-time work, you could try generating mipmaps for them and doing a lookup on them (trilinear if they are changing scale per frame or bilinear if not).
The key bit beyond bilinear sampling is you need to decide what sample rate (what resolution) you want the averaged values to be at. The easiest would be to use the largest width & height values as then you only need bilinear filtered sampling.
If you have a specific resolution that’s smaller than the max dimensions of either of the larger textures you’ll likely need to do something akin to trilinear filtered sampling. Trilinear sampling relies on mip maps, so using a more accurate filter like bicubic might be easier to use if the resolution you’re sampling at is no than half the resolution of the original textures, or you’re okay with a little aliasing. Beyond that you may need to look into Lanczos resampling, or sinc, but rescaling images is a deep rabbit hole to jump down.
Is there a reason to not just do it on the GPU with textures? Does this have to live on the CPU? And do the textures have to be of different dimensions?
I could create 2 textures and average them together every frame with a shader. I just thought since it could be done once and forget it, it might be better to create one new texture and just be done. Is there a better alternative?
Sampling two textures and averaging is stupendously cheap to do in a shader.
fixed4 a = tex2D(_TexA, i.uv);
fixed4 b = tex2D(_TexB, i.uv);
return (a + b) * 0.5;
Doing that every frame vs sampling a single texture is unlikely to be noticeably different performance wise unless it’s part of a much more complex shader or you’re running into memory limitations (bandwidth or usage).
It really depends on where the color arrays are coming from and what you needed the data for. If it really is just needing two images averaged together, and they already exist as compressed texture assets, doing this on the CPU is going to be quite wasteful as the resulting averaged image is going to be uncompressed and more expensive to use than sampling two compressed textures.
And you can still do it once on the gpu and baked it in a render texture, which would be faster anyway, and allows you test many blending in real time at edition time …