So I have a question about texture sampling I understand it as:
Point sampling = 1 sample
Bilinear = 4 Samples
Trilinear = 8 Samples
Anisotropic = 8 x AnisoLevel
So if I have forced on Anisotropic is the bilinear / trilinear overridden? And if I set it to point sampling is Anisotropic disabled??
And lastly is anisotropic sampling used for every pixel or just the angled ones and what is the cost for pixels where it is not needed?
Roughly accurate for Point, Bilinear, and Trilinear.*
Anisotropic is more complicated. Depending on the platform / GPU / API the point / bilinear / trilinear and anisotropic are not actually mutually exclusive. For some combinations anisotropic forces something like trilinear-like filtering, but on others you can actually have anisotropic point filtering. But we’re going to ignore those and focus on trilinear-like anisotropic which is the more common case.
If you’re talking about the reference implementation of Anisotropic filtering, then yes, it’s effectively 8xAniso. However in reality it never is.
Every GPU implements it’s own proprietary version of Anisotropic filtering that looks to match the visual results with an approximation. What exactly that approximation is, are a trade secret for each company that is not revealed publicly. But there are some common things that can be gleaned from just looking at the resulting images on different GPUs.
First thing is the number of extra samples is slope dependent. Some reference implementations also do this, as it’s obviously wasteful to do anisotropic filtering on a texture that’s perfectly aligned with the camera plane. So the full sample count will generally only happen on faces being viewed at an angle. Though it should be noted that “slope” here doesn’t necessarily mean the angle of the surface, but how stretched the texture itself is across a surface, so a surface that’s aligned to the camera plane might still get a lot of additional anisotropic samples if the texture is stretched out non-uniformly across that surface. A perhaps interesting aspect of the slope based sample limit is it’s independent of the aniso level. The slope sample limit and aniso level can be thought of two separate limits, so when rendering a floor with Aniso 8x or Aniso 16x, they’ll behave identically up until that 8x “cap”, and only the extreme slops beyond that will look better with 16x.
You can see that behavior in this gif where each step increases the quality of the texture filtering in the distance, but the foreground doesn’t improve in quality any more than the previous aniso level.
The next thing is I mentioned they do an approximation. And I mentioned exactly what they do is proprietary and trade secrets. However we can still infer a little bit of what’s going on. In the above image if you look at the center line you can see it’s not perfectly sharp. It kind of “bulges” out and gets blurry a few times along the length. That shouldn’t happen with proper anisotropic filtering. Some common alternatives to the reference anisotropic filtering are EWA (Elliptically Weighted Average) sampling and FELINE (Fast Elliptical Lines), both of which produce better quality filtering compared to basic anisotropic filtering. I’ve had some people tell me internal implementations are closer to EWA, but simplified so the quality is kind of “almost as good as straight anisotropic” while letting them use fewer samples.
One last thing to be mindful of is each “sample” being done by the hardware is far cheaper than sampling from a shader. So the cost of 128 samples for anisotropic filtering (if any GPU actually used the reference implementation) would be way cheaper than doing 128 point samples from the shader. The expensive part is the memory access, and because hardware texture units work on blocks of texture memory that is cached, most if not all of those samples can be cached at the same time for all of those samples. You can benchmark on modern hardware if you want, but 16x Anisotropic filtering ceased to be a performance concern for desktop GPUs over bilinear filtering nearly a decade ago.
Thanks a lot for the comprehensive answer! Also if I sample a texture in the vertex shader I guess point sample and billinear are the only options with tex2dlod ?
Btw I hope someone is paying you for your support to the community, you are literally everywhere shader related!
Point, Bilinear, and Trilinear all work with tex2Dlod(). Technically anisotropic filtering “works” in that you can have a texture with anisotropic enabled and sample it with tex2Dlod(), but it won’t be doing any of the anisotropic filtering and it’ll be no different than trilinear as there’s no way to calculate the slope.