Good day community, I’ve been searching for a solution, but couldn’t find one.
I am using the “Cutout” rendering option getting that nasty pixelated edge result.
Is there any way to get the edges anti-aliased?
There seems to be a problem with the textures because I am getting a similar result with my Java engine.
Unity Project:
Java Project:
Cutout is using alpha test, so the texture pixels are either fully transparent or fully opaque, no blending. You could use alpha blending but you’d likely have some sorting issues. One option is antialiasing w/ alpha-to-coverage (which masks the hard edge a bit) but I’m not actually sure if Unity supports it out of the box (probably not).
You can make a custom shader using AlphaToMask On (which enables alpha to coverage), but it doesn’t work with Surface Shaders correctly with out some careful touches and a custom shadow caster pass. It also kind of needs at least MSAA 4x enabled to be useful. If you’re using deferred you’re kind of out of luck going that route since MSAA and deferred don’t work together.
The common “solution” for this is using a post process anti-aliasing like FXAA or SMAA, or more recently a temporal AA. Another solution is to use a two pass shader of some kind that mixes traditional alpha blending and alpha test. Those are kind of common for hair shaders. I posted an example of one on the forum someplace…
