Okay, I'm not sure if I should ask the question here or over in some ATI Graphic Card related Forums, but as the Problem only appears in Unity 3D, I'll start here.
I have a new ATI 5870 Card... now, I have changed all the 3D Settings to Application specific settings in the Catalyst Control Center, but still in Unity no Anti-Aliasing is used, no matter what I choose in Unity as AA. This problem is only existing in unity though, when I start any game I can see AA is applied.
Is this a known Problem, does someone else expierienced the same problem and found a solution, or does anyone have an idea what exactly I'm doing wrong?
This problem has been such a pain for me, but I think I can provide an answer to this issue.
The state of Anti-aliasing in Unity
Unity has three render modes:
- Deferred - (Pre-pass) will render stuff in advance. Does not support AA.
- Forward - Pass whole parts in to do lighting, etc. For complex meshes, there's this fun anti-aliasing issue which generally creates white dots/noise/mess along your edges wherein the normals of the surface change too much over too small an area (can be demonstrated by moving far back or on dense meshes which smooth across the surface.
- Vertex Lit - Lighting is calculated at each vertex. AA is beautiful, but has other limitations.
What's up with this ATI card?
Different cards, drivers, configurations and settings were tested and here's the conclusion: The newest version of ATI's Catalyst Control Center is apparently borked to some extent with some 3D applications. If you remove Catalyst (or install drivers for another video card at the same time as Catalyst is installed apparently?), then you will experience anti-aliasing in Unity as described above.
Here's the best part: This problem can actually give you better anti-aliasing and can give you anti-aliasing when there shouldn't be. If you run your Unity-built app in full-screen (and go back out if you like), anti-aliasing will happen even in deferred rendering where it is not supported. This is neat, but less helpful. To truly take advantage of the anti-aliased awesome, you need a doctorate in the magic of Catalyst and/or a lot of luck because it seems that Catalyst isn't writing/applying its own settings properly or some such. If you go into the 3D settings and set it to custom settings and then in the All tab uncheck and check HDVision's Use Application Settings checkbox (there might be other settings that I missed) and then the awesome anti-aliasing should happen whether you are in full-screen or not. Changing the settings in the AA tab is apparently not the same as doing so in the All tab?
As an additional datapoint, anti-aliasing works perfectly on my 5870 using forward rendering in Unity on a Mac, so it sounds like a driver problem on Windows. Actually, I shouldn't say perfectly...the exception is web players, where no AA at all is used unless you switch to full-screen mode, if the web player allows that. However this seems to be a problem with OS X 10.6 on all cards, rather than the 5870 specifically. Stand-alones running in windowed mode use AA just fine, so it's not a full-screen issue.