I've been trying to use the TextureMask shader available in unifycommunity : http://www.unifycommunity.com/wiki/index.php?title=TextureMask
But here is the result I get in unity. As you can see my mask texture is a simple psd file. But why the edges are so jagged?
Thanks,
I opened your package and tried it out. Two things weren't right for what you're trying to do:
1) You shouldn't use Alpha Testing in your shader but Alpha blending, like so:
Blend SrcAlpha OneMinusSrcAlpha
instead of
AlphaTest LEqual [_Cutoff]
2) The texture you use, as I found it in your package, doesn't have any alpha channel. I checked the 'Generate Alpha From Greyscale' box on the mask texture importer to sort it out, and now it's fine.
That's the result of alpha testing. Setting "Alpha cutoff" to zero should get rid of that.
solved our problem based on your sample file. Your PSD has a black background with a white radial on top, so for all effects there's no alpha in play there. What I did was transform that into a black with transparent. On Photoshop you should be able to see the checkered background, exported it as PNG-24bit and replaced the png as Culling Mask for "Plate"