How to correctly do an alpha channel without jagged edges?

Unity Engine
1

I have a character mesh and on top of that is is another mesh used for clothing. Within a Substance I have a variety of alphas using a switch, to select between the dress styles. The Alpha works when using Cutout in the dropdown on Unity 5’s Standard Shader, but it has jagged edges. If I switch to Transparent, it also works but the transparent area looks like glass. What would be the best way to set this up?

Thanks.

2

A picture of the glass looking transparent area would be helpful and also the standard shader layout with what other maps are used.
Guess - I think you need to add a specular map also.