We are creating a game that has an anime look, and the most well known way to achieve trees that "Look" like anime is by having them be billboarded images.
Is there any way to be able to have trees that worked like this, but were still able to be used as Trees in the unity terrains? any way to change the soft occlusion leaves shader to allow for billboarding of the leaves?
Any GameObject in Unity can be used as a tree in terrain. Create the tree as you would like it, with any shader assigned to it, and any collision assigned to it. Then simply go into the terrain config and pick this model as your tree.
Unity uses a couple of different builtin shaders for its trees, but unlike the other builtin shaders, the billboard shader is hidden and won't appear in the shader dropdown list. To use the billboard shader on your custom tree, you'll need to download the shader source and add them to your project. Get it here. Copy TerrainShaders\Trees\BillboardTree.shader to your project and edit it. Change the shader name from "Hidden/TerrainEngine/BillboardTree" to "TerrainEngine/BillboardTree".
Now, the next part is a little harder. When you model the leaves, all 4 verts of each quad need to be in the exact same position. Also for the leaves you need 2 uv sets. The first set is used for normal texturing. The second uv set is for "billboard extrusion", where U is the world distance to move the vertex right, and V is up. (use negative UV values to move left and down). You should write some maxscript/mel to help you do this, rather than ask the artist to conjure this tech voodoo by hand.
If you want swaying trees, copy and rename the appropriate hidden builtin shader into your project; and the art needs to be specially made so that the alpha of the per-vertex color value specifies the "sway amount".