If you can achieve the effect you want with vertex colors, at what point should you use a texture map, instead, if ever? Is there any sort of compression available where vertex colors are concerned? If you don’t need vertex colors, are they stored in the mesh anyway? I’m finding myself with a lot of unorganized-in-brain questions about this, so if you can just share whatever thoughts and knowledge you care to, on the subject, it will probably help me.
In other engines I only really use vertex colours for blending between textures. It’s quite a handy tool for blending terrain grafted to static scenery fairly seamlessly.
Unity doesn’t currently have a shader material that supports this outside of the actual terrain splatting. Would be nice for those of us who use high end 3D apps that make vertex painting and masking dead easy and prefer to lay out levels in their native 3d apps.
Otherwise, I guess vertex colours are mostly useful for low spec systems, faking AO and lighting, and giving some percieved depth to the scene by baking colours to statis scenery. I’m not sure what the overheads are, since the colours have to be stored somewhere. Never really thought about it too much.
That’s interesting, and I’d love to see an example. I’ve seen other threads, on this forum and outside, where people talk about using vertex colors for blending. It seem to me that texture maps would be considerably better for that, but my inexperience is the main reason I started this thread!
Another problem with this, for me, is that Blender is a jerk and won’t let me paint the Alpha channel of vertex colors. 
So I can only really blend three textures, not four, using vertex colors.
The main difference is blending by vertex gives you splatting with unlimited resolution. Texture blend maps hit their limits once you start dealing with large objects like environments.
True, but there’s interpolation to deal with. What would you do? Create two edge loops to define where you want blending to occur? (Similar to what you’d do with defining creases in subdivision models, when you don’t want to use weighting?)
Yup, if you need extra resolution, you can add additional vertices at just the areas where you need the detail.
I think in many cases you need much less data to define the blending with vertex colours than with an extra texture.
I use vertex colors and vertex alpha all the time to create lighting and shadow effects with additive and multiply materials, without having to use any texture. You’re able to get perfect lines on shadows this way because you aren’t dealing with resolution, and Unity does a great job of compressing meshes, so using a few extra verts is not really a big concern.
That’s awesome. I’m seeing effects like that all over the place, playing through the Metroid Prime Trilogy right now.
What 3D tool do you use to create vertex colors with alpha?
I have to admit this is an awesome use of vertex coloring !
( Why didn’t I think about it earlier ? :roll: )
I use 3dsmax and do all vertex colors with direct value editing, as opposed to painting.
If you only use alpha, I think that’s easy enough. But then you only get to blend two textures. You can use the vertex colors as a sort of splatmap, and blend four, but you’ll have to make sure the RGBA values add up to 1 (or 255, of you prefer that representation). How would you make sure that happens?
R+G+B+A Divided by (255*4) 
What??
Close, to be sure you can use:
R = R/(R+G+B+A)
G = G/(R+G+B+A)
B = B/(R+G+B+A)
A = A/(R+G+B+A)
I’d write a script to enforce this on a mesh, either at runtime or as an editor processing step that saves a new mesh asset.
I don’t see how this stuff applies when you’re creating something. How can you design a vertex color splatmap artistically?
It’s a shame you can’t use shaderFX with unity. I’ve been using ogre for a while and that has the excellent ofusion exporter that does shaderfx to ogre shaders in realtime.
I made a vertex colour blending shader that blends three normal/specular terrain textures using two vertex colours where red and green are two textures, and anything else was the third. Worked very well. You can see the node graph for it in the pic below.
Larger legible pic in link below
Would be cool if Unity could get some kind of node based shader editor like that. Unfortunate that it doesn’t support any standard DXSAS (HLSL or CG) shaders directly.
Random tip: If you need the precision required by vertex color blending and all you have are the texture blended shaders on the wiki (and don’t want to write your own):
-
Set a ramp as the mix map. You can use a 2D ramp if you are blending 4 textures, a 1D ramp similar to the regular falloff ramp in the projectors if you’re just blending 2 textures.
-
Use your UV2 coordinates as if they were 2-channel vertex colors.
Another point in favour of vertex colours is on the iPhone, it won’t break material batching the same way setting the colour through the material property will. Obviously you can’t use it the same way GraphicalGeek is here if you do that, but it’s great if you want multiple instances of an object in a variety of colours without additional draw calls.
Indeed! 
I posted another thread of this but thought I’d include it here as well.