Advice on a method for identifying rock/dirt in 1 mesh/terrain ?

Given a terrain object or a single large terrain mesh which has differently texture materials, does anyone have advice on architecting a way to determine if the collision point is of a particular physics material ?

An example would be I have a desert (plane) and going through the middle is a textured road (gravel). It wouldn't make sense to break the ground mesh up into several parts. I can think of a few ways but none seem elegant or would work in a generalized sense.

Some off the head ideas :

  1. I could have a material map and paint the world and do a texture lookup, but this doesnt extend to things like buildings and trees.

  2. I could have a scripe with an enum attached to every object, but some objects can be made of 2-3 different materials.

For a Unity terrain, use TerrainData.GetAlphamaps to read the splatmap data. Each pixel then corresponds to a location, and the r/g/b/a values correspond to the four textures.