https://forum.unity3d.com/threads/navmesh-layers.116653/#post-778529
The thing about Layer weight costs though applied per navmesh polygon, is that these aren’t “directionally (gravitionally/gradient) aware” layers. They are merely fixed “roughness” layers. By directionally(and gravitionally/gradient) aware, what i mean is that taking a route that passes through a specific terrain layer polygon that approaches “uphill” could yeild much higher cost weight (per euclidean distance travelled along polygon) than going the other way round (or in some other arbituary angle direction) which approaches “downhill” or is “relatively flatter”, even if they pass through the same “slope” layer polygon. Given these information, is the navmesh engine able to find the least cost path to a given location, factoring any uphill/downhill/directional gradient biases across a single given navmesh polygon? Because in such a case, i think (i might be wrong though) that the “top-view” straight line distances from one navmesh polygon to another, doesn’t necessarily mean lowest cost when the multiple “angles of approaches” towards next adjacient polygon across any given point along it’s adjoining edge, may each yield a different cost arc depending on the gradient along the given polygon when approaching a given polygon edge at varying angles. As heavy handed as it sounds, is the only way to go about this is to yield multiple graph arc costs (created on the fly depending on target polygon location) per game precision unit along each adjoining polygon edge? Because in such a situation, I doubt the navmesh pathfinder will be able to find the “actual” shortest/least-costing path given slopes…
Most navmesh implementations i see assume the shortest straight line distance path within a polygon and across multiple polygons within the same weight group to be the “shortest” path, albeit iif the straight line passes through a layer polygon of greater/lesser weight, it simply adds additional weight to it linearly and considers whether detouring to a different polygon of a different layer weight is the better option, but never add the “angle” of approach factor within current polygon to that cost calculation.
Think of it as if ziggaing uphill is required…but if going downhill, the zig-zagging can be skipped because the “downward” movement yeilds lowest cost weight per unit travelled. Zigzag: Not the shortest route, but often the most efficient | UW News
Then again, i’m not sure if any other game engine models this correctly for Navmeshes. The only way to go about is to use a grid and custom arclic graph, which will sacrifice some precision (unless you go per-pixel or something…).
On a sidenote, is it mathematically wrong to assume that (even without considering any gravity/gradient upward/downward cost bias), that the pure 3D Euclidean distance travel cost across any set of interconnected navmesh polygons (even sloping polygons in 3D) within the same navmesh layer, is always shortest when it’s a 2D projected straight line, regardless of it’s angle of approach? I doubt this is the case, so assuming a steep slope (but not steep enough) is still deemed “traversible” by your navmesh engine settings, does it mean that it will still take the 2D top view straight line path through the slope (and thus move at a much greater 3D distance due to the sloping movement), and not consider a detour around the slope which might yield a shorter 3D distance?
However, is it mathematically right to assume that the 2D straight line distance from one point on a sloping 3D polygon, to another point on the edge WITHIN the SAME sloping polygon, is always the shortest 3D distance, right?
On a general note regarding navmeshes, shouldn’t the rough directional edge weight cost from a polygon to another polygon, take into account gradient slope (which adds to the distance to form the rough hypotenus distance along slope), be determined uniquely for each polygon to polygon direction using something like: sqrt(1+(Gradient between polygon centroids))? At least, this does give a rough semblance of varying/directional sloped edge costs, even if it’s not 100% accruate because it only determines this roughly between polygon centroid to centroid distance ratio (compared to flat gradient), where actual agent movement doesn’t reflect this anyway. Would it mean layers or costs should be made appliable on directional Edges between polygons, and not on the polygons themselves? To apply this in a Unity Navmesh context, It would be assumed that interconnected polygons from an agent’s starting position, that exist contiguously with connecting edges having the same edge cost weight, can be part of the same navmesh layer group, right? But since you have to determine how the polygons are grouped situationally depending on which edges are being crossed to get from one location to another, depending on direction of approach from current agent position’s starting polygon (to determine list of possible edges crossed and thus imply different navmesh layer grouping), can this be done in Unity since I think Unity bakes the navmesh layers beforehand and these can’t be changed per polygon at runtime, right?
