Stencil buffer ring that follows the height of terrain?

I was given this great shader by bgolus that so far works quite well for my needs but I think now I’m getting enough of an understanding of shaders to properly put what I want down into words so I can get some research done on the topic.

I know it’s possible to have a stencil buffer ring, I actually managed to make a very primitive one using the sphere and a texture with just a glow strip going through it but is it possible to have the ring somehow follow the height of the terrain consistently.

I think this would be ideal just in terms of the look and you can see that the effect works because the sphere is hidden beneath the terrain but I obviously would like this to be consistent regardless of the height of the terrain does anyone know of any kind of trick or mathematics I would need to look up to get this sort of thing done?

Or am I looking at this entirely wrong as a method? I drew a line in blue in the image that shows what I’m looking for so it’s all clear.

There’s no special trick you can use to do this that’s pure GPU based. The closest would be a depth based intersection test, but that’s not perfect, and won’t handle cases where the terrain isn’t visible “behind” the ring (like on the edge of a cliff).

Ultimately there are two options. Brute force c# based and slightly less brute force GPU based.

For the c# option you take a cylinder mesh, and then modify it on the CPU side in c# every frame by doing height tests against the terrain at the expected position for each set of vertices around the ring. If you truly only care about the terrain, that’s pretty inexpensive and there’s a handy function to use to get the terrain height at any world position.

The only caveat is if you have multiple terrains, you have to figure out which one you’re over first, but that’s not that hard either.

For the GPU based option you would feed a height map to the ring and sample that in the vertex shader to offset the mesh vertices. This doesn’t need to be as detailed as the real terrain, being a little bit lower res helps smooth out some of the small details anyway. You can either capture this in real time with a camera that follows the ring around outputting a height map, or bake it out all at once for all terrain. Then it’s just a matter of passing the height texture and it’s position extents & height range to the shader so it can sample it correctly. This is more work to get right, but faster, especially the pre-baked option (assuming your area isn’t gigantic or dynamic).

This all make sense but how would it work in practice? The C# option seems like it would work fine for me as I’m using only one terrain so that isn’t an issue I can find very little on this sort of method online so it would be nice to have a good explanation somewhere of how to do all this especially for people who are looking into the same issue.

What I mean is, I get the idea of each technique separately, even found a post just now which shows how to change the vertices but how would this work changing the ring dynamically across the terrain because I want the ring to grow and shrink obviously.

It’s very weird how I can find tons of information on all sorts of other mechanics but it’s almost like everyone is afraid of even touching this one lol.

The easiest option may be to dynamically generate the cylinder in c# to begin with rather than modifying an existing mesh.

The high level is:

  1. Determine center position & radius of selection area.
  2. Determine number of sides you want your cylinder to have (either a fixed value, or dynamic based on diameter w/ min number).
  3. From the center position, radius, & sides, calculate the world xz position for each point of the circle, and find the terrain height. Generate mesh.
Vector3[] vertices = new Vector3[sides * 2 + 2];
Vector2[] uvs = new Vector2[sides * 2 + 2];
int[] tris = new int[sides * 2 * 3];
for (int i=0; i<=sides; i++) {
  // calculate angle and then sine & cosine for position around unit circle
  float radAngle = 2f * Mathf.PI / (float)sides * i;
  float s = Mathf.Sin(radAngle);
  float c = Mathf.Cos(radAngle);

  // get world space position of scaled and positioned vertices
  Vector3 pos = new Vector3(c * radius, 0f, s * radius) + center;

  // sample terrain height
  float terrainHeight = terrain.SampleHeight(pos);

  // assign vertex positions for top and bottom vertices of cylinder
  vertices[i * 2 + 0] = new Vector3(pos.x, terrainHeight - height, pos.z);
  vertices[i * 2 + 1] = new Vector3(pos.x, terrainHeight + height, pos.z);

  // assign UVs
  float u = (float)i / (sides);
  uvs[i * 2 + 0] = new Vector2(u, 0f);
  uvs[i * 2 + 1] = new Vector2(u, 1f);
}

// build triangle list
for (int i=0; i<sides; i++)
{
  tris[i * 2 + 0] = i * 2 + 0;
  tris[i * 2 + 1] = i * 2 + 1;
  tris[i * 2 + 2] = i * 2 + 2;

  tris[i * 2 + 3] = i * 2 + 2;
  tris[i * 2 + 4] = i * 2 + 1;
  tris[i * 2 + 5] = i * 2 + 3;
}

// assign arrays to mesh
mesh.vertices = vertices;
mesh.uv = uvs;
mesh.triangles = tris;

Something like that. I can’t guarantee the winding order on the triangles is correct though. The UV and triangle list don’t really need to updated every time if the number of sides doesn’t change, though you’d want to call RecalculateBounds() if you only modify the vertex positions.

Also this assumes the mesh renderer is always at 0,0,0 with no rotation, and is moving the generated mesh around the scene. You can generate the vertex positions as a local offset instead easily enough by modifying the above code.

Wow, that’s a lot of maths, I get the general idea and thank you for the detailed explanation but it looks like I’m going to have to practice some basic mesh generation first before I even come close to implementing this. The thing is if I were to implement this I would have to make the mesh generate around the buildings that I create which is going to involve even more code, the stencil buffer should take care of the blending issue but I’m going to have to have a think about this.

I can see why you mentioned doing a cylinder though, because it can stick to the ground and stretch with the way the vertices works but yikes lol.

Of course Brackey’s already has a well explained tutorial for this.

I’ll go through this and see if I can implement it at all using your suggestion, going to have to wrap my head around the idea of generating meshes generally though.

The other half of this whole thing is … most people don’t bother with any of this. Instead they just slap a projector in the scene, or use a custom terrain shader that draws a circle on the ground and call it done.

I’m not most people >_< :smile: I can understand why they’d find it frustrating though, I’m getting the whole idea now on why it’s done this way I just need to understand mesh generation and then I’ll be good. Another method I think I saw somewhere online involved using a raycast somehow.

You replace the SampleHeight with a ray cast if you want to include collision object other than the terrain. Or if your circle ranges are small enough that a non-deforming circle / cylinder doesn’t look too odd just hovering slightly above the terrain. I did this for Falcon Age, I do a number of ray casts around the outside ring of where the player’s teleport pointer is and find an average plane normal and offset height so it is unlikely to intersect with anything.

Ahhh, thanks for that this is starting to make sense now, when I get down to it I’ll post the results here.

I’ve managed to find a lot on mesh generation now and I’m getting my head around it, what is the ‘sides’ variable though @bgolus is is it a float of some kind? I don’t see it declared in your code.

Should be an positive integer value. I didn’t declare it in the code because it should either be part of the function declaration, a property of the class, or hardcoded somewhere, depending on how you want to implement it.

Easiest option is of course to just hardcoded it to some value, like 64, which will give you a decently smooth looking cylinder. Other options are to adjust it between a min and max based on the radius, or slightly more sanely, conceptually, the diameter. For larger radius it’s probably a good idea to keep the spacing between each segment to be relatively close to the min size of the terrain mesh density. But depending on the size of the ring you need and the number of sides you want, it might not ever be a problem.

1 Like

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class MeshGeneratedrInfluenceBorder : MonoBehaviour

{
    public int sides;
    public float height;
    public float radius;
    public float center;

    public Terrain terrain;
    Mesh mesh;

    Vector3[] vertices;
    Vector2[] uvs;
    int[] tris;

    private void Start()

    {
        CreateMesh();
    }


    private void CreateMesh()

    {
        mesh = GetComponent<Mesh>();

        vertices = new Vector3[sides * 2 + 2];
        uvs = new Vector2[sides * 2 + 2];
        tris = new int[sides * 2 * 3];

        for (int i = 0; i <= sides; i++)

        {
            float radAngle = 2f * Mathf.PI / sides * i;
            float s = Mathf.Sin(radAngle);
            float c = Mathf.Cos(radAngle);


            Vector3 pos = new Vector3(c * radius + center, 0f + center, s * radius + center);

            float terrainHeight = terrain.SampleHeight(pos);

            vertices[i * 2 + 0] = new Vector3(pos.x, terrainHeight - height, pos.z);
            vertices[i * 2 + 1] = new Vector3(pos.x, terrainHeight + height, pos.z);

            float u = (float)i / (sides);
            uvs[i * 2 + 0] = new Vector2(u, 0f);
            uvs[i * 2 + 1] = new Vector2(u, 1f);
        }

        for (int i = 0; i < sides; i++)

        {
            tris[i * 2 + 0] = i * 2 + 0;
            tris[i * 2 + 1] = i * 2 + 1;
            tris[i * 2 + 2] = i * 2 + 2;

            tris[i * 2 + 3] = i * 2 + 2;
            tris[i * 2 + 4] = i * 2 + 1;
            tris[i * 2 + 5] = i * 2 + 3;
        }

        mesh.vertices = vertices;
        mesh.uv = uvs;
        mesh.triangles = tris;
    }

}

It almost seems to be there, I’m obviously doing this wrong as I haven’t messed with mesh generation code but I’m getting an idea now of where you were going with this @bgolus unfortunately I don’t have a lot of knowledge when it comes to mesh generation and playing with vertices. As you can see it does seem to be getting some kind of data a bit from the terrain but it’s only a slight bump so I’ve screwed up somewhere with the implementation. What’s confusing me is that it looks like this when I have the center float at 0 but when I change the value to something high like 100 it actually somehow slants just as if it were following the terrain but then when I go to runtime it goes even more extreme, is this center float actually about the center or have I gone completely off base with the position data and that’s why it’s acting so funny?

You’ll also note that there is an issue with spacing and it doesn’t seem to be forming a cylinder correctly because of that, I know this can all be fixed I just need to go through it bit by bit as I’m learning. It’s really fascinating testing this stuff out and seeing it all work though and I can see why you suggested this idea.

The center value was supposed to be the center of the cylinder in 3D space. So a Vector3 value you’re setting via the script, presumably based on a ray cast from a pointer or some other thing that’s being moved around. If you want this to be attached to a transform that you’re using the world position of, that transform either needs to be something that doesn’t have this script attached as it explicitly needs to be at the world origin as it’s written, or it needs to be modified to take into account the current transform’s position.

Right now the position you call SampleHeight with and the position you build the mesh from are the same. If you want it to be attached to that object then you need to treat the vertex positions as being relative to that object’s transform and add the transform’s position to the circle vertex position. Or use the TransformPoint and InverseTransformPoint functions.

1 Like

Took a pass at the script. I made one significant typo in my example. It should have been tris[i * __*6*__ + #] not * 2. I also modified a few other things, like having it run on Update() and having separate top and bottom offsets rather than a single “half height” property. I also missed that Unity doesn’t recalculate the bounds when setting .tris after for the first time.

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

[ExecuteInEditMode]
public class MeshGeneratedrInfluenceBorder : MonoBehaviour
{
    public int sides = 16;
    public float radius = 5f;
    public float top = 1f;
    public float bottom = -0.5f;

    public Terrain terrain;
    Mesh mesh;

    Vector3[] vertices;
    Vector2[] uvs;
    int[] tris;

    private void Update()
    {
        CreateMesh();
    }

    private void CreateMesh()
    {

        if (mesh == null)
        {
            var meshFilter = GetComponent<MeshFilter>();
            if (meshFilter == null)
                return;
            mesh = new Mesh();
            meshFilter.sharedMesh = mesh;
        }
        vertices = new Vector3[sides * 2 + 2];
        uvs = new Vector2[sides * 2 + 2];
        tris = new int[sides * 2 * 3];

        Vector3 center = transform.position;
        for (int i = 0; i <= sides; i++)
        {
            float radAngle = 2f * Mathf.PI / sides * i;
            float s = Mathf.Sin(radAngle);
            float c = Mathf.Cos(radAngle);
            Vector3 pos = new Vector3(c * radius, 0f, s * radius);
            float terrainHeight = terrain.SampleHeight(pos + center);
            vertices[i * 2 + 0] = new Vector3(pos.x, terrainHeight + bottom - center.y, pos.z);
            vertices[i * 2 + 1] = new Vector3(pos.x, terrainHeight + top - center.y, pos.z);
            float u = (float)i / (sides);
            uvs[i * 2 + 0] = new Vector2(u, 0f);
            uvs[i * 2 + 1] = new Vector2(u, 1f);
        }
        for (int i = 0; i < sides; i++)
        {
            tris[i * 6 + 0] = i * 2 + 0;
            tris[i * 6 + 1] = i * 2 + 1;
            tris[i * 6 + 2] = i * 2 + 2;
            tris[i * 6 + 3] = i * 2 + 2;
            tris[i * 6 + 4] = i * 2 + 1;
            tris[i * 6 + 5] = i * 2 + 3;
        }
        mesh.vertices = vertices;
        mesh.uv = uvs;
        mesh.triangles = tris;
        mesh.RecalculateBounds();
    }
}
1 Like

You have helped me out so much @bgolus so thanks for that, I had been researching this damn mechanic for months and was constantly running into problems so I’ll probably even update the original thread I made on the topic of influence borders generally and link this thread to help people stumbling on this problem as well.

There is only one final problem left, I have gotten the borders now fully functional and just do a simple boolean check on when the player exits and enters their own influence border. Now I can live with this issue it’s more an aesthetics thing but as you can see there’s ugly overlap popping back in again now that we’re generating an actual ring rather than a sphere like with the old way.

I remember seeing in my old thread people making lots of mentions about masking and so on? I have no idea though how you would potentially blend the borders in using masks with this method, do you have any ideas?

This by the way is how I’ve handled the collision, you’ll note as well that I’ve changed the colour according to ownership so that’s all functional now. There’s a sphere collider I’ve put in which I’ve linked up to the radius float that the mesh generation uses. I should be able to change that now according to things like how much population I have and so on.

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class MeshGeneratedInfluenceBorder : MonoBehaviour

{
    public int sides = 16;
    public float radius = 5f;
    public float top = 1f;
    public float bottom = -0.5f;

    public Terrain terrain;
    Mesh mesh;

    Vector3[] vertices;
    Vector2[] uvs;
    int[] tris;

    private void Update ()

    {
        CreateMesh();
        GetComponent<SphereCollider>().radius = radius;
    }


    private void CreateMesh()

    {
        terrain = GameObject.FindGameObjectWithTag("Terrain").GetComponent<Terrain>();

        if (mesh == null)

        {
            var meshFilter = GetComponent<MeshFilter>();
            if (meshFilter == null)

            {
                return;
            }

            mesh = new Mesh();
            meshFilter.sharedMesh = mesh;
            GetComponent<SphereCollider>().radius = radius;

        }

        vertices = new Vector3[sides * 2 + 2];
        uvs = new Vector2[sides * 2 + 2];
        tris = new int[sides * 2 * 3];

        Vector3 center = transform.position;

        for (int i = 0; i <= sides; i++)

        {
            float radAngle = 2f * Mathf.PI / sides * i;
            float s = Mathf.Sin(radAngle);
            float c = Mathf.Cos(radAngle);


            Vector3 pos = new Vector3(c * radius, 0f, s * radius);

            float terrainHeight = terrain.SampleHeight(pos + center);

            vertices[i * 2 + 0] = new Vector3(pos.x, terrainHeight + bottom - center.y, pos.z);
            vertices[i * 2 + 1] = new Vector3(pos.x, terrainHeight + top - center.y, pos.z);

            float u = (float)i / (sides);
            uvs[i * 2 + 0] = new Vector2(u, 0f);
            uvs[i * 2 + 1] = new Vector2(u, 1f);
        }

        for (int i = 0; i < sides; i++)

        {
            tris[i * 6 + 0] = i * 2 + 0;
            tris[i * 6 + 1] = i * 2 + 1;
            tris[i * 6 + 2] = i * 2 + 2;

            tris[i * 6 + 3] = i * 2 + 2;
            tris[i * 6 + 4] = i * 2 + 1;
            tris[i * 6 + 5] = i * 2 + 3;
        }

        mesh.vertices = vertices;
        mesh.uv = uvs;
        mesh.triangles = tris;
        mesh.RecalculateBounds();
    }

}
GetComponentInChildren<Renderer>().material.SetColor("_Color", Color.red);

Note: For people checking this thread out, this line is just a pseudo code example, obviously be sure to change GetComponent to suit your needs with how you’ve organised your hierarchy.

So the last step you’re missing for what you’re actually trying to reproduce is the outline isn’t using any kind of special GPU tricks. It’s a single mesh produced by blending two (or more) circles together and finding the edge. Two circles is easy enough, more than that gets a little more complicated. Search for “2D metaball mesh”.