Finding the closest point from a point to a 4-points-face?

I have a point and a face that’s defined by 4 points.
I would like to find the position on the face that is the closest to my point (in 3D).

5232104--521906--upload_2019-12-1_9-45-8.png

How can I do this?

I’d probably start by calculating the closest F-point, and then the next closest point from the points connected to that. Unless i’m not missing some case, you should not have the two points making up the closest line independant of your shape; in your example F4-F3. Now we reduced the problem to finding the closest point p on a line F4-F3 to some other point P. The closest point p on a line F4-F3 to some other point P, is where its perpendicular vector hits P.
But since that’s about all i remember, i’ll rather post you some links from people who seemingly know what they are talking about: linear algebra - Closest point on a line to another point - Mathematics Stack Exchange

Edit: I overread the 3D part, i’ll have to think about it again. The idea should be similar. It’s just a lot harder to think about it with a shape instead of a line. If it was a whole plane instead it would be rather easy again. What do you need this for?

Your question is a little bit unclear. Do you want the closest point (out of the 4 that make up the face) or just the closest point still inside the bounds of the face?
The first could be done by comparing the distance between each of the 4 points that make up the face with the point(p). The latter could be done by raycasting from the point(p) to the center of the face and checking where the ray hits (hit.point).

He has a 2D shape in 3D space and a point P. He now wants to find the point p on the 2D shape such that its distance to P is minimal. I overread the 3D part when i wrote my reply.
As for your suggested solution, sadly Raycasting to the center of the shape wont work. Not even for 2D, and even less for 3D. An easy way to visualize this would be to put P directly under the F of F3. Now the closest point would be the corner, but the raycast would hit something a bit further down. In 3D space we could miss the entire shape until we hit the center (example, lifting P towards your eyes in 3D just a tiny bit).

I’m not the best at math, so maybe there is an easy solution.
A work around could be this:

    1. Check if P is inside an imaginary 3D object you get if you extruded the defined shape infinitely on its perpendicular axis. If this is the case, we know P is above the shape, thus we can act as if it was a plane, making the calculation easier.
    1. If 1 is not true, then check if P is on the same plane as the shape. If that’s the case, we can threat it as a 2D problem, which i described in my last post.
    1. If neither 1 or 2 are true, then P lies outside the shape in 3D shape, neither on the plane of it, nor directly above it. This is where i’m not entirely sure, but the closest point in this case should still be the same as in 2, so we could maybe somehow project P onto the same plane as the shape and then calculate the result for 2).

I guess i’ll feel stupid as soon as somebody with better math knowledge arrives and shows a super simple solution lol.

Ok I managed to do it, but I don’t know that it’s pretty. But at least I think it’s performant.

What I do is this:

  1. I create a plane using Unity’s Plane with 3 points (out of the 4 that constitute the 4-points-face).
  2. I use the Plane.ClosestPointOnPlane() method to find the closest point to the plane. This almost gets me what I need, the only problem is that it’s not confined to the 4-points-face.
  3. I check if the point is inside the 4 corners of the rectangle, if it’s not inside: I find the closest point to the edges of the rectangle. This will be the closest point to the rectangle.

This is the code for operation #3:

    /// <summary>
    /// Check if a point is inside the points of the face and then modify it if it's not. You must enter the points in the correct order of the face.
    /// </summary>
    Vector3 ClosestPointToRectangleFace(Vector3[] _face, Vector3 _p)
    {
        // 1_ get sqrDist to closest face point
        var closestDistance = float.MaxValue;
        var chosenIndex = -1;
        for (var i = 0; i < _face.Length; i++)
        {
            var f = _face[i];
            var sqrDist = Vector3.SqrMagnitude(f - _p);
            if (sqrDist < closestDistance)
            {
                chosenIndex = i;
                closestDistance = sqrDist;
            }
        }

        // 2_ get the distance to the 2 neighbour points of the face's closest point.
        // if we have a smaller distance to the 2 neighbour points than the distance between the chosen point and the neighbour point: we are inside the face.
        var chosenPoint = _face[chosenIndex];
        Gizmos.DrawWireSphere(chosenPoint, .2f);

        var cnt = _face.Length;
        var nextNeighbour = _face[(chosenIndex + 1) % cnt];
        var prevNeighbour = _face[(-1 + chosenIndex + cnt) % (cnt)];

        Gizmos.color = Color.magenta;
        Gizmos.DrawWireSphere(nextNeighbour, .2f);
        Gizmos.color = Color.red;
        Gizmos.DrawWireSphere(prevNeighbour, .2f);

        var projectedPoint1 = Vector3Ext.NearestPointOnLine(_p, nextNeighbour, chosenPoint);
        var projectedPoint2 = Vector3Ext.NearestPointOnLine(_p, prevNeighbour, chosenPoint);

        Gizmos.DrawWireCube(projectedPoint1, Vector3.one);
        Gizmos.DrawWireCube(projectedPoint2, Vector3.one);

        // We are inside of the face.
        if (projectedPoint1.IsBetween(chosenPoint, nextNeighbour) && projectedPoint2.IsBetween(chosenPoint, prevNeighbour))
        {
//            Gizmos.color = Color.green;
//            Gizmos.DrawWireCube(_p, .5f * Vector3.one);
//
            return _p;
        }
        else // we are outside the face! That means the closest point is necessarily on the lines defined by the 4 corners points of the face.
        {
//            Gizmos.color = Color.red;
//            Gizmos.DrawWireCube(_p, 1f * Vector3.one);
            var closestSqrDistance = float.MaxValue;
            var closestPoint = Vector3.zero;
            for (var i = 0; i < _face.Length; i++)
            {
                var currentPoint = _face[i];
                var nextPoint = _face[(chosenIndex + 1) % cnt];
                var prevPoint = _face[(-1 + chosenIndex + cnt) % (cnt)];
                var prevNearestPoint = Vector3Ext.NearestPointOnFiniteLine(_p, currentPoint, prevPoint);
                var nextNearestPoint = Vector3Ext.NearestPointOnFiniteLine(_p, currentPoint, nextPoint);

                var distanceToNextNearestPoint = Vector3.SqrMagnitude(nextNearestPoint - _p);
                if (distanceToNextNearestPoint < closestSqrDistance)
                {
                    closestPoint = nextNearestPoint;
                    closestSqrDistance = distanceToNextNearestPoint;
                }
                var distanceToPrevNearestPoint = Vector3.SqrMagnitude(prevNearestPoint - _p);
                if (distanceToPrevNearestPoint < closestSqrDistance)
                {
                    closestPoint = prevNearestPoint;
                    closestSqrDistance = distanceToPrevNearestPoint;
                }
            }

            return closestPoint;
        }
    }

These are the helper functions I used:

  public static Vector3 NearestPointOnFiniteLine(Vector3 _point, Vector3 start, Vector3 end)
    {
        var line = (end - start);
        var len = line.magnitude;
        line.Normalize();

        var v = _point - start;
        var d = Vector3.Dot(v, line);
        d = Mathf.Clamp(d, 0f, len);
        return start + line * d;
    }

    public static Vector3 NearestPointOnLine(Vector3 pnt, Vector3 linePnt1, Vector3 linePnt2)
    {
        var lineDir = linePnt2 - linePnt1;
        lineDir.Normalize();
        var v = pnt - linePnt1;
        var d = Vector3.Dot(v, lineDir);
        return linePnt1 + lineDir * d;
    }

    //https://stackoverflow.com/questions/328107/how-can-you-determine-a-point-is-between-two-other-points-on-a-line-segment
    public static float sqrDist(Vector3 a, Vector3 b)
    {
        return Vector3.SqrMagnitude(a - b);
    }
    public static bool IsBetween(this Vector3 a, Vector3 b, Vector3 c)
    {
        var totalDist = sqrDist(b, c);
        if (sqrDist(a, b) <= totalDist && sqrDist(a, c) <= totalDist)
        {
            return true;
        }
        else return false;
    }

Here’s how I used it in my code:

// step 1
Plane downPlane = new Plane(lowerBottomRight, lowerBottomLeft, lowerUpperRight);
// step 2
Vector3 downPosOnInfinitePlane = downPlane.ClosestPointOnPlane(playerPos);
// step 3
Vector3 downPosInsideRectangle = ClosestPointToRectangleFace(downFace, downPosOnInfinitePlane);

I kind of did what you suggested.
I need it to expulse a player out of a box collider in only specific directions. The box can be rotated scaled etc…

I’m not sure that it’s completely foolproof yet, so I’ll do a bunch of tests then I’ll show you what I use it for in more details. Basically I need to get rigidbodies out of an object that suddenly is not a trigger anymore. If I let Unity physics handle it, it can make my rigidbodies glitch through the ground so I must preemptively move the rigidbodies out of the box collider so that they don’t get lost.

EDIT: Ok it works!

Here’s the result: https://i.imgur.com/7RYpQAW.gifv
You can see it’s getting the closest position out of the box collider per face from the start position of the player. (the red box).