Position correspondence of a 3D point between a quadrilateral and a rectangle

I would like to find the coordinates of a 3D point P’ on a rectangle matching the relative position of a point P on the plane formed by a quadrilateral as shown below (P isn’t necessarily inside the quad). The four points of the quad Q1, Q2, Q3 and Q4 are mapped to the four points of the rectangle R1, R2, R3 and R4 respectively.

This problem is related to perspective transformations and the solutions I’ve found so far on the web are mostly very generic or for 2D transforms which don’t handle points outside the quad (e.g. this one). I was wondering if there was a relatively simple solution to compute that point using Unity’s functions.

Here’s an approach using barycentric coordinates.
Here’s the core routines, which convert a point and triangle to a triple of barycentric coordinates, and then the reverse. You can then use this on 4 triangles in your quad (eg, one triangle per vertex) and compute the four sets of barycentric coordinates from the source quad and then average the results in the destination quad.

Running example code is on GitHub here.
It runs right in the editor. Drag stuff around w/ the editor.

And the core math:

using UnityEngine;

public static class QuadsAndPoint
{
    // Given triangle ABC and a point,
    // return the barycentric coordinates of the point.
    static public float[] BaryFromTriangleAndPoint(Vector3 P, Vector3 A, Vector3 B, Vector3 C) {
        var ret = new float[3];

        ret[0] = ((B.y - C.y) * (P.x - C.x) + (C.x - B.x) * (P.y - C.y))
                 /
                 ((B.y - C.y) * (A.x - C.x) + (C.x - B.x) * (A.y - C.y));
        
        ret[1] = ((C.y - A.y) * (P.x - C.x) + (A.x - C.x) * (P.y - C.y))
                 /
                 ((B.y - C.y) * (A.x - C.x) + (C.x - B.x) * (A.y - C.y));
        ret[2] = 1 - ret[0] - ret[1];

        return ret;
    }

    // Inverse of BaryFromTriangleAndPoint().
    static public Vector3 PointFromTriangleAndBary(Vector3 A, Vector3 B, Vector3 C, float[] coeffs) {
        return A * coeffs[0] + B * coeffs[1] + C * coeffs[2];
    }
}

Thanks for sharing, I found a lot of interesting information here. A really good post, very thankful and helpful that you will write many more posts like this one.