How to manually calculate localToWorldMatrix/worldToLocalMatrix?

For learning purpose, I am trying to calculate transform.localToWorldMatrix and transform.worldToLocalMatrix manually. I found if a child object has a single parent, TRS matix made by Matrix4x4.TRS(), which I think it is just a transformation matrix storing the gameobject’s position, rotation, and scale, is the same as localToWorldMatrix and its inverse matrix is the same as worldToLocalMatrix, as the link below says.

https://gamedev.stackexchange.com/questions/139003/calculate-vector3-global-point-projecting-it-in-local-space-using-unity-and-c

But If a child’s parent has another parent, then that would not work well. So I figured out I need multiplication of each parent 's transformation matrix. Since Unity uses column major matrices, a world vertex position should be calculated as SuperParentMatrix * InternalParentMatrix * localVertex. I tried it as the code below, but it outputs a wrong value. What am I doing wrong? and How can I calculate it correctly?

public class MultiParentTest : MonoBehaviour
{
    [SerializeField]
    Transform[] parents;

    [SerializeField]
    GameObject child;

    // Start is called before the first frame update
    void Start()
    {
        LogPositionInfo();
    }

    // Update is called once per frame
    void Update()
    {

    }

    [ContextMenu("Log Position Info")]
    void LogPositionInfo()
    {
        if (parents != null && parents.Length == 2 && child != null)
        {
            Matrix4x4 l2wMat = parents[0].GetTRSMatrix() * parents[1].GetTRSMatrix();   // why wrong?
            //Matrix4x4 l2wMat = parents[1].localToWorldMatrix;   // correct

            var pos = child.transform.position;
            var posInfo = $"Correct World Position:

({pos.x}, {pos.y}, {pos.z})
";
pos = l2wMat.MultiplyPoint3x4(child.transform.localPosition);
posInfo += $"localToWorld:
({pos.x}, {pos.y}, {pos.z})
";

            Matrix4x4 w2lMat = parents[1].GetTRSMatrix().inverse * parents[0].GetTRSMatrix().inverse;   // why wrong?
            //Matrix4x4 w2lMat = parents[1].worldToLocalMatrix;   // correct

            pos = child.transform.localPosition;
            posInfo += $"Correct Local Position:

({pos.x}, {pos.y}, {pos.z})
";
pos = w2lMat.MultiplyPoint3x4(child.transform.position);
posInfo += $"worldToLocal:
({pos.x}, {pos.y}, {pos.z})
";

            Debug.Log(posInfo);
        }
    }
}

static public class TransformExtension
{
    static public Matrix4x4 GetTRSMatrix(this Transform transform)
    {
        return Matrix4x4.TRS(transform.position, transform.rotation, transform.lossyScale);
    }
}

Description of my scene that uses the code above:
made 3 GameObject as parent0, parent1, and child. parent0 is a parent of parent1 and parent1 is a parent of child. set different transform values of them and put parent0 and parent1 into Transform parents as parent0 is the first element, and put child into GameObject child.

What you’re doing makes no sense. You use the calculated worldspace parameters of the transform to create your matrix. This would already contain the full transformation, though it doesn’t work as you can’t rely on lossyScale. If there is any non-uniform scaling going on in the hierarchy it’s impossible to specify a scale vector3. That’s why it’s named “lossy”.

What you actually have to do is use the localPosition, localRotation and localScale to form a matrix which does not transform from local into worldspace but from local into parent space. Then chain all those matrices to get all the way up to worldspace. As you might know a Transform that has no parent implicitly has the worldspace as parent and position == localPosition, rotation == localRotation and lossyScale == localScale.

So your extension method need to do this:

static public Matrix4x4 GetTRSMatrix(this Transform transform)
{
    return Matrix4x4.TRS(transform.localPosition, transform.localRotation, transform.localScale);
}

However I don’t quite understand why you want to do that. Unity already does all that for you and most likely has all the matrices cached internally since they are required by Unity anyways.

You may also want to have a look at my Matrix crash course. Also when you want to understand matrices better, I highly recommend the 3b1b series on linear algebra