WorldToViewportPoint and ViewportToWorldPoint math

Questions & Answers
1

I’m having a hard time recreating the exact transformation using just the camera projection and view matrices. I’ve tried various approaches, using both the camera projection and the GL camera projection.

If everything works right, these two variables, _result and result, should match:

static Vector3 worldToViewportPoint(Vector3 point3D) {
	Matrix4x4 P = GL.GetGPUProjectionMatrix(Camera.main.projectionMatrix, false);
	Matrix4x4 V = Camera.main.transform.worldToLocalMatrix;
	Matrix4x4 VP = P * V;
	var result = VP.MultiplyPoint(point3D);
	var _result = Camera.main.WorldToViewportPoint(point3D);
    
	return _result;
}

Similarly, I need to calculate the ViewportToWorldPoint, which I imagine I would solve if the method above had the correct matrices. The reason we need this is for a computer vision library running beneath unity using linear algebra.

2

Try this:

static Vector3 worldToViewportPoint(Vector3 point3D)
{
    Matrix4x4 P = Camera.main.projectionMatrix;
    Matrix4x4 V = Camera.main.transform.worldToLocalMatrix;
    Matrix4x4 VP = P * V;

    Vector4 point4 = new Vector4(point3D.x, point3D.y, point3D.z, 1.0f);  // turn into (x,y,z,1)
    Vector4 result4 = VP * point4;  // multiply 4 components

    Vector3 result = result4;  // store 3 components of the resulting 4 components

    // normalize by "-w"
    result /= -result4.w;

    // clip space => view space
    result.x = result.x / 2 + 0.5f;
    result.y = result.y / 2 + 0.5f;

    // "The z position is in world units from the camera."
    result.z = -result4.w;


    var _result = Camera.main.WorldToViewportPoint(point3D);
    // result == _result

    return _result;
}

The idea is to use 4-components matrix operations as V and P are of “Matrix4x4” type. Also please notice regular matrix multiplication “VP * point4” instead of “VP.MultiplyPoint(point3D)”

3

@victorbisaev That worked perfectly, thank you. Now for the opposite, view to world point.

I reversed the clip space/view space and am still off. I assume its the denormalization part I’m missing.

static Vector3 viewportToWorldPoint(Vector3 point2D) {
   
   Matrix4x4 P = Camera.main.projectionMatrix;
   Matrix4x4 V = Camera.main.transform.worldToLocalMatrix;
   Matrix4x4 VP = P * V;

   Vector3 point2DCopy = point2D;
   
   // view space => clip space
   point2DCopy.x = 2.0f * (point2DCopy.x - 0.5f);
   point2DCopy.y = 2.0f * (point2DCopy.y - 0.5f);
   
   Vector4 point4 = new Vector4(point2DCopy.x, point2DCopy.y, point2DCopy.z, 1.0f);  // turn into (x,y,z,1)
   
   Vector4 result4 = VP.inverse * point4;  // multiply 4 components

   Vector3 result = result4;  // store 3 components of the resulting 4 components

   var _result = Camera.main.ViewportToWorldPoint(point2D);
   
   return _result;
}