Projection matrix

Unity Engine
1

Hello everyone,

I am currently trying to figure out the full projection matrix P for a given image, such that

w * [u,v,1] = P * [x,y,z,1]

,where P is a 3x4 projection matrix, u and v are the image coordinates and x,y,z are the world coordinates of the imaged point. So I took a photo (small application written with Unity and C#) and tried to get cameraToWorldMatrix as well as projectionMatrix with

photoCaptureFrame.TryGetCameraToWorldMatrix(…);
photoCaptureFrame.TryGetProjectionMatrix(…);

Initially I thought that now P = projectionMatrix * inverse(cameraToWorldMatrix), but this seems not to be correct. Because of the format of the projection matrix (https://developer.microsoft.com/en-us/windows/mixed-reality/locatable_camera) I also end up with a 4x4 matrix instead of the desired 3x4 matrix.

Can anyone give me a hint how to finally compute the matrix P?

2

P should just be the projection matrix you get from TryGetProjectionMatrix. And I’m not 100% sure, but I don’t think projection matrices can be 3x4.

It sounds like you’re asking for a shovel when what you need is a hole. What problem are you trying to solve?

3

Hi,

For the projection matrix I am referring here to a perspective transformation as defined for example by OpenCV (Camera Calibration and 3D Reconstruction — OpenCV 2.4.13.7 documentation), where the projection matrix is defined as A[R|t] with A = calibration matrix and [R|t] as the rotation-transformation.

So I would like to recompute the matrix P = A[R|t] for a given image taken with the hololens.

4

Oh, I see now. Thanks for the link. That’s actually a different definition for what a projection matrix is - and I have to admit, I’ve never heard of a projection matrix being defined that way.

The projection matrix we provide is closer to the format described here: OpenGL Projection Matrix

[R|t], in the case of our API, should be transcribed in the matrix you get from TryGetCameraToWorldMatrix. You may need to invert it, not sure, as I’m afraid I can’t immediately translate to what you linked - like I said, I’ve never heard of a projection matrix being defined that way. I hope this helps.