I saw this video here:
In the video they mention they modified Unity’s projection matrix to achieve the effect of the image updating based on the player’s physical position. I was wondering if anyone here knew how to achieve this? I’m most concerned about how to implement modifying the projection matrix in a script to properly skew the objects according to the camera’s angle.
Did you try a
http://forum.unity3d.com/_**search**_/?type=post
Reposting some old code
public Transform[] Corners;
public Transform lookTarget;
public bool drawNearCone, drawFrustum;
Camera theCam;
void Start () {
theCam = camera;
}
void Update () {
Vector3 pa, pb, pc, pd;
pa = Corners[0].position; //Bottom-Left
pb = Corners[1].position; //Bottom-Right
pc = Corners[2].position; //Top-Left
pd = Corners[3].position; //Top-Right
Vector3 pe = theCam.transform.position;// eye position
Vector3 vr = ( pb - pa ).normalized; // right axis of screen
Vector3 vu = ( pc - pa ).normalized; // up axis of screen
Vector3 vn = Vector3.Cross( vr, vu ).normalized; // normal vector of screen
Vector3 va = pa - pe; // from pe to pa
Vector3 vb = pb - pe; // from pe to pb
Vector3 vc = pc - pe; // from pe to pc
Vector3 vd = pd - pe; // from pe to pd
float n = -lookTarget.InverseTransformPoint( theCam.transform.position ).z; // distance to the near clip plane (screen)
float f = theCam.farClipPlane; // distance of far clipping plane
float d = Vector3.Dot( va, vn ); // distance from eye to screen
float l = Vector3.Dot( vr, va ) * n / d; // distance to left screen edge from the 'center'
float r = Vector3.Dot( vr, vb ) * n / d; // distance to right screen edge from 'center'
float b = Vector3.Dot( vu, va ) * n / d; // distance to bottom screen edge from 'center'
float t = Vector3.Dot( vu, vc ) * n / d; // distance to top screen edge from 'center'
Matrix4x4 p = new Matrix4x4(); // Projection matrix
p[0, 0] = 2.0f * n / ( r - l );
p[0, 2] = ( r + l ) / ( r - l );
p[1, 1] = 2.0f * n / ( t - b );
p[1, 2] = ( t + b ) / ( t - b );
p[2, 2] = ( f + n ) / ( n - f );
p[2, 3] = 2.0f * f * n / ( n - f );
p[3, 2] = -1.0f;
theCam.projectionMatrix = p; // Assign matrix to camera
if ( drawNearCone ) { //Draw lines from the camera to the corners f the screen
Debug.DrawRay( theCam.transform.position, va, Color.blue );
Debug.DrawRay( theCam.transform.position, vb, Color.blue );
Debug.DrawRay( theCam.transform.position, vc, Color.blue );
Debug.DrawRay( theCam.transform.position, vd, Color.blue );
}
if ( drawFrustum ) DrawFrustum( theCam ); //Draw actual camera frustum
}
Vector3 ThreePlaneIntersection ( Plane p1, Plane p2, Plane p3 ) { //get the intersection point of 3 planes
return ( ( -p1.distance * Vector3.Cross( p2.normal, p3.normal ) ) +
( -p2.distance * Vector3.Cross( p3.normal, p1.normal ) ) +
( -p3.distance * Vector3.Cross( p1.normal, p2.normal ) ) ) /
( Vector3.Dot( p1.normal, Vector3.Cross( p2.normal, p3.normal ) ) );
}
void DrawFrustum ( Camera cam ) {
Vector3[] nearCorners = new Vector3[4]; //Approx'd nearplane corners
Vector3[] farCorners = new Vector3[4]; //Approx'd farplane corners
Plane[] camPlanes = GeometryUtility.CalculateFrustumPlanes( cam ); //get planes from matrix
Plane temp = camPlanes[1]; camPlanes[1] = camPlanes[2]; camPlanes[2] = temp; //swap [1] and [2] so the order is better for the loop
for ( int i = 0; i < 4; i++ ) {
nearCorners[i] = ThreePlaneIntersection( camPlanes[4], camPlanes[i], camPlanes[( i + 1 ) % 4] ); //near corners on the created projection matrix
farCorners[i] = ThreePlaneIntersection( camPlanes[5], camPlanes[i], camPlanes[( i + 1 ) % 4] ); //far corners on the created projection matrix
}
for ( int i = 0; i < 4; i++ ) {
Debug.DrawLine( nearCorners[i], nearCorners[( i + 1 ) % 4], Color.red, Time.deltaTime, false ); //near corners on the created projection matrix
Debug.DrawLine( farCorners[i], farCorners[( i + 1 ) % 4], Color.red, Time.deltaTime, false ); //far corners on the created projection matrix
Debug.DrawLine( nearCorners[i], farCorners[i], Color.red, Time.deltaTime, false ); //sides of the created projection matrix
}
}
Create an empty GO for lookTarget, with 4 children to mark the corners, put this script on the camera.
This looktarget construction represents where your screen is (so doesnt move), and then you move the camera around to track the users position (eyes/head)
@hpjohn thanks for the info.
So let’s say I have a character walking around a the scene. If I wanted to keep the character on screen at all times, would I want to make sure the character always stays next to the lookTarget and within the 4 corners that were created?
The corners represent the real screen in virtual space. Everything needs to happen on the other side of it
Oh, I see now. It seems your implementation depends on the corners being setup to make a vertical screen that is viewed from the side?
If I wanted to have different views such as a top down view would I have to modify the projection or would could I just modify the corners. Currently, it seems the corners have to be setup in a very particular way.
EDIT:
It looks like simply rotating the camera itself fixes that. Thanks for sharing your implementation with me.
Yeah, gotta have the rotations of the camera and lookTarget the same, so for a top down view, rotate both 90 degrees on x (think)
Would you mind explaining the math behind your implementation? I feel this would help me better understand the scene. I was always under the impression that the projection matrix is applied to the 3D scene to map the 3d scene to 2D screen coordinates.
With your example it seems weird to me that the screen plane is in front of the 3D scene rather the other way around.
I don’t get what you mean
The screen plane (near clip plane) is always located somewhere between the camera transform and the things it’s looking at.
The projection matrix is just maths to map anything within a truncated rectangular-based pyramid to a set of 2d coordinates (technically still 3d)
Usually the projection matrix is calculated from the values for FOV and aspect, but this always results in a symetrical frustum
more reading: OpenGL Projection Matrix
I just realized my wording was off. I was becoming somewhat confused with what the blue and red outlines were representing with regards to Unity’s white outline view frustum. I forgot that now that we are overriding the projection the white frustum no longer mattered. Also, for some reason when I saw you mention screen plane and I was thinking screen coordinates.
It’d be nice to be able to disable Unity’s camera frustum outline in the editor.
If you collapse the camera component it goes
After messing with values and my models, it seems like with the final camera view my 3D models don’t quite “pop out” the way it does in the video. Do you have any suggestions that might make the illusion seem better?
What makes it look that way in the vid is that they are tracking the users position to change the view in-game. If you set up positional tracking, it’ll look like that.
Ok, I’ll try to link it up to Kinect. Also, it seems my shadows aren’t drawing with the custom projection matrix. Have you noticed this issue?
Yes. You and a dozen other people
https://www.google.co.uk/search?q=unity projection matrix shadows
Apparently you might get lucky with forward rendering, or setting other values in the camera >>>>
So, it seems like my issue is little different from others. I already have forward rendering setup. And shadows do show up, it’s just when the camera moves away from the lookTarget they began to grow fainter and then the shadows disappear all together.
Perhaps, the light has to be set in a very specific position to prevent this?
After further tests, it seems like the directional light’s position in the Unity scene does not affect how the light is shown in the scene.
Position of a directional light will not have any effect. There is no concept of the source of a directional light, its just a direction.
Maybe your far clip plane is too far
It looks like for some reason the field of view on the camera affects the shadows, which is strange because I thought the field of view no longer is taken into account the projection matrix implementation.
@hpjohn , could you tell me the best (recommended) way to setup the gameobject hierarchy? I just wanted to make sure that the results I’m getting are the intended results.
Here’s my gameobject hierarchy currently:
Id be tempted to put looktarget as a child of the camera parent. If you ever need to rotate the virtual screen, you have to rotate the camera the same, so simpler to rotate the whole construction
@hpjohn , I was trying to setup a scene with this camera where a model/mesh always stays in the center of the screen, and normally I’d use camera.transform.LookAt(target.transform), but with the custom projection, things really break down. Might you have any suggestions for this with the custom projection?
