Off axis projection with Unity

Unity Engine
1

Hi,

I have been trying to get the off-axis projection with the camera in unity to get the perspective of a scene from different positions. Initially, I thought I would have to fiddle with the camera’s projection matrices myself and write code to build an assymetric frustum. However, I found something which is already out there and tried using it. Here’s the link: http://en.wikibooks.org/wiki/Cg_Programming/Unity/Projection_for_Virtual_Reality

The above has a script for implementing assymetric frustum but it doesn’t seem to me to be working properly somehow. The script however seems written fine to me and is more or less which I would have written as well (except the fixed corners for the viewing plane specified as (±5, 0, ±5)).

The frustum doesn’t seem to be taking the corners of the plane specified and on moving the camera, it also doesn’t seem to stay stuck to the corners of the plane specified, In addition, I am getting a console error as well with the message at:

      camera.projectionMatrix = p * rm * tm;

saying !IsMatrixValid(matrix)
UnityEngine.Camera:set_projectionMatrix(Matrix4x4)

#pragma strict
 
public var projectionScreen : GameObject;
public var estimateViewFrustum : boolean = true;
 
function LateUpdate() {
   if (null != projectionScreen)
   {
      var pa : Vector3 =  
         projectionScreen.transform.TransformPoint(
         Vector3(-5.0, 0.0, -5.0));
         // lower left corner in world coordinates
      var pb : Vector3 = 
         projectionScreen.transform.TransformPoint(
         Vector3(5.0, 0.0, -5.0));
         // lower right corner
      var pc : Vector3 = 
         projectionScreen.transform.TransformPoint(
         Vector3(-5.0, 0.0, 5.0));
         // upper left corner
      var pe : Vector3 = transform.position;
         // eye position
      var n : float = camera.nearClipPlane;
         // distance of near clipping plane
      var f : float = camera.farClipPlane;
         // distance of far clipping plane
 
      var va : Vector3; // from pe to pa
      var vb : Vector3; // from pe to pb
      var vc : Vector3; // from pe to pc
      var vr : Vector3; // right axis of screen
      var vu : Vector3; // up axis of screen
      var vn : Vector3; // normal vector of screen
 
      var l : float; // distance to left screen edge
      var r : float; // distance to right screen edge
      var b : float; // distance to bottom screen edge
      var t : float; // distance to top screen edge
      var d : float; // distance from eye to screen 
 
      vr = pb - pa;
      vu = pc - pa;
      vr.Normalize();
      vu.Normalize();
      vn = -Vector3.Cross(vr, vu); 
         // we need the minus sign because Unity 
         // uses a left-handed coordinate system
      vn.Normalize();
 
      va = pa - pe;
      vb = pb - pe;
      vc = pc - pe;
 
      d = -Vector3.Dot(va, vn);
      l = Vector3.Dot(vr, va) * n / d;
      r = Vector3.Dot(vr, vb) * n / d;
      b = Vector3.Dot(vu, va) * n / d;
      t = Vector3.Dot(vu, vc) * n / d;
 
      var p : Matrix4x4; // projection matrix 
      p[0,0] = 2.0*n/(r-l); 
      p[0,1] = 0.0; 
      p[0,2] = (r+l)/(r-l); 
      p[0,3] = 0.0;
 
      p[1,0] = 0.0; 
      p[1,1] = 2.0*n/(t-b); 
      p[1,2] = (t+b)/(t-b); 
      p[1,3] = 0.0;
 
      p[2,0] = 0.0;         
      p[2,1] = 0.0; 
      p[2,2] = (f+n)/(n-f); 
      p[2,3] = 2.0*f*n/(n-f);
 
      p[3,0] = 0.0;         
      p[3,1] = 0.0; 
      p[3,2] = -1.0;        
      p[3,3] = 0.0;             
 
      var rm : Matrix4x4; // rotation matrix;
      rm[0,0] = vr.x; 
      rm[0,1] = vr.y; 
      rm[0,2] = vr.z; 
      rm[0,3] = 0.0;    
 
      rm[1,0] = vu.x; 
      rm[1,1] = vu.y; 
      rm[1,2] = vu.z; 
      rm[1,3] = 0.0;    
 
      rm[2,0] = vn.x; 
      rm[2,1] = vn.y; 
      rm[2,2] = vn.z; 
      rm[2,3] = 0.0;    
 
      rm[3,0] = 0.0;  
      rm[3,1] = 0.0;  
      rm[3,2] = 0.0;  
      rm[3,3] = 1.0;            
 
      var tm : Matrix4x4; // translation matrix;
      tm[0,0] = 1.0; 
      tm[0,1] = 0.0; 
      tm[0,2] = 0.0; 
      tm[0,3] = -pe.x;  
 
      tm[1,0] = 0.0; 
      tm[1,1] = 1.0; 
      tm[1,2] = 0.0; 
      tm[1,3] = -pe.y;  
 
      tm[2,0] = 0.0; 
      tm[2,1] = 0.0; 
      tm[2,2] = 1.0; 
      tm[2,3] = -pe.z;  
 
      tm[3,0] = 0.0; 
      tm[3,1] = 0.0; 
      tm[3,2] = 0.0; 
      tm[3,3] = 1.0;            
 
      // set matrices
      camera.projectionMatrix = p * rm * tm;
      camera.worldToCameraMatrix = Matrix4x4.identity; 
      // we put everything into the projection matrix: 
      // because our "viewing matrix" might look at a  
      // point that is off the screen.
 
      if (estimateViewFrustum)
      {
         // rotate camera to screen for culling to work
         var q : Quaternion;
         q.SetLookRotation((0.5 * (pb + pc) - pe), vu); 
             // look at center of screen
         camera.transform.rotation = q;
 
         // set fieldOfView to a conservative estimate 
         // to make frustum tall enough
         if (camera.aspect >= 1.0)
         { 
            camera.fieldOfView = Mathf.Rad2Deg * 
               Mathf.Atan((vu.magnitude + vr.magnitude) 
               / va.magnitude);
         }
         else 
         {
             // take the camera aspect into account to 
             // make the frustum wide enough 
             camera.fieldOfView = 
                Mathf.Rad2Deg / camera.aspect *
                Mathf.Atan((vu.magnitude + vr.magnitude) 
                / va.magnitude);
         }      
      }
   }
}

Any suggestions on how to correct this and get the proper effect are very welcome.

2

This page has a much shorter example on setting the camera to a custom projection
Plus, are you sure its not the “estimateViewFrustum” just making the cone look wrong, even though the camera is projecting correctly; the frustum gizmo cannot correctly display an offaxis projection.
When I did this, i made it Debug.drawlines instead

3

Hey, I found the above link as well and this one was working pretty well out of the box with the hardcoded values of left, right, bottom, top when I tested it with. However, I still have to test it by placing the camera to the user’s head position, and calculating the left, right, bottom, top from the projection screen. This again would boil down to the same thing that the above script in wikibooks is doing because eventually I wanted to get that VR system kinda environment working.

I could still write the code to place the camera at user’s head moving with the values coming from the tracker but the above wikibooks code also takes into account the rotation that might be there in the projection screen. For example, if the projection where I am getting the off-axis projection output is tilted at an angle of 45 degree to the XY plane.

I tested it both estimateViewFrustum as well as with the option turned off but I wasn’t able to notice any differnece or an output that seemed correct to me. The camera didn’t show any output and I always get a blue screen everywhere I place my test object.

Thanks for the idea about Debug.DrawLines. I wasn’t aware about that and I was checking everytime switching to Game Output but I’ll just put these lines right in.

4

Looked up my old project, and looks like i never bothered with the rotation and translation matrix

using UnityEngine;
	public Transform[] Corners;
	public Transform lookTarget;
	public bool drawNearCone, drawFrustum;

	Camera theCam;

	void Start () {
		theCam = camera;
	}

	void Update () {
		Vector3 pa, pb, pc, pd;
		pa = Corners[0].position;
		pb = Corners[1].position;
		pc = Corners[2].position;
		pd = Corners[3].position;

		Vector3 pe = theCam.transform.position;// eye position

		Vector3 vr = ( pb - pa ).normalized; // right axis of screen
		Vector3 vu = ( pc - pa ).normalized; // up axis of screen
		Vector3 vn = Vector3.Cross(vr, vu).normalized; // normal vector of screen

		Vector3 va = pa - pe; // from pe to pa
		Vector3 vb = pb - pe; // from pe to pb
		Vector3 vc = pc - pe; // from pe to pc
		Vector3 vd = pd - pe; // from pe to pd

		float n = -lookTarget.InverseTransformPoint(theCam.transform.position).z; // distance to the near clip plane (screen)
		float f = theCam.farClipPlane; // distance of far clipping plane
		float d = Vector3.Dot(va, vn); // distance from eye to screen
		float l = Vector3.Dot(vr, va) * n / d; // distance to left screen edge from the 'center'
		float r = Vector3.Dot(vr, vb) * n / d; // distance to right screen edge from 'center'
		float b = Vector3.Dot(vu, va) * n / d; // distance to bottom screen edge from 'center'
		float t = Vector3.Dot(vu, vc) * n / d; // distance to top screen edge from 'center'

		Matrix4x4 p = new Matrix4x4(); // Projection matrix 
		p[0, 0] = 2.0f * n / ( r - l );
		p[0, 2] = ( r + l ) / ( r - l );
		p[1, 1] = 2.0f * n / ( t - b );
		p[1, 2] = ( t + b ) / ( t - b );
		p[2, 2] = ( f + n ) / ( n - f );
		p[2, 3] = 2.0f * f * n / ( n - f );
		p[3, 2] = -1.0f;

		theCam.projectionMatrix = p; // Assign matrix to camera

		if (drawNearCone) { //Draw lines from the camera to the corners f the screen
			Debug.DrawRay(theCam.transform.position, va, Color.blue);
			Debug.DrawRay(theCam.transform.position, vb, Color.blue);
			Debug.DrawRay(theCam.transform.position, vc, Color.blue);
			Debug.DrawRay(theCam.transform.position, vd, Color.blue);
		}

		if (drawFrustum) DrawFrustum(theCam); //Draw actual camera frustum

	}

	Vector3 ThreePlaneIntersection(Plane p1, Plane p2, Plane p3) { //get the intersection point of 3 planes
		return ( ( -p1.distance * Vector3.Cross(p2.normal, p3.normal) ) +
				( -p2.distance * Vector3.Cross(p3.normal, p1.normal) ) +
				( -p3.distance * Vector3.Cross(p1.normal, p2.normal) ) ) /
			( Vector3.Dot(p1.normal, Vector3.Cross(p2.normal, p3.normal)) );
	}

	void DrawFrustum (Camera cam) {
		Vector3[] nearCorners = new Vector3[4]; //Approx'd nearplane corners
		Vector3[] farCorners = new Vector3[4]; //Approx'd farplane corners
		Plane[] camPlanes = GeometryUtility.CalculateFrustumPlanes(cam); //get planes from matrix
		Plane temp = camPlanes[1]; camPlanes[1] = camPlanes[2]; camPlanes[2] = temp; //swap [1] and [2] so the order is better for the loop

		for (int i = 0; i < 4; i++) {
			nearCorners[i] = ThreePlaneIntersection(camPlanes[4], camPlanes[i], camPlanes[( i + 1 ) % 4]); //near corners on the created projection matrix
			farCorners[i] = ThreePlaneIntersection(camPlanes[5], camPlanes[i], camPlanes[( i + 1 ) % 4]); //far corners on the created projection matrix
		}

		for (int i = 0; i < 4; i++) {
			Debug.DrawLine(nearCorners[i], nearCorners[( i + 1 ) % 4], Color.red, Time.deltaTime, false); //near corners on the created projection matrix
			Debug.DrawLine(farCorners[i], farCorners[( i + 1 ) % 4], Color.red, Time.deltaTime, false); //far corners on the created projection matrix
			Debug.DrawLine(nearCorners[i], farCorners[i], Color.red, Time.deltaTime, false); //sides of the created projection matrix
		}
	}
}

And in the scene Ive got

v HeadConstruction
	v LookTarget
		CornerA
		CornerB
		CornerC
		CornerD // Not actually needed except for visualistion
	Camera // Script goes here
5

@hpjohn : With this being my first hands-on tryout with Unity, I am not sure how to assign the Transform to the Transform variables/array variables exposed from the script. I saw above about the elements in your Scene but can’t seem to make out what game objects/elements those things are. What should I be creating in the scene view and assigning in the Main Camera’s script in the inspector to replicate what you got above?

Here’s what my camera look like when I tried the above: https://www.dropbox.com/s/kgdulakcwui5p31/CamTest4.zip [Four corners do not stay still as in your animation above]

From the cool gif animation you posted, the output seems to be the same as to what I have been looking for.

6

Make 6 objects, plus the camera (i used spheres so you can see where they are, but you could just use empty objects, or disable their renderers later)
Assuming you have the default camera in a blank scene:
Move the camera down 1 unit to (0,0,-10)
4 objects (corners A-D) are positioned in a rectangle at (4,3,0), (-4,-3,0) etc
LookTarget is position in the middle of them at (0,0,0)
Drag the 4 corners onto LookTarget in the Hierarchy to make them children
Create one more object at the same position as the camera (0,0,-10), as head construction and drag look target and the camera onto it as children
Put the script on the camera, and then drag the 4 corners and looktarget into the slots in the inspector

This is a pretty complex thing to want in a first time project
What happened to basic pong/simple tetris/simple driving

7

You forgot text based adventure games!

Computers have spoiled us all.
Now first timers want to make spaceships that land on planets from space and can go from system to system buying, selling and shooting everything in their path AND look good. Unity3D at least makes it somewhat possible to achieve this out of the box. I’d say swing for the fences boyo’ (if it’s not your day job) you’ll soon find out if you bit off more than you can chew.