(Oculus Rift) Casting a ray from mouse through distortion matrix

Hi everyone. I’ve searched the board, as well as the oculus board, and stack overflow. Couldn’t really find something that helped.

I’m working on a vehicle simulation. Before we started using the oculus, it was just a regular first person perspective. You used a racing wheel/pedals to drive and the mouse to control all the buttons and switches etc. We use raycasting from the mouse point on the screen into the world to interact with the various controls in the vehicle.

Now that we’re using the oculus, the raycast isn’t taking into account the distortion matrix used on the oculus cameras. So you’re not actually casting a ray at what you’re visually clicking on. Using Debug.DrawRay I found that it was slightly off. Just to be sure, I disabled the lens correction via inspector on the OVRCameraController and sure enough the raycasting was working again.

The ray itself is calculated the usual way one does when firing from the mouse point:

Would anyone have any idea how I can adjust my ray so it works with lens correction on?


Oculus requires two cameras right? Are you raycasting from the center of the two cameras? You might have to do some logic to see what camera to use for raycasting, or raycast from the center of the two cameras.

Or, better yet, simulate the mouse in your game from the mouse input. That way, it is under the Oculus Distortion. Then, you can raycast from the simulated cursor.

Use something similar to a fps mouse script, except change it to move a cursor:

using UnityEngine;
using System.Collections;
/// MouseLook rotates the transform based on the mouse delta.
/// Minimum and Maximum values can be used to constrain the possible rotation
/// To make an FPS style character:
/// - Create a capsule.
/// - Add the MouseLook script to the capsule.
///   -> Set the mouse look to use LookX. (You want to only turn character but not tilt it)
/// - Add FPSInputController script to the capsule
///   -> A CharacterMotor and a CharacterController component will be automatically added.
/// - Create a camera. Make the camera a child of the capsule. Reset it's transform.
/// - Add a MouseLook script to the camera.
///   -> Set the mouse look to use LookY. (You want the camera to tilt up and down like a head. The character already turns.)
[AddComponentMenu("Camera-Control/Mouse Look")]
public class MouseLook : MonoBehaviour {
	public enum RotationAxes { MouseXAndY = 0, MouseX = 1, MouseY = 2 }
	public RotationAxes axes = RotationAxes.MouseXAndY;
	public float sensitivityX = 15F;
	public float sensitivityY = 15F;
	public float minimumX = -360F;
	public float maximumX = 360F;
	public float minimumY = -60F;
	public float maximumY = 60F;
	float rotationY = 0F;
	void Update ()
		if (axes == RotationAxes.MouseXAndY)
			float rotationX = transform.localEulerAngles.y + Input.GetAxis("Mouse X") * sensitivityX;
			rotationY += Input.GetAxis("Mouse Y") * sensitivityY;
			rotationY = Mathf.Clamp (rotationY, minimumY, maximumY);
			transform.localEulerAngles = new Vector3(-rotationY, rotationX, 0);
		else if (axes == RotationAxes.MouseX)
			transform.Rotate(0, Input.GetAxis("Mouse X") * sensitivityX, 0);
			rotationY += Input.GetAxis("Mouse Y") * sensitivityY;
			rotationY = Mathf.Clamp (rotationY, minimumY, maximumY);
			transform.localEulerAngles = new Vector3(-rotationY, transform.localEulerAngles.y, 0);
	void Start ()
		// Make the rigid body not change rotation
		if (rigidbody)
			rigidbody.freezeRotation = true;