Gyroscope Quaternion to Unity Camera (HELP PLEASE!)

[RobbieDingo]

Hello,

Anyone got any ideas on how to solve this please?

I’ve got a Normalised Quaternion, which is the output from the iPhone’s Gyroscope (obtained via Prime31’s DeviceMotion Plugin).

I’m trying to use this to directly control the camera view in Unity, so I would have a 360 degree rotation (ie. tilt the iPhone up = Unity camera points up, rotate the iPhone 90 degrees to the right = Unity camera turned 90 degrees right, etc.) - this is similar to the control system in N.O.V.A 2.

The problem I have when applying the Quaternion given from the Gyroscope is as follows:-

• Everything is fine whilst facing forward, I can turn the camera a bit to the left or right, and up and down OK, and also return to facing forward without too much drift.

• HOWEVER, if I turn 90 degrees to the right (or left), AND THEN attempt to look upward, the Unity camera rolls instead of tilting upward.

• From the position of 90 degrees to the right, I have to roll the iPhone in order to make the view tilt upward.

Can anyone suggest a way to interpolate the Quaternion so that the Unity camera is pointing in the same direction that the back of the iPhone is facing?

Many thanks for your assistance - it’s driving me mad.

Rob.

[RobbieDingo]

bump.

[prime31]

@RobbieDingo, have you tried the getRawGyroData method? Quaternions can be tricky beasts and for what you are trying to do using the raw gyro velocity might work with a little smoothing applied. The latest version also has a getAttitude method that will give you pitch, roll and yaw. That might actually be the best option of all for your case.

[RobbieDingo]

@Prime31, Thanks for your reply - Unfortunately, I have tried both the getRawGyroData getAttitude methods now, I’m getting the same problem as above with getAttitude method.

(Please note - getRawData is not really an option here as I have to add the RawGyroData to the current rotation and this very quickly it gives A LOT of drift - it would not be accurate enough for what I want to do).

With the getAttitude method, I found that if I multiply Pitch, Roll and Yaw by 60 (and invert/negate pitch roll), I can match the unity camera with the angle of the phone (it works REALLY well from DeviceMotionBinding.reset() IF I don’t turn or tilt the phone too much).

However, again, by the time I’ve physically turned the phone 90 degrees to the left or right, if I then attempt to look up and down, the result in the unity camera view on screen is a roll.

This code will duplicate my problem as I see it:-

function Start () {
	// kill keyboard frame
	// ...
		
	DeviceMotionBinding.start();
	var attitudeData;
	
        // DeviceMotionBinding.reset();  (Done from a GUI Button)
}

function Update () {

        attitudeData = DeviceMotionBinding.getAttitude();

	var GX = attitudeData.pitch * 60;
	var GY = attitudeData.roll * 60;
        var GZ = attitudeData.yaw * 60;

        Camera.mainCamera.transform.eulerAngles = Vector3( -GX, -GY, GZ);

}

I think that the Gyro data from all methods is correct and as it should be, but I can not think of a good way to interpret this data to work how I need it.

I’m thinking that I need to push the Attitude data for example through some kind of a translation or matrix so that each element is converted to take into account the relative offset of the other elements???

BUT - I’m just not sure how to go about doing that…? Hoping you, or someone here will have a suggestion.

[spinaljack]

How about placing the camera inside several layers of empty game objects, one for each axis of rotation, and then only rotating the y-axis on the top most object and only rotating the x-axis (looking up and down) on the child of that game object.

I haven’t tried it but there was a similar problem with flight sim controls which was solved like this.

Try it with the attitude data.

There’s probably a programatic method of doing it but this method is easier to get your head around.

[RobbieDingo]

@spinaljack, thanks a lot for your idea - unfortunately I just tried it and it does not work either

I used eulerAngles for the top game object and localEulerAngles for the two children objects.

(It gives exactly the same result as above)

I guess logically, when you think about it, your idea isn’t doing anything different, because it is just following the Gyro data which, when I’ve already turned 90 degrees, is outputting a roll instead of an up and down motion.

I can’t believe there isn’t a solution to this available anywhere online, I done lots of searching, I would have thought the problem would be a well known one.

Still driving me nuts this one…

Any other ideas anyone!?

[RobbieDingo]

Message I emailed to Mike at Prime31:- (copying here in case anyone is following progress?)

I’m still having trouble trying to get data from the iphone’s gyro in a format that will work for 360 degree tracking via the Prime31 DeviceMotion plugin in unity.

• The RAW GYRO method almost gives me a workable solution, however this method is very prone to drift due the additive nature - so by the time I’ve moved the device for a few seconds, the unity camera is no longer alined correctly to the orientation of the device.

• The QUATERNION or ATTITUDE methods do give much more stable results, but unfortunately the values given do not relate to the orientation of the device itself, rather they appear to independently relate to the gyroscope’s reset() reference?

So, for example, if I’m standing facing directly north holding my device upright with the screen to my face, reset(), then twist the device from portrait to landscape, the attitude.yaw element will show movement as expected, OK. However, if I then turn to face east (or west) a similar twist of the device will now show movement on attitude.pitch instead.

I’ve tried everything I can think of (and them some!) to combine the data in the Quaternion or the Attitude elements (p, r y) in an effort to translate to something that actually relates to the direction that the device is pointing - but as you can see from my example above, because BOTH yaw and pitch will be used to represent a twist (depending upon which way the device itself is facing) - I can’t find a way to successfully track the device’s orientation.

I’m sure a solution must be possible here because I’ve seen this kind of thing successfully implemented in many iOS apps.

I note from Apple’s Dev Centre docs that there is also a ‘Rotation Matrix’ method for the Gyro, but this is not currently accessible from the DeviceMotion Plugin? Do you think that this method would help to resolve my problem at all? - and if so, is this something you would consider exposing in the next update of your DeviceMotion plugin?

I would appreciate your thoughts on whether the ‘Rotation Matrix’ is the missing link here - or do you think I just need to keep trying ways to translate the Attitude data I currently have?

With kind regards, and thanks again for your wonderful plugins!

[outtoplay]

So the gyro plugin from Prim31 does offer gyro control over the camera in IOS apps?
I meant to email them about it, but since this thread popped up I figured I would inquire here. There’s no info, description, demo for this plight on their site. I was specifically interested in the now infamous Nintendo DSIware Diarama video on YouTube.

Thanks,
B.

[RobbieDingo]

yes it works great - I’m just having trouble with this particular application of the gyro data.

[prime31]

@RobbieDingo, the rotation matrix can definitely be added to the plugin but I don’t think it is the missing link you seek. I left the rotation matrix out because most folks grasp on euler angles and quaternions is far, far greater than having to do matrix math for processing the gyro data. Have you dug around the Apple dev forums? You may find some good algorithms for applying smoothing to the raw gyro data in there that could lead you down the right path.

[RobbieDingo]

Thanks, yeah Mike, believe me I’ve looked everywhere!

Besides, using the raw gyro data (even in it’s rawest, un-smoothed form) leads to results which exhibit quite a lot of drift. (try a quick movement, followed by a slow returning movement to see what I mean).

Yes I can easily smooth the raw gyro data, that’s very easy to achieve, but in doing so I’m effectively smoothing data which is already not accurate enough, thus leading to worse results if you see what I mean?

Some people reading this might think that the problem is that I’m expecting too much from the gyro maybe?

However, the Attitude data provided by the Gyro is rock solid in comparison to the technique of accumulating and summing raw gyro data over time.

Whilst it is very straightforward to make the UT camera roughly track the movement of the device using the raw gyro method (smoothed or un-smoothed) - I’m convinced that this is not the way to do it if you need any kind of accuracy over time, as I do.

Grrrr.

[spinaljack]

Has someone got an example of using raw gyro data to turn a camera?
It doesn’t have to be 1:1 perfect

[spinaljack]

@RobbieDingo

Have you tried using Mathf.Sin to combine the x and z axes relative to the y-axis?

something like this:

function Update () {

attitudeData = DeviceMotionBinding.getAttitude();

var GX = attitudeData.roll * 60;
var GY = attitudeData.pitch * 60;
var GZ = attitudeData.yaw * 60;

var yRad = Mathf.Deg2RadGY;
var resultantX = GXMathf.Sin(yRad+Mathf.PI*.5) + GZMathf.Sin(yRad);
var resultantZ = GZMathf.Sin(yRad+Mathf.PI*.5) + GX*Mathf.Sin(yRad);

Camera.mainCamera.transform.eulerAngles = Vector3( -resultantX, -GY, resultantZ);
}

May need swapping symbols around but basically Sin(angle+1/2PI) will be 1 when facing forward and 0 when facing left or right and -1 when facing backwards. Then when you’re at an angle between 0 and 90 the resultant X value will be a combination of GX and GZ

[spinaljack]

I’ve noticed I’ve had to swap pitch and roll around, dunno if you have the same result

Okay now it works alright up until 90 degrees to the side and then suddenly the camera rotates out of control

So confused!

EDIT

Okay that’s probably due to the euler angles, lets try clamping the values…

EDIT EDIT

Doesn’t help, I think there’s a problem with the attitude data. When turning up to the 90 degree mark the camera goes crazy. This didn’t happen when using the normalized gyro rotation…

[spinaljack]

For anyone who’s interested I’ve got a workable solution:

#pragma strict
function Start () {
	// kill keyboard frame
	..
		
	DeviceMotionBinding.start(); 
}

function Update () {
    var quat : Quaternion = DeviceMotionBinding.getNormalizedQuarternion();
		
    var GX = quat.eulerAngles.x;
    var GY = quat.eulerAngles.y;
    var GZ = quat.eulerAngles.z;
		
    if(GX>180) GX-=360;
    if(GX<-180) GX+=360;
		
    if(GZ>180) GZ-=360;
    if(GZ<-180) GZ+=360;


    var xRad = Mathf.Sin(Mathf.Deg2Rad*(GY+90.0));
    var zRad = Mathf.Sin(Mathf.Deg2Rad*(GY+180.0));
    var resultantX = (GX*xRad)-(GZ*zRad);
    var resultantZ = (GX*zRad)+(GZ*xRad);
    
    if (resultantX < -360)
		resultantX += 360;
    if (resultantX > 360)
		resultantX -= 360;
		
    if (resultantZ < -360)
		resultantZ += 360;
    if (resultantZ > 360)
		resultantZ -= 360;
    
    transform.eulerAngles = Vector3(resultantX, -GY, resultantZ);
}

The only issue is that it stops working if you turn your iDevice upside down

I suspect you can solve the upside down problem with one more sin calculation for the z axis but I’m tired of this problem so hopefully someone can fill in that last equation.

[RobbieDingo]

Cool, thanks spinaljack.

[mikriloulou]

Did you guys manage to find a good solution to that problem? I am trying to do the same thing as RobbieDingo did and I have tried several thinks. I used spilaljack’s solution and it works pretty fine for the Y axis(if I keep the iPad in Landscape left orientation) but if I rotate a bit on the X or the Z the camera goes crazy. It seems that you guys were the only ones in the forum trying to deal with this, so I wonder whether you worked that out.

[marjan]

Here is my solution. Worked perferctly for the following:

Camera is in center, rotationg the device will transform the cam 360 degree on any axis.

(i hope i copied out the right scipt, didn´t work on that for some weeks)

using UnityEngine;
using System.Collections;


public class DeviceMotionGUIManager2 : MonoBehaviour
{
	// CameraTexture Vars
	
	public GameObject target; // the object to apply the camera texture to
    private Texture2D texture; // local reference of the texture
	
	// Gyro Vars
	public GUIText text1;
	public GUIText text2;
	public GUIText text3;
	
	private float radToDegF;
	

	void Start()
	{
		radToDegF = (180.0f / Mathf.PI);
		Debug.Log("radToDegF = " + radToDegF);
		
		texture = ARBinding.startCameraCapture( false, ARQuality.Low );
	    target.renderer.material.mainTexture = texture;
	    ARBinding.updateMaterialUVScaleForTexture( target.renderer.sharedMaterial, texture );
	}
	
	void Update()
	{

		// modify the cameras angle slightly as the devices moves around
		// for a more convincing effect you can modify the cameras FOV
		//Camera.mainCamera.transform.eulerAngles = _startRotation + new Vector3( attitude.pitch * -30.0f, attitude.roll * 30.0f, attitude.yaw * 20.0f );

	/*
		text1.text = string.Format( "pitch: {0}", newPitch );
		text2.text = string.Format( "roll: {0}", newRoll );
		text3.text = string.Format( "yaw: {0}", newYaw);
	*/
		Quaternion gyroQuaternion  = DeviceMotionBinding.getNormalizedQuarternion();
	
		Camera.mainCamera.transform.rotation = gyroQuaternion;
		Vector3 ea = gyroQuaternion.eulerAngles;
		Camera.mainCamera.transform.eulerAngles = new Vector3( -ea[0], -ea[1], ea[2]);

	}
	
	
	void OnGUI()
	{
		float yPos = 0.0f;
		float xPos = 5.0f;
		float width = 160.0f;
		float buttonHeight = 40.0f;
		float heightPlus = buttonHeight + 5.0f;
		
		
		if( GUI.Button( new Rect( xPos, yPos, width, buttonHeight ), "Start DeviceMotion" ) )
		{
			DeviceMotionBinding.start();
			DeviceMotionBinding.reset();
			
		}
		
		
		if( GUI.Button( new Rect( xPos, yPos += heightPlus, width, buttonHeight ), "Stop DeviceMotion" ) )
		{
			DeviceMotionBinding.stop();
		}
		
		
		if( GUI.Button( new Rect( xPos, yPos += heightPlus, width, buttonHeight ), "Reset DeviceMotion" ) )
		{
			DeviceMotionBinding.reset();
		}


		if( GUI.Button( new Rect( xPos, yPos += heightPlus * 2, width, buttonHeight ), "Previous Scene" ) )
		{
			Application.LoadLevel( "DeviceMotionTestScene" );
		}
	}
}

[spinaljack]

Camera.mainCamera.transform.rotation = gyroQuaternion;
Vector3 ea = gyroQuaternion.eulerAngles;
Camera.mainCamera.transform.eulerAngles = new Vector3( -ea[0], -ea[1], ea[2]);

Why would you use both transform.rotation and transform.eulerAngles?

[marjan]

I don´t know. I was just testing this, had the same problem as others, were the camera rotates or flips at some point.
After a lot of trying i got to a point were everything seemed ok.
The Code is messy, people need to clean it up themselves.