Edit: This was solved. Quaternion.LookRotation C# (re-)implementation (+ standalone 3x3 orthogonal rotation matrix → quaternion conversion) can be found in post #9 . + bonus content: Quaternion.ToAngleAxis in post #12 .
Heya,
First of all, I’m doing this for learning purposes. Please don’t instruct me to not reinvent the wheel.
Next, before jumping to conclusions, let me say this: I’ve tried everything one can find on Google. Or ChatGPT. Or Copilot. Or Wikipedia. Or Siggraph papers. Coding gems. Decompiled Unity source. You name it. It is all wrong. Or I’m not getting something big time.
Of course, it’s extremely likely that I’m missing a step here and tbh I’m not that versed in eigenvalues and matrix decomposition, SO(3), hyperspheres, Lie algebras etc. Not to mention the handedness jungle, randomly transposed matrices, various quaternion standards (ijkw, wijk, wxyz, xyzw …), as well as Unity itself being non-transparent about it…
I’m honestly a dum-dum when it comes to “ah you just do #&^# @&^@# and then because w is octohugging the perplexdome of the verticultural jynxopsis you can #$&# @&# and look how easy is that, you arrive at ∫Z(x^2) e^ipi and that’s your solution.”
I’m sorry but if I wanted to feel depressed I’d play Starfield instead.
Anyway I also checked out the source for Unity mathematics, but this particular thing (in float3x3) looks like pizza from hell. It’s impossible to tell what’s going on with all the conversions from uint to float, extracting signs then XORing them back…
These two are correct however, so I have something to lean onto
var r = Quaternion.LookRotation(fwd, up);
and
Vector3 rgt = cross(up, fwd);
up = cross(fwd, rgt);
var r = new Matrix4x4(rgt, up, fwd, new Vector4(0f, 0f, 0f, 1f)).rotation;
(Both of these will jump to the C++ side for the actual conversion and I believe it’s the exact same code.)
And so obviously I used the latter as a basis for my analysis, because the actual matrix is clear as day.
| Rx Ux Fx 0 |
| Ry Uy Fy 0 |
| Rz Uz Fz 0 |
| 0 0 0 1 |
Here’s what I tried:
If I learned anything it’s that the quaternion is obtained by extracting the ‘trace of the matrix’ T
T = Rx + Uy + Fz;
And then w = sqrt(1 + T) / 2
Most of the implementations I’ve seen have branching where they take some piece of the matrix with the “biggest meat on it” and reshuffle all components in order to avoid divisions by zero. I’ve also seen many odd takes that claim to be numerically stable etc, on StackOverflow and in Unity Answers, but when you take it all in, some are just naive, or somehow messed up, and many are literal rewrites of the same thing over and over.
After some time, I’ve grown impatient with trying to find the whole thing in a pristine condition, so I’ve decided to probe the values themselves in a contrived scenario.
First, let’s say we don’t care about the signs. Then we note that every single implementation uses the above formula for exactly one of the final components (of course T is true only for w and the expression will change for x, y, z, so let’s call this N instead).
Because this evaluation only needs a diagonal, it doesn’t matter whether I’ve transposed the matrix (so at least that’s out). And it should be guaranteed that the value of sqrt(1 + N) / 2 must appear somewhere in the quaternion. However there are 4 possible expressions for N
N = Rx + Uy + Fz; // w (T)
N = Rx - Uy - Fz; // x
N = -Rx + Uy - Fz; // y
N = -Rx - Uy + Fz; // z
This is the scenario I came up with:
With some Target at (2, 8, 2), we get the following “LookRotation” basis vectors
R ( 0.23570, 0.00000, -0.23570)
U (-0.22222, 0.11111, -0.22222)
F ( 0.23570, 0.94281, 0.23570)
How to arrive here
var target = new Vector3(2f, 8f, 2f);
var fwd = target.normalized; // unit
var up = Vector3.up; // unit
var rgt = cross(up, fwd); // Edit: I should've normalized this; it's so funny I didn't write 'unit' here -- my brain apparently knew it all along
up = cross(fwd, rgt); // re-orthogonalized
Debug.Log($"R {rgt:F5}");
Debug.Log($"U {up:F5}");
Debug.Log($"F {fwd:F5}");
The diagonal of this matrix (Rx, Uy, Fz) (or in another notation m00, m11, m22) is thus
(0.23570, 0.11111, 0.23570)
LookRotation (and the other Matrix4x4.rotation example) will return this quaternion
(-0.53340, 0.31246, 0.22094, 0.75434)
How to arrive here
var q = Quaternion.LookRotation(new Vector3(2f, 8f, 2f).normalized, Vector3.up);
Debug.Log(q);
But when I ran sqrt(1 + N) / 2 (for all N as shown above) none of the values I got from it matched with the bolded values. NONE! To be safe, I included 4 more sign permutations (for a complete set of 2^3=8) and nothing. I also tried normalizing the quaternion, which shouldn’t be required, to no avail. I tried as well permuting the signs on output or changing sqrt(1 + N) / 2 to absurdity. I don’t even know why I tried that, I guess I’m at the point of randomizing all of the things I tried so far in hopes of stumbling into some miraculous insight.
So the question is, how on Earth one obtains the bolded values above?
Apart from the obvious results in Google, here’s a useful PDF that mentions two of the most prominent solutions I’ve seen before (and any AI or Quora bots will readily recite these like a psalm). I’ve tried numerous variations of the both examples shown here and nothing made any sense. Edit: there is also this on Wikipedia, that states the exact same thing.
Where is my error?
Edit: Fixed trace of matrix from 1+Rx+Uy+Fz to just Rx+Uy+Fz as it should be (1 is pulled out to sqrt(1 + N) / 2). Nothing changes because of it, I just thought it would look more correct.
Edit2: After I found the bug, I added some remarks above. The issue was that I should’ve normalized the right vector because up and forward aren’t guaranteed to be orthogonal at that moment.
