- How does the CG shader in relation to unity treat matrices? For example, let’s say I’m making an object to tangent space matrix.
float3x3 _Obj2Tan = float3x3(base.tangent.xyz, binormal.xyz, base.normal);
lightObjectDir = mul(_Obj2Tan, lightObjectDir);
Is the float3x3 constructor treating each value as a column or as a row? I ask because if it’s being treated as a row (and correct me if I am wrong) then it’s instead a “tangent space to world” matrix versus if it’s a column, now this matrix would be the transpose (the inverse) so it is in fact, a “Object Space to Tangent Space” matrix.
If it is being treated as row, then wouldn’t I need to do a transpose of the above matrix so instead of this:
_Obj2Tan =
[t.x t.y t.z ]
[b.x b.y b.z]
[n.x n.y n.z]
to this:
_Obj2Tan =
[t.x b.x n.x]
[t.y b.y n.y]
[t.z b.z n.z]
In terms of other operations, such as if I set or got _Obj2Tan[0] or _Obj2Tan[0]=, is this setting/getting the row or column?
- Operations converting matrices such as float3x3(_matrixA) don’t touch the matrix order, just capture the upper-left 3x3 portion correct?
http://http.developer.nvidia.com/CgTutorial/cg_tutorial_chapter08.html
If you look at Example 8-5, it looks like they’re doing it how you are.
Saw that originally but must of forgotten that. However, if the CG shader uses row vector matrices (pre-multiplication), how does anything work if unity is passing column vector matrices?
Furthermore, with this fact, this slightly confuses me further. If the matrices from unity are coming in somehow ready to be pre-multiplied by a vector (converted to row vectors), what is an inverse and local to world are now reversed. So the inverse of the TBN matrix in Unity are row-vector matrices while the inverse in the CG shader are column major matrices. I don’t see however that an inverse matrix is being computed but rather another row-vector matrix which is not an object to TBN space matrix but rather a TBN to world matrix which is part of my confusion.
I’m not 100% sure of my answer, so I hope someone else will jump in. I just know Cg is pretty much HLSL which uses row vectors, but OpenGL uses column vectors. Seems confusing that you have to change the way you do your math in your shaders. I did some googling, but this was the best I could find http://www.opengl.org/discussion_boards/ubbthreads.php?ubb=showflat&Number=232964
If matrix is on the left like mul(_Obj2Tan, someVector), and if Unity has a uniform column/row convention, also as the vector on the right must be a column to be legit, then the matrix itself should be a column matrix.
But as I didn’t see anyone from Unity clarifies this, I wouldn’t say it for sure. So in practice, sometimes when doing matrix transform on my own, I just manually create a group of equations, instead of using mul() method.
This matrix is row major of course. If I do this:
tan2Obj = float3x3(base.tangent.xyz, binormal.xyz, base.normal);
lightObjectDir = mul( lightTanDir, tan2Obj );
I change the light direction in tangent space to object space. But now the matrix is the first parameter of mul function.
float3x3 _Obj2Tan = float3x3(base.tangent.xyz, binormal.xyz, base.normal);
lightObjectDir = mul(_Obj2Tan, lightObjectDir);
So it means I transpose this matrix, and it is an inversion of matrix in this case because the tangent space matrix is an orthonormal matrix.
It is clear that object space direction is transformed to tangent space direction.