Camera normals to Worldspace normals

Unity Engine
,
1

Hi!

I’ll want to implement a postprocessing effect to recoloring the viewed surfaces.

8257524--1080903--upload_2022-7-6_12-41-40.png
8257524--1080903--upload_2022-7-6_12-41-40.png859×456 214 KB

8257524--1080912--upload_2022-7-6_12-47-59.png
8257524--1080912--upload_2022-7-6_12-47-59.png969×422 257 KB

The fragment code:

fixed4 frag (v2f i) : SV_Target
{
   float3 normals = decode_normal(tex2D(_CameraDepthNormalsTexture, i.uv));
   float3 world_up = float3(0.0, 1.0, 0.0);
   float t = dot(normals, world_up);

    return lerp(tex2D(_MainTex, i.uv), _SurfaceColor, smoothstep(_SurfaceSensibility, 1.0, t));
}

The fragment code works well, the main problem is, the effect is not projected in world space, but in viewspace.

Example:

8257524--1080906--upload_2022-7-6_12-47-3.png
8257524--1080906--upload_2022-7-6_12-47-3.png1278×695 359 KB

8257524--1080909--upload_2022-7-6_12-47-30.png
8257524--1080909--upload_2022-7-6_12-47-30.png1201×634 247 KB

How can i convert the view normal to world space normals ignoring the camera rotation and position?

2

There are two options. If you know the normals are in view space, you can either transform the “world up” vector into view space, or transform the normals from world space into view space.

fixed4 frag (v2f i) : SV_Target
{
   float3 normals = decode_normal(tex2D(_CameraDepthNormalsTexture, i.uv));
   float3 world_up = mul(UNITY_MATRIX_V, float3(0.0, 1.0, 0.0)); // transform from world to view space
   float t = dot(normals, world_up);

    return lerp(tex2D(_MainTex, i.uv), _SurfaceColor, smoothstep(_SurfaceSensibility, 1.0, t));
}

fixed4 frag (v2f i) : SV_Target
{
   float3 normals = mul(decode_normal(tex2D(_CameraDepthNormalsTexture, i.uv)), UNITY_MATRIX_V); // transform from view to world space
   float3 world_up = float3(0.0, 1.0, 0.0);
   float t = dot(normals, world_up);

    return lerp(tex2D(_MainTex, i.uv), _SurfaceColor, smoothstep(_SurfaceSensibility, 1.0, t));
}

Note, there’s no view to world space matrix, but the world to view space matrix is guaranteed to be an orthogonal matrix (fancy way of saying the matrix isn’t oddly scaled or warped in any way), and a handy property of an orthogonal matrix is the transpose and inverse are identical. You use mul() with a vector before the matrix, it applies the transpose of that matrix, so that’s the view to world matrix.

1 Like
3

Hi ben, nice to meet you.

I’ve tryed as you say, to transform the view normals to world normals.
But i’ve not notice any change at all, until i try to check differences:

fixed4 frag(v2f i) : SV_Target
{
    float3 normals_world_space = mul(decode_normal(tex2D(_CameraDepthNormalsTexture, i.uv)), UNITY_MATRIX_V);
    float3 normals_view_space = decode_normal(tex2D(_CameraDepthNormalsTexture, i.uv));
    return float4(normals_world_space - normals_view_space, 1.0);
}

And as i notice, there are no differences (Black == 0)
8262468--1081833--upload_2022-7-8_12-32-31.png

It doesn’t change even with transposing the vector:

fixed4 frag (v2f i) : SV_Target
{
    float3 normals_view_space = decode_normal(tex2D(_CameraDepthNormalsTexture, i.uv));
    float3 world_up = mul(UNITY_MATRIX_V, float3(0.0, 1.0, 0.0));
    float t = dot(normals_view_space, world_up);
    return lerp(tex2D(_MainTex, i.uv), _SurfaceColor, smoothstep(_SurfaceSensibility, 1.0, t));
}

Sideview:
8262468--1081848--upload_2022-7-8_12-40-10.png

top view:

8262468--1081851--upload_2022-7-8_12-42-41.png
8262468--1081851--upload_2022-7-8_12-42-41.png792×527 239 KB

4

Whoops, right. This is a post process, so the UNITY_MATRIX_V will be an identity matrix (no rotation or translation, and a uniform scale of 1). You want to use unity_WorldToCamera, like this:

float3 normals_view_space = decode_normal(tex2D(_CameraDepthNormalsTexture, i.uv));
float3 world_up = mul(unity_WorldToCamera, float3(0.0, 1.0, 0.0)) * float3(1,1,-1); // because reasons ...
float t = dot(normals_view_space, world_up);

To explain that * float3(1,1,-1), Unity’s View space is -Z forward, but the unity_WorldToCamera matrix is not, it is +Z forward. The reason is that matrix is the Camera game object transform (without scale) rather than the camera’s view matrix. So you have the flip the Z direction to make it match view space.

5

It works, thanks!

8265369--1082355--upload_2022-7-9_17-54-18.png

Just for asking,did you suggest some good article or suggestion on how Unity (And 3D game engine), handle these spaces convertion?

6

I don’t have any specific recommendations, no. Though there are several that go over it in various ways.

I’ve been a fan of this image though (not my creation, and I’m not sure if the original source still exists).

640×1361 394 KB

And I’ve posted longer descriptions of the matrices, like I did here:

Thought that image isn’t Unity specific, that’s describing the common setup for OpenGL (which Unity mostly adheres to). And doesn’t talk about the difference between the “camera” and “view” matrices I mentioned above, nor other things like Unity’s use of a reversed Z depth. Nor the fact if you’re not rendering using OpenGL, the NDC z is from 0.0 to 1.0, not -1.0 to 1.0 (which is unique to OpenGL as it’s one of the first graphics APIs, and literally every graphics API after didn’t do that, because it’s dumb).

2 Likes