Pre-rendered background with depth testing (isometric)

Unity Engine
1

I’m trying to found how to use the pre-rendered background (isometric camera) with 3D objects with the working depth testing so the 3D object are occluded by the background objects (like the pillars of the eternity). I’ve searched for some stuff how it is done. What I found, is that the the pillars of eternity is rendering some kind of depth map, but haven’t found any details what kind of the depth map are rendering and how are they using it in unity.

I found that they’re rendering the height of the each pixel in the depth map. But I don’t think the only height is sufficient. I think that each pixel of the depth map should have x, y, z position encoded. Then in unity in fragment shader reconstruct the position from this map and set the depth value for each fragment. I’ve tried this approach but somehow the depth value computed from the position (I’m multiplying the MVP with position reconstructed from depth map) is always same (doesn’t change when I move the background) and wrong (there is some small gradient).

The attached image is showing the the wrong depth (right image) and how it should like (left image).

Fragment shader:

fixed4 frag_mult(v2f_vct i, out float depth_o : SV_DEPTH) : SV_Target
{
    depth_o = 1;
    fixed4 col = tex2D(_MainTex, i.texcoord);
    clip(col.a - 0.25f);

    float4 col_depth = tex2D(_DepthTex, i.texcoord);
    float4 x_depth = tex2D(_XTex, i.texcoord);
    float4 y_depth = tex2D(_YTex, i.texcoord);
             
    float x = (DecodeFloatRGBA(x_depth) * 4 - 2);
    float y = (DecodeFloatRGBA(y_depth) * 4 - 2);
    float z = (DecodeFloatRGBA(col_depth) * 4 - 2);

    float4 vertex_p = float4(x, z, y, 0);
    float4 vertex_p_cam = vertex_p;
    vertex_p_cam = mul(UNITY_MATRIX_MVP, vertex_p);

    float depth = vertex_p_cam.z;
    depth_o = depth;
    col = fixed4(depth_o, depth_o, depth_o, col.a);

    return col;
}

So, does anyone has any suggestion?

Thank You

2168474--143460--1.png
2168474--143460--1.png705×495 25.7 KB

2

Ahhh, I’ve wrongly set the “W” component of vertex_p to “0”, which should be “1”.

...
float4 vertex_p = float4(x, z, y, 1);
...
3

I’m curious for another solution.

I have this sprite (which is mesh in XY plane) rotated 90 around X (to be in the XZ plane) and Y scaled by 2. I still want to reconstruct the depth from the provided texture. But it looks like the rotation is accounted when computing the depth value and the final value is therefore wrong.

So what I need is to reconstruct the position without taking the rotation in the account. Maybe I’m doing it all wrong so someone please correct me if my thougts aren’t right. So I was thinking if I can somehow zero out the rotation when using MVP matrix (the position should still be computed in the world space but without the rotation).

Thank You