Hello!

This is my first post and I am very new to unity and computer graphics in general.

So please do not be too harsh. :smile:

I am trying to reconstruct the world position from a depth image.

My shader:

```
// Upgrade NOTE: replaced 'mul(UNITY_MATRIX_MVP,*)' with 'UnityObjectToClipPos(*)'
Shader "Hidden/Depth"{
Properties
{
_MainTex ("Base (RGB)", 2D) = "white" {}
_DepthLevel ("Depth Level", Range(1, 3)) = 1
}
SubShader
{
Pass
{
CGPROGRAM
#pragma vertex vert
#pragma fragment frag
#include "UnityCG.cginc"
uniform sampler2D _MainTex;
uniform sampler2D _CameraDepthTexture;
uniform fixed _DepthLevel;
uniform half4 _MainTex_TexelSize;
struct input
{
float4 pos : POSITION;
half2 uv : TEXCOORD0;
};
struct output
{
float4 pos : SV_POSITION;
half2 uv : TEXCOORD0;
};
output vert(input i)
{
output o;
o.pos = UnityObjectToClipPos(i.pos);
o.uv = MultiplyUV(UNITY_MATRIX_TEXTURE0, i.uv);
// Flip the image
#if UNITY_UV_STARTS_AT_TOP
if (_MainTex_TexelSize.y < 0)
o.uv.y = 1 - o.uv.y;
#endif
return o;
}
fixed4 frag(output o) : COLOR
{
float depth = UNITY_SAMPLE_DEPTH(tex2D(_CameraDepthTexture, o.uv));
depth = pow(Linear01Depth(depth), _DepthLevel);
return depth;
}
ENDCG
}
}
}
```

I am not trying to shoot rays through the pixels, because I want to learn more

about camera parameters and matrices (or at least, I do not think I want to).

What I actually want to do is use the camera parameters and the depth I get from

the image. An idea, to easily get the parameters I need, would be to try to use

the view and inverse projection matrix. I do not even know, if I need the view matrix.

Can I calculate an imaginary line with these parameters (start point, end point from the depth

value and angle from one of the matrices) to inverse calculate the world position from

screen/image space (my depth image)?

That would probably be my ultimate goal.

Is it bad, that my camera has a world position with x = 5 and not 0?

I just realized that while writing this.

What i also tried, was multiplying the Vector from the pixel coordinate

from the image/view space and the z from my depth values with the inverse

projection matrix.

I tried to reverse the perspective divide process, which, from what I am understanding,

takes part after the projection from world to image/world space. Then I multiplied that

vector with the inverse projection matrix of the camera and added my depth value.

I know, that we use the w component of the Vec4 to keep the original z value

of the vertex for further operations with textures and stuff.

The "zValue" I am using here tho is the ??linear depth?? from my image

(0 to 1, from the RGB value(s) of the pixel(s)).

So i multiplied that with my far clip plane as shown below to get back

the actual distance in world space (w).

Do i even need to do this, or can I also just use the

projection matrix somehow for this?

**However, the x and y coordinates are way off after that. Only the z value is legit**

**(since I am reading it from my depth image).**

```
Vector3 ScreenToWorld(Vector3 screenPos, float zValue)
{
Matrix4x4 matInv = _camera.projectionMatrix.inverse;
screenPos.x *= zValue * _camera.farClipPlane; // Trying to reverse the perspective divide here
screenPos.y *= zValue * _camera.farClipPlane;
Vector4 WorldPos = matInv * screenPos;
WorldPos.z = zValue * _camera.farClipPlane;
return WorldPos;
}
```

This is all done with a camera in unity and a depth image generated from scene with a shader,

because I can not get my hand on a let's say Kinect for example, nor do I want to yet.

But this would be the project after this one to then calculate from actual camera parameters.

Could I also use the "real" camera parameters from the camera in unity? From what I know,

it would be more handy to use matrices for now.

**Just to make it clear, I do not want to use anything in the scene itself.**

**Only code to do the image processing to get the world position.**

**I do not want to use camera.ScreenToWorldPoint to shoot a ray through**

**the pixel and get the point at the depth value to get the world position.**

**I already did this and it works.**

**Maybe I want to write something similar to it tho with the help of the**

**camera parameters/matrices.**

Help with both solutions would be very appreciated!

It would also be nice, if someone could also help me with understanding the theory behind

all of this a little better (maybe even for both solutions).

Thank you so much! <3