Confused on NDC to World Space coords in calculating the direction of a ray in ray tracing

[Midnaaaaaa]

This shader in unity tries to ray trace a sphere, but doesnt actually work because when the camera approaches the sphere, it doesnt appear and when the camera shouldnt be looking at it, it shows up.

 fixed4 frag (v2f i) : SV_Target
 {
     Ray ray;
     ray.origin = _WorldSpaceCameraPos;

     float3 ndc = float3(i.uv * 2.0f - 1.0f, 1); // Convert texture coordinates to NDC
     float4 clip = float4(ndc * i.vertex.w, i.vertex.w);
     float4 view = mul(unity_CameraInvProjection, clip);
     float4 world = mul(unity_CameraToWorld, view);

     ray.direction = normalize(world.xyz - ray.origin);
     
     HitInfo h = RaySphere(ray, float3(0,0, 2), 0.5); //Ray, sphere center, sphere radius

     if(h.didHit){
         return float4(1,0,0,1);
     }
     else return float4(0,0,0,1);
 }

Here if the sphere is at 0,0,2 with radius 0.5 in the positive z axis and the camera is looking directly at the z+ axis, if the camera is somewhere greater than 2.5 the sphere shouldnt be visible but as it can be seen in the image, the sphere is showing.

I don’t know what I am doing wrong, I know that I could be using the ndc coordinates as local coordinates of the camera and just transform this local coords to world space, but I wanted to see if doing this approach it would do the same results.

Any idea of what am I doing wrong?

[c0d3_m0nk3y]

I don’t know if this is what you meant with using the ndc coordinates as local coordinates of the camera but I’d write it like that:

float4 ndc = float3(i.uv * 2.0f - 1.0f, 1, 1);
float4 view = mul(unity_CameraInvProjection, ndc);
float4 world = mul(unity_CameraToWorld, view);
world /= world.w;

I’ve never seen anybody use i.vertex.w like that. It is probably not wrong because you can multiply homogenous coordinates with any non-zero value but you still have to divide by w in the end.

[Midnaaaaaa]

What i said is that something like this could be done:

float3 ndc = float3(i.uv * 2.0f - 1.0f, 1); //This ensures that the uv coords are centered, which actually means that it aligns with the local space of the camera because the center of the screen is in fact the 0,0,0.

And then if you consider this point as relative to the object space of the camera, then you can just multiply by the local to world matrix like this:

float3 pixelTarget = mul(LocalToWorld, float4(ndc,1)); //LocalToWorld is the model transform matrix of the camera.

And then it would work.

But the idea was to do basically the backwards process not to use the NDC coords as coords relative to the local space of the camera. I don’t know if I am explaining myself.

Also, the ndc shouldn’t actually be multiplied by the CameraInvProjection you should first pass it to clip space coords, that’s why i multiplied by the homogenous coord beforehand.

[c0d3_m0nk3y]

@Midnaaaaaa

Also, the ndc shouldn’t actually be multiplied by the CameraInvProjection you should first pass it to clip space coords, that’s why i multiplied by the homogenous coord beforehand.

You don’t need to do that when you are going in inverse direction. Clip space coordinates are just non-normalized NDC space coordinates. You could multiply the NDC coordinates with any non-zero value to get clip space coordinates, but there is no point in doing that. NDC coordinates are also valid clip space coordinates, but not the other way around.

@Midnaaaaaa

float3 pixelTarget = mul(LocalToWorld, float4(ndc,1)); //LocalToWorld is the model transform matrix of the camera.

This is not correct. I think your description of spaces is off.
Object Space: Coordinates relative to position/orientation of an object
World Space: Absolute Coordinates
View Space: Coordinates relative to the position/orientation of the camera. The coordinates haven’t been projected yet, so the visible coordinates are within a view frustum.
Clip Space: Same as NDC space before the homogeneous division. So the visible coordinates are between -w to +w for x/y/z.
NDC space: Coordinates after the homogeneous division. Visible coordinates are between -1 to +1 for x/y/z. Note that the camera position is not inside this box. (0, 0, -1) is the center of the screen on the near plane.

Object Space → Model Matrix → World Space → View Matrix → View Space → Projection Matrix → Clip Space → Homogeneous Division → NDC Space

You can’t go from NDC space to world space without performing the inverse view projection.

*) There are some small differences between OpenGL and DirectX and inverse z-buffers that I am ignoring

[Midnaaaaaa]

Yeah I understand the spaces, but the idea I am trying to explain maybe I explained it a little bit bad.

In the image we see the Object Space of the camera, if we consider the NDC coords that go [-1, 1] in all axis, we could consider that the NDC coords are points that are relative to this object space and make them go from object space (from the camera) to world space.

float3 ndc = float3(i.uv * 2.0f - 1.0f, 1); //This ensures that the uv coords are centered, which actually means that it aligns with the local space of the camera because the center of the screen is in fact the 0,0,0.
float3 pixelTarget = mul(LocalToWorld, float4(ndc,1)); //LocalToWorld is the model transform matrix of the camera.

In fact this code is from a youtube video from Sebastian Lague, but I wanted to know why he did this transformation, and that’s my reasoning.

But although this works, I wanted to think on how to solve this without making this coordenates be relative to the object space of the camera.

Maybe I am explaining bad, sorry

[Midnaaaaaa]

I tried this, but for some reason (which I dont understand and i would like to), when i divide by the homogenous coord, the depth is flipped, but if i divide by -world.w then it looks right, but when I rotate the camera, the rotations are inverted, why is that?

[c0d3_m0nk3y]

You probably need to negate NDC y.

@c0d3_m0nk3y

float4 ndc = float3(i.uv * 2.0f - 1.0f, 1, 1);
ndc.y = -ndc.y;
float4 view = mul(unity_CameraInvProjection, ndc);
float4 world = mul(unity_CameraToWorld, view);
world /= world.w;

[Midnaaaaaa]

But now things are visible when they are in the z- axis if camera is at 0,0,0 looking to z+ axis.

[Midnaaaaaa]

Ok i figured out what was going on, explained in the following post:

[URGENT] a STRANGE problem about using unity_CameraInvProjection

Recently I searched many articles about how to do full-screen Raymarching. There is a point that transform points(actually a quad with 4 points generated by Blit) from ndc space to view space and compute the direction of frustum corner ray. Then we can get the interpolated raymarching direction from camera to each pixel. The theory is easy but… the specific methods really cause a headache. Here is the method1: v2f vert(appdata v) { v2f o; Ve…