Ray picking objects with a 'Curved World' Shader

Hi

I am creating a game with a curved world and use the shader from Animal Crossing Curved World Shader | Alastair Aitchison which works really fine. My problem is that Raycasts do not hit objects as their real position is a bit higher than their visual appearance (as they are warped down depending on distance).

How can I modify the mouse picking to account for this effect? Maybe a custom Projection Matrix for the Camera, but math like this is beyond my knowledge.

Any hints are appreciated.

After some thinking I came up with this solution by adjusting the original RaycastHit by the inverted curvature factor. Two downsides that are yet to be worked out are that it is not perfectly accurate as I had to tweak the curvature factor from 0.023f in shader to 0.03f in c# code. And clicking the “sky” is still picking terrain that is far off in the distance. But other than that it is working pretty good.

private RaycastHit? adjustHit(RaycastHit hit)
{
    var pt = hit.point;
    var dist = hit.distance;

    Vector4 vv = new Vector4(pt.x, pt.y, pt.z, 1.0f);

    vv.x -= Camera.main.transform.position.x;
    vv.y -= Camera.main.transform.position.y;
    vv.z -= Camera.main.transform.position.z;

    // Increase y by curvature factor (originally -m_Curvature)
    vv = new Vector4(0.0f, ((vv.z * vv.z) + (vv.x * vv.x)) * m_Curvature, 0.0f, 0.0f);

    var finalVec = new Vector3(pt.x, pt.y + vv.y, pt.z);

    // Send raycast again by finding screen pos of world position hit
    RaycastHit hit2;
    var screenPos = Camera.main.WorldToScreenPoint(finalVec);
    var ray = Camera.main.ScreenPointToRay(screenPos);
    if (Physics.Raycast(ray, out hit2, 100f))
    {
        return hit2;
    }

    return null;
}