Raycasting with Lense Distortion

Questions & Answers
1

I’m having an issue with raycasting while using Unity’s Lens Distortion image effect.

Raycasts lose accuracy because they use the camera’s projection matrix without accounting for the distortion of the image effect. I tried to find the matrix or algorithm used for the distortion but I haven’t found anything.

Does any know what algorithm is used and how to modify the raycasting logic to account for the distortion?

Documentation about the effect is here:
https://docs.unity3d.com/Packages/com.unity.postprocessing@2.0/api/UnityEngine.Rendering.PostProcessing.LensDistortion.html

4

The answer by @anaswattar got me close to what I needed, but I did some more digging and found that the formula is changed a little between PostProcessing V2 and the version I was using (Universal Render Pipeline 12.1.6)

The location of the code changed too:

  • The C# code can be found at com.unity.render-pipelines.universal@12.1.6\Runtime\Passes\PostProcessPass.cs

  • The shader code can be found at com.unity.render-pipelines.universal@12.1.6\Shaders\PostProcessing\UberPost.shader

I also converted the code a little differently than the other answer:

`

    float intensity = 0.5f, scale = 1f;
    Vector2 lensCenter = Vector2.one * 0.5f;
    Vector2 lensXYMult = Vector2.one;
    Vector2 half = Vector2.one * 0.5f;

    float amount = 1.6f * Mathf.Max(Mathf.Abs(intensity * 100f), 1f);
    float theta = Mathf.Deg2Rad * Mathf.Min(160f, amount);
    float sigma = 2f * Mathf.Tan(theta * 0.5f);
    Vector2 center = lensCenter * 2f - Vector2.one;
    Vector4 p1 = new Vector4(
        center.x,
        center.y,
        Mathf.Max(lensXYMult.x, 1e-4f),
        Mathf.Max(lensXYMult.y, 1e-4f)
    );
    Vector4 p2 = new Vector4(
        intensity >= 0f ? theta : 1f / theta,
        sigma,
        1f / scale,
        intensity * 100f
    );

    Vector2 uv = (_uv - half) * p2.z + half;
    Vector2 distAxis = new Vector2(p1.z, p1.w);
    Vector2 distCenter = new Vector2(p1.x, p1.y);
    Vector2 ruv = distAxis * (uv - half - distCenter);
    float ru = ruv.magnitude;

    if (p2.w > 0.0)
    {
        float wu = ru * p2.x;
        ru = Mathf.Tan(wu) * (1f/(ru * p2.y));
        uv = uv + ruv * (ru - 1f);
    }
    else
    {
        ru = (1f/ru) * p2.x * Mathf.Atan(ru * p2.y);
        uv = uv + ruv * (ru - 1f);
    }
    
    return uv;

`

Ultimately this helped me do the kind of raycasting I needed to do. In my case I don’t need to worry about the distortion changing, so you could hook up those variables to read information from the post process settings. Hope that my digging saves someone else some time.

3

Hello,

I’m facing the same problem.

I have 2 questions:

1.- Is settings a LensDistortion? And how it work?
[SerializeField] private LensDistortion settings;

2.- What is the variable half?

Thanks in advance,

Cheers.

2

Ok after some research I was able to solve my problem.

The source of the lens distortion shader is here

PostProcessing/PostProcessing/Shaders/Builtins/Distortion.hlsl at v2 · Unity-Technologies/PostProcessing · GitHub

and its C# instructions are here

I ended up translating the shader code to C# so I could perform the same distortions on my mouse position. Here is my code:

public Ray RaycastWithDistortion(Vector2 screenPosition)
{
    if (settings.active)
    {
        return cam.ViewportPointToRay(DistortAndNormalizeScreenPosition(screenPosition));
    }
    return cam.ScreenPointToRay(screenPosition);
}

Vector2 half = new Vector2(.5f, .5f);

public Vector2 DistortAndNormalizeScreenPosition(Vector2 screenPosition)
{
    return DistortUV(cam.ScreenToViewportPoint(screenPosition));
}

Vector2 DistortUV(Vector2 uv)
{
    float amount = 1.6f * Mathf.Max(Mathf.Abs(settings.intensity.value), 1f);
    float theta = Mathf.Deg2Rad * Mathf.Min(160f, amount);
    float sigma = 2f * Mathf.Tan(theta * 0.5f);
    Vector4 distortionAmount = new Vector4(settings.intensity.value >= 0f ? theta : 1f / theta, sigma, 1f / settings.scale.value, settings.intensity.value);

    uv = ((uv - half) * distortionAmount.z) + half;

    Vector2 distortionCentre = new Vector2(settings.centerX.value, settings.centerY.value);
    Vector2 distortionScale = new Vector2(Mathf.Max(settings.intensityX.value, 1e-4f), Mathf.Max(settings.intensityY.value, 1e-4f));

    Vector2 ruv = distortionScale * (uv - half - distortionCentre);

    float ru = ruv.magnitude;

    if (distortionAmount.w > 0)
    {
        ru = Mathf.Tan(ru * distortionAmount.x) / (ru * distortionAmount.y);
    }
    else
    {
        ru = distortionAmount.x * Mathf.Atan(ru * distortionAmount.y) / ru;
    }
    uv = uv + (ruv * (ru - 1f));
    return uv;
}