Shadow Issues with custom Camera Matrix

(CASE IN-9123) (Using URP 12)
I’m having issues with realtime shadows when using my custom camera worldToCameraMatrix.

My goal is to have a camera projection that will make the game look top down 2D but allow for shadows, depth etc. - by using planes standing straight up but rendering facing camera. I got this from this article: Adding some perspective to your Unity 2D game.

I have the camera projection setup as I want it, but there is a problem with shadows not rendering at all when camera goes past a certain distance from the world center. In certain spots one can clearly see the shadow sphere as well.

x2fo2c

This is how I change camera projection:

  public class CustomProjection : MonoBehaviour
    {
        [SerializeField] private bool _resetCameraMatrixOnEnd;
        [SerializeField] private Camera _camera;
        [SerializeField] private Vector4 _matrixYOffset = new Vector4(0, -1, 1, -0.1f);
        [SerializeField] private Vector4 _matrixZOffset = new Vector4(0, 0, 0, 0);

        private void Start()
        {
            _camera = GetComponent<Camera>();
        }

        private void OnEnable()
        {
            RenderPipelineManager.beginCameraRendering += OnBeginCameraRendering;
            RenderPipelineManager.endCameraRendering += OnEndCameraRendering;
        }

        private void OnDisable()
        {
            RenderPipelineManager.beginCameraRendering -= OnBeginCameraRendering;
            RenderPipelineManager.endCameraRendering -= OnEndCameraRendering;
            _camera.ResetWorldToCameraMatrix();
        }

        private void OnBeginCameraRendering(ScriptableRenderContext scriptableRenderContext, Camera camera)
        {
            SetMatrix(camera);
        }

        private void OnEndCameraRendering(ScriptableRenderContext scriptableRenderContext, Camera camera)
        {
            ResetMatrix(camera);
        }

        private void SetMatrix(Camera camera)
        {
            if (camera != _camera && camera.cameraType != CameraType.Game)
                return;
       
            var matrix = camera.transform.worldToLocalMatrix;
            matrix.SetRow(2, -matrix.GetRow(2));
            matrix.SetColumn(1, 1e-3f * matrix.GetColumn(1) - _matrixYOffset);
            matrix.SetColumn(2, matrix.GetColumn(2) - _matrixZOffset);
            camera.worldToCameraMatrix = matrix;
        }

        private void ResetMatrix(Camera camera)
        {
            if (camera != _camera && camera.cameraType != CameraType.Game && _resetCameraMatrixOnEnd)
                camera.ResetWorldToCameraMatrix();
        }
    }

Camera settings:

I’ve also attached the small test project and moving the camera around should show the shadows popping in and out clearly.

Not sure if this is a bug or if Im misunderstanding camera projection and it should be done in another way.
Any help would be appreciated, thanks!

8301927–1088826–CamerProjectionShadowsIssues.unitypackage (99.9 KB)

Bump
Also, it seems that URP and the standard RP have different results/behavior. So if any moderator sees this, feel free to move it to the URP section.