I’m not exactly sure what the issue with the shadows is - I initially thought it had something to do with shadow acne, and I increased both the bias and the normal bias for the directional light. To further the quality of the shadows is high resolution.
In case anyone ever faces this issue in the future, I was able to resolve the problem by employing the suggestion of both @Mouton and @Octoman.
Specifically if you go to Project Settings > Quality and in shadow setting select Shadow Projection and change that to close fit rather than a stable fit.
And then scale the object to a slightly larger size and everything should be good.