As you can tell, if you’re looking in the direction of movement, the bullet spawns inside your gun, while if you’re looking in the opposite direction, it’s spawning in front of the gun. What’s probably happening is that you’re instantiating the bullet, then the player gets moved, so your bullet’s spawn position is one frame late.
I think the best way to fix that is to make sure that the bullet’s spawned after the player’s been moved. How to do that best depends on how you’re moving, but you could try putting the spawning in LateUpdate, as then you’re sure that both your update code and physics interpolation has happened.