It only turns you to the left in world space, because the ‘to’ vector is left in world space.
Try doing:
float dt = 1.0;//make this smaller so you can do it over a period of time in an update funct or something
transform.forward = Vector3.Slerp(transform.forward, transform.left, dt);
though setting that on the transform probably isn’t the best result, especially if you want to rotate around different axis.
Use quaternions for even nicer system that can rotate how ever you like.