First, including all your script would be very helpful in answering this question. I see four (potential) issues here.
The code needs to be executed repeatedly in Update() or in a Coroutime with a ‘yield’ in order to make use of Lerp() for a gradual rotation. I’m guessing based on your symptom, you are only executing it once.
If you execute it over many frames, then you have the problem that you are setting your end rotation again each frame. That is, it is recalculating a new end rotation each time Lerp() is executed.
transform.rotation.* are not angles and cannot be treated as angles
For any given physical rotation, there are multiple euler angle representations, and Unity will change representation on you.
Take a look at this link. In particular, look at the EulerAngle section towards the end.
You might also examine the Rotation() function in this script: