2D Weapon Sway

So I am trying to make a game where there is an axe that follows the mouse. The mouse holds the handle of the axe, so the head of the axe should have a sway. I now have a bug where if the axe goes for a long time in one direction, it snaps back to 20 degrees ( my max angle ). I can’t figure out how to make it smooth. Thanks !

private void Move()
{
    // weapon movement
    Vector3 MousePosition = Camera.main.ScreenToWorldPoint(touchPositionAction.ReadValue<Vector2>());
    if (MousePosition.x < -0.2f && MousePosition.y < -1.5f)
    {
        MousePosition.x = -0.2f;
    }
    Vector3 AxePosition = axe.transform.position;
    Vector2 Difference = new Vector2(MousePosition.x - AxePosition.x, MousePosition.y - AxePosition.y);

    float speed = Mathf.Pow((Difference.magnitude)*8, 1.3f);
    Vector2 Direction = Difference.normalized;
    Vector2 newVelocity = Direction * speed;
    axeRigidbody.linearVelocity = newVelocity;

    // weapon sway
    float z = axe.transform.eulerAngles.z;
    if (z > 180f) z -= 360f;

    float targetAngular = 0f;
    if (speed > 1f && Mathf.Abs(z) < 20f)
    {
        targetAngular = -axeRigidbody.linearVelocity.x * swayMultiplier;

    }
    else
    {
        targetAngular = -z * restoreMultiplier;
    }

    float newAngular = Mathf.Lerp(axeRigidbody.angularVelocity, targetAngular, Time.deltaTime * swaySmoothing);
    newAngular = Mathf.Clamp(newAngular, -maxAngularVelocity, maxAngularVelocity);
    axeRigidbody.angularVelocity = newAngular;
}

sounds like some debugging needed, is it because the mouse goes off the edge of the window?

No It’s because when the axe’s rotation is bigger than 20 degrees it goes back but I can’t figure out how to smooth it out