Rotate an object with help of hand

I would like to rotate an object with my hand only in the Y axis. I am able to achieve this as well in my implementation.
The problem comes in saving the last rotation, when the hand tries to rotate the object again, the object’s start rotation snaps to a new rotation completely.
How do I solve this problem?

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class RotateObj : MonoBehaviour {

   private Quaternion currentRot;
   private Vector3 startPos;
   public Transform hand;
   private bool offsetSet;

   void Start () {

   }

   void Update () {
       Rotate ();
   }

   void SetOffsets () {
       if (offsetSet)
           return;
           
       startPos = Vector3.Normalize (hand.transform.position - this.transform.position);
       currentRot = this.transform.rotation;
       
       offsetSet = true;
   }

   void Rotate () {
       SetOffsets ();

       Vector3 closestPoint = Vector3.Normalize (hand.transform.position - this.transform.position);
       var rot = Quaternion.FromToRotation (startPos, closestPoint);
       rot = Quaternion.Euler (0, rot.eulerAngles.y * 10f, 0);
       this.transform.rotation = rot * currentRot;

   }

}

Can you explain the desired UI input properly? What does the “hand” do and what kind of device?