Rotate horizontally a non grabbable object

I’m new to Unity, I’m trying to rotate objects with a fixed position, they won’t move for any reason. The idea is to point the object with the XR Ray, and the press the trigger button. While the button is till pressed and you move the controller left or right, the object should rotate to that side. It should be only a horizontal rotation, example
I’m using XR Toolkit and the app will be used with a Quest 2.

What does the user see when interacting? Is this like a fishing pole attached to the target object, bending as the user moves their hand left/right (and at the same time the target object rotates)? I think this would be achievable but will need extra code.

Alternatively, if you are able to use just the joystick on the controller, you might be able to do this by having a grabbable object locked in place and use the target rotation feature of the XR toolkit. If something has a RigidBody, you can freeze its position using a setting called Constraints. (Edit: I just tried this and the RigidBody constraints are lifted when you select the item, so this trick won’t work, sorry).

I strongly recommend that you install the XR Toolkit demo, and try the different interactions. You might find something which works for you and you can just copy the prefab or even just take that project and modify it: