Oculus Quest grabbed gameobjects floating when walking???

Recently I have been able to make a vr game using unity, however when I grab a gameobject and then walk around the gameobject tends to float around. How do I fix that?

I have tried parenting to the OVRrig and the Trackingspace, but gameobjects snap to the players forehead. I also tried to make a new transform point and they did the samefloating around bit. So right now they are parenting to the actual grip transform of the custom hands.
You can see the problem in this video.

So I guess, this question is in the wrong spot on the forum. Where should I put this question?

It seems it’s updating the position (Update vs LateUpdate, etc) in wrong event or your parent object does that