Unity XR Interaction Manager spawns in empty?

So I have been working on a very simple VR game where the only thing you do is throw darts at a board (for now atleast), but I have some problems with the pickup of the darts, everytime I pick up a dart from a table it is lagging and teleporting alot so I cant really throw it. After some debugging I came to notice that:

  1. for some reason my XR Interaction Manager (XRIM) is empty and I cant attach the right script to it.
  2. Because the XRIM is empty I can attach it to objects but it wont do anything as it does not have the script.
  3. Everytime I start the game unity decides to spawn in a own XRIM but that is empty aswell.

The weird thing is that I can still pickup objects like you should but as I said they are teleporting around a lot and do not stay in my “hands”.

Empty XRIM unity spawns in

For some reason it fixes itself after a while in the game but for the first 1 minute I cant really hold anything in my “hands” as it is teleporting around them constantly.

I’ve tried to copy the XRIM and make my own script with it but it wont allow me as it has something to do with “protection levels”.

I’ve also checked and redownloaded the toolkit from package manager several times and none have helped.

(I am using a Oculus Quest 3 headset if that matters)

Thx for any help