I tink this is apple doing space wrap using depth buffer.
they use the depth data to recreate the space before the new frame is drawn to increase frame rate, so if when you having a low frame rate likely you will get the earlier rendered color projected onto the current calculated position (by space wrap with earlier depth buffer and current movement data), it looks just like the space got distorted, this likely happens after render queue of opaque, so the transparent queue wont be affected.
quest seems also doing similar but more like just recalculating rotation than full 6dof movement, so it would also not distort the background. I don’t know if there’s a way, no matter in Unity or Xcode, to set the space wrap mode from rotation and position into only rotation. if this could be done, there may be chance to solve the problem.