Vision Pro Play to Device Host Pinch Gesture Inconsistent

When using Play to Host Device, the pinch gesture is really inconsistent-- It wasn’t working for days (though eye tracking worked immediately), and today it just randomly started working, but even while playing in the same “play session”, it won’t work for a minute, will work for ~5 seconds, and then will arbitrarily stop again. Does anyone know how to fix this?