I encountered a similar bug on the new iPad with LiDAR.
It seems to me, the issue is related to devices that can track 6 degrees of freedom on the front camera. And after you switch to a back camera, the camera position and rotation will not be updated. Here is a memo I left for my future self in the comments 
"Bug in ARKit: camera position and rotation will not receive updates with world camera and session state will always be <= SessionInitializing if all these conditions are met:
- your AR device supports face tracking with 6 degrees of freedom
- ARFaceManager is enabled
- requestedTrackingMode is set to TrackingMode.PositionAndRotation
- AR Foundation >= 4.0"
The solution is to set tracking mode to TrackingMode.RotationOnly when using face tracking.