Simultaneous face tracking and rear camera AR

[edwon]

I followed the example in the github examples project, but when I implement in the same way in my project, it switches to the face camera as soon as I start recording using NatCorder.

The only difference in the scene is the presence of an ARFaceManager. If I disable that, it doesnt switch the face camera when recording.

Any clues? I wish I could control which camera it shows in the ARCameraBackground better.

[enhawk]

AFAIK you can’t activate both cameras at the same time…

[edwon]

Yes you can. With ARKit 3.0 and iOS13 with an iPhoneXR or XS or equivalent iPad, you can get facial tracking data from the front camera at the same time as using AR with the rear camera. It’s a new thing they just announced and Unity supports it. It’s mentioned in the ARFoundation docs somewhere.

[enhawk]

wow! I think theres a thread about it in the ARFoundation forum, maybe more luck there? This forum is depreciated

Does AR Foundation support ARKit 3 Simultaneous Front and Back Camera?

Hi I wonder does the latest AR Foundation support ARKit 3 Simultaneous Front and Back Camera feature? I checked RearCameraWithFrontCameraFaceMesh scene from AR Foundation samples, but it doesn’t demonstrate how to use front camera to enable face tracking and at the same time using device full orientation and position information. Does Unity plan to create a demo like that? Also I am curious whether user can get human depth and stencil texture from front camera, just like the human occlusion d…

[ina]

@edwon

Yes you can. With ARKit 3.0 and iOS13 with an iPhoneXR or XS or equivalent iPad, you can get facial tracking data from the front camera at the same time as using AR with the rear camera. It’s a new thing they just announced and Unity supports it. It’s mentioned in the ARFoundation docs somewhere.

Yup
https://github.com/Unity-Technologies/arfoundation-samples/tree/5.0
Rear Camera (ARKit)
iOS 13 adds support for face tracking while the world-facing (i.e., rear) camera is active. This means the user-facing (i.e., front) camera is used for face tracking, but the pass through video uses the world-facing camera. To enable this mode in ARFoundation, you must enable an ARFaceManager, set the ARSession tracking mode to “Position and Rotation” or “Don’t Care”, and set the ARCameraManager’s facing direction to “World”. Tap the screen to toggle between the user-facing and world-facing cameras.