Debugging help for Face Tracking + MARS

Hi - I don’t know if this is a MARS specific issue but face tracking is not working at all for me. I’ve tried simple hello world type tests, both on a new ipad and iphone 12 and its not working. In simulation mode it is working on the example video feed but once i build it to my ios device – nothing. Body tracking works fine, only face tracking is not working. All the newest packages are installed via package manager.

The only thing I can think of is to revert to older ARKit / Unity foundation package version, but there isn’t even a way in Package Manager to revert to older versions anymore

Unity version: 2021.2.18 (also on 2021.2.16)
ARKit / AR Foundation version: 4.2.2
IOS version: 15.4.1

Here is the debug logs from xcode when I run my app::

2022-04-05 11:38:27.931558-0500 mirror[2254:341142] Built from '2021.2/staging' branch, Version '2021.2.18f1 (0c6e675195cf)', Build type 'Development', Scripting Backend 'il2cpp'

2022-04-05 11:38:27.932633-0500 mirror[2254:341142] MemoryManager: Using 'Default' Allocator.

[UnityMemory] Configuration Parameters - Can be set up in boot.config

    "memorysetup-bucket-allocator-granularity=16"

    "memorysetup-bucket-allocator-bucket-count=8"

    "memorysetup-bucket-allocator-block-size=4194304"

    "memorysetup-bucket-allocator-block-count=1"

    "memorysetup-main-allocator-block-size=16777216"

    "memorysetup-thread-allocator-block-size=16777216"

    "memorysetup-gfx-main-allocator-block-size=16777216"

    "memorysetup-gfx-thread-allocator-block-size=16777216"

    "memorysetup-cache-allocator-block-size=4194304"

    "memorysetup-typetree-allocator-block-size=2097152"

    "memorysetup-profiler-bucket-allocator-granularity=16"

    "memorysetup-profiler-bucket-allocator-bucket-count=8"

    "memorysetup-profiler-bucket-allocator-block-size=4194304"

    "memorysetup-profiler-bucket-allocator-block-count=1"

    "memorysetup-profiler-allocator-block-size=16777216"

    "memorysetup-profiler-editor-allocator-block-size=1048576"

    "memorysetup-temp-allocator-size-main=4194304"

    "memorysetup-job-temp-allocator-block-size=2097152"

    "memorysetup-job-temp-allocator-block-size-background=1048576"

    "memorysetup-job-temp-allocator-reduction-small-platforms=262144"

    "memorysetup-temp-allocator-size-background-worker=32768"

    "memorysetup-temp-allocator-size-job-worker=262144"

    "memorysetup-temp-allocator-size-preload-manager=262144"

    "memorysetup-temp-allocator-size-nav-mesh-worker=65536"

    "memorysetup-temp-allocator-size-audio-worker=65536"

    "memorysetup-temp-allocator-size-cloud-worker=32768"

    "memorysetup-temp-allocator-size-gfx=262144"

-> applicationDidFinishLaunching()

Found 3 interfaces on host : 0) 10.93.2.11 1) 162.165.5.241 2) 192.168.100.13

Multi-casting "[IP] 10.93.2.11 [Port] 55000 [Flags] 2 [Guid] 171254066 [EditorId] 485543332 [Version] 1048832 [Id] iPhonePlayer(8,iPhone):56000 [Debug] 0 [PackageName] iPhonePlayer [ProjectName] mirror" to [225.0.0.222:54997]...

2022-04-05 11:38:28.116547-0500 mirror[2254:341142] Metal GPU Frame Capture Enabled

2022-04-05 11:38:28.116659-0500 mirror[2254:341142] Metal API Validation Enabled


Setting UIViewControllerBasedStatusBarAppearance to NO is no longer supported.

Apple actively discourages that, and all application-wide methods of changing status bar appearance are deprecated


-> applicationDidBecomeActive()

[Subsystems] Discovering subsystems at path /private/var/containers/Bundle/Application/DA5AF9E7-D186-43CC-AF4E-1B1634649441/mirror.app/Data/UnitySubsystems

[Subsystems] No descriptors matched for  examples in UnitySubsystems/UnityARKit/UnitySubsystemsManifest.json.

[Subsystems] 1 'inputs' descriptors matched in UnitySubsystems/UnityARKit/UnitySubsystemsManifest.json

[Subsystems] No descriptors matched for  displays in UnitySubsystems/UnityARKit/UnitySubsystemsManifest.json.

[Subsystems] 1 'meshings' descriptors matched in UnitySubsystems/UnityARKit/UnitySubsystemsManifest.json

GfxDevice: creating device client; threaded=1; jobified=1

Initializing Metal device caps: Apple A14 GPU

Initialize engine version: 2021.2.18f1 (0c6e675195cf)

XRGeneral Settings awakening...

UnityEngine.DebugLogHandler:Internal_Log(LogType, LogOption, String, Object)

UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[])

UnityEngine.Logger:Log(LogType, Object)

UnityEngine.Debug:Log(Object)

UnityEngine.XR.Management.XRGeneralSettings:Awake()


2022-04-05 11:38:28.936450-0500 mirror[2254:341142]

    // 2D joint skeleton

    enum JointIndices

    {

        Invalid = -1,

        Head = 0, // parent: Neck1 [1]

        Neck1 = 1, // parent: Root [16]

        RightShoulder1 = 2, // parent: Neck1 [1]

        RightForearm = 3, // parent: RightShoulder1 [2]

        RightHand = 4, // parent: RightForearm [3]

        LeftShoulder1 = 5, // parent: Neck1 [1]

        LeftForearm = 6, // parent: LeftShoulder1 [5]

        LeftHand = 7, // parent: LeftForearm [6]

        RightUpLeg = 8, // parent: Root [16]

        RightLeg = 9, // parent: RightUpLeg [8]

        RightFoot = 10, // parent: RightLeg [9]

        LeftUpLeg = 11, // parent: Root [16]

        LeftLeg = 12, // parent: LeftUpLeg [11]

        LeftFoot = 13, // parent: LeftLeg [12]

        RightEye = 14, // parent: Head [0]

        LeftEye = 15, // parent: Head [0]

        Root = 16, // parent: <none> [-1]

    }

2022-04-05 11:38:28.940726-0500 mirror[2254:341142]

    // 3D joint skeleton

    enum JointIndices

    {

        Invalid = -1,

        Root = 0, // parent: <none> [-1]

        Hips = 1, // parent: Root [0]

        LeftUpLeg = 2, // parent: Hips [1]

        LeftLeg = 3, // parent: LeftUpLeg [2]

        LeftFoot = 4, // parent: LeftLeg [3]

        LeftToes = 5, // parent: LeftFoot [4]

        LeftToesEnd = 6, // parent: LeftToes [5]

        RightUpLeg = 7, // parent: Hips [1]

        RightLeg = 8, // parent: RightUpLeg [7]

        RightFoot = 9, // parent: RightLeg [8]

        RightToes = 10, // parent: RightFoot [9]

        RightToesEnd = 11, // parent: RightToes [10]

        Spine1 = 12, // parent: Hips [1]

        Spine2 = 13, // parent: Spine1 [12]

        Spine3 = 14, // parent: Spine2 [13]

        Spine4 = 15, // parent: Spine3 [14]

        Spine5 = 16, // parent: Spine4 [15]

        Spine6 = 17, // parent: Spine5 [16]

        Spine7 = 18, // parent: Spine6 [17]

        LeftShoulder1 = 19, // parent: Spine7 [18]

        LeftArm = 20, // parent: LeftShoulder1 [19]

        LeftForearm = 21, // parent: LeftArm [20]

        LeftHand = 22, // parent: LeftForearm [21]

        LeftHandIndexStart = 23, // parent: LeftHand [22]

        LeftHandIndex1 = 24, // parent: LeftHandIndexStart [23]

        LeftHandIndex2 = 25, // parent: LeftHandIndex1 [24]

        LeftHandIndex3 = 26, // parent: LeftHandIndex2 [25]

        LeftHandIndexEnd = 27, // parent: LeftHandIndex3 [26]

        LeftHandMidStart = 28, // parent: LeftHand [22]

        LeftHandMid1 = 29, // parent: LeftHandMidStart [28]

        LeftHandMid2 = 30, // parent: LeftHandMid1 [29]

        LeftHandMid3 = 31, // parent: LeftHandMid2 [30]

        LeftHandMidEnd = 32, // parent: LeftHandMid3 [31]

        LeftHandPinkyStart = 33, // parent: LeftHand [22]

        LeftHandPinky1 = 34, // parent: LeftHandPinkyStart [33]

        LeftHandPinky2 = 35, // parent: LeftHandPinky1 [34]

        LeftHandPinky3 = 36, // parent: LeftHandPinky2 [35]

        LeftHandPinkyEnd = 37, // parent: LeftHandPinky3 [36]

        LeftHandRingStart = 38, // parent: LeftHand [22]

        LeftHandRing1 = 39, // parent: LeftHandRingStart [38]

        LeftHandRing2 = 40, // parent: LeftHandRing1 [39]

        LeftHandRing3 = 41, // parent: LeftHandRing2 [40]

        LeftHandRingEnd = 42, // parent: LeftHandRing3 [41]

        LeftHandThumbStart = 43, // parent: LeftHand [22]

        LeftHandThumb1 = 44, // parent: LeftHandThumbStart [43]

        LeftHandThumb2 = 45, // parent: LeftHandThumb1 [44]

        LeftHandThumbEnd = 46, // parent: LeftHandThumb2 [45]

        Neck1 = 47, // parent: Spine7 [18]

        Neck2 = 48, // parent: Neck1 [47]

        Neck3 = 49, // parent: Neck2 [48]

        Neck4 = 50, // parent: Neck3 [49]

        Head = 51, // parent: Neck4 [50]

        Jaw = 52, // parent: Head [51]

        Chin = 53, // parent: Jaw [52]

        LeftEye = 54, // parent: Head [51]

        LeftEyeLowerLid = 55, // parent: LeftEye [54]

        LeftEyeUpperLid = 56, // parent: LeftEye [54]

        LeftEyeball = 57, // parent: LeftEye [54]

        Nose = 58, // parent: Head [51]

        RightEye = 59, // parent: Head [51]

        RightEyeLowerLid = 60, // parent: RightEye [59]

        RightEyeUpperLid = 61, // parent: RightEye [59]

        RightEyeball = 62, // parent: RightEye [59]

        RightShoulder1 = 63, // parent: Spine7 [18]

        RightArm = 64, // parent: RightShoulder1 [63]

        RightForearm = 65, // parent: RightArm [64]

        RightHand = 66, // parent: RightForearm [65]

        RightHandIndexStart = 67, // parent: RightHand [66]

        RightHandIndex1 = 68, // parent: RightHandIndexStart [67]

        RightHandIndex2 = 69, // parent: RightHandIndex1 [68]

        RightHandIndex3 = 70, // parent: RightHandIndex2 [69]

        RightHandIndexEnd = 71, // parent: RightHandIndex3 [70]

        RightHandMidStart = 72, // parent: RightHand [66]

        RightHandMid1 = 73, // parent: RightHandMidStart [72]

        RightHandMid2 = 74, // parent: RightHandMid1 [73]

        RightHandMid3 = 75, // parent: RightHandMid2 [74]

        RightHandMidEnd = 76, // parent: RightHandMid3 [75]

        RightHandPinkyStart = 77, // parent: RightHand [66]

        RightHandPinky1 = 78, // parent: RightHandPinkyStart [77]

        RightHandPinky2 = 79, // parent: RightHandPinky1 [78]

        RightHandPinky3 = 80, // parent: RightHandPinky2 [79]

        RightHandPinkyEnd = 81, // parent: RightHandPinky3 [80]

        RightHandRingStart = 82, // parent: RightHand [66]

        RightHandRing1 = 83, // parent: RightHandRingStart [82]

        RightHandRing2 = 84, // parent: RightHandRing1 [83]

        RightHandRing3 = 85, // parent: RightHandRing2 [84]

        RightHandRingEnd = 86, // parent: RightHandRing3 [85]

        RightHandThumbStart = 87, // parent: RightHand [66]

        RightHandThumb1 = 88, // parent: RightHandThumbStart [87]

        RightHandThumb2 = 89, // parent: RightHandThumb1 [88]

        RightHandThumbEnd = 90, // parent: RightHandThumb2 [89]

    }

[Subsystems] Loading plugin UnityARKit for subsystem ARKit-Input...

[Subsystems] UnityARKit successfully registered Provider for ARKit-Input

[Subsystems] UnityARKit successfully registered Provider for ARKit-Input

[Subsystems] UnityARKit successfully registered Provider for ARKit-Meshing

[Subsystems] Loading plugin UnityARKit for subsystem ARKit-Meshing...

New input system (experimental) initialized

2022-04-05 11:38:29.090361-0500 mirror[2254:341142] Unbalanced calls to begin/end appearance transitions for <SplashScreenController: 0x131010c20>.

UnloadTime: 2.807792 ms

MARS GeoLocation is not configured to request location service permission for this project. See 'GeoLocationModule' asset.

UnityEngine.DebugLogHandler:Internal_Log(LogType, LogOption, String, Object)

UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[])

UnityEngine.Logger:Log(LogType, Object)

UnityEngine.Debug:Log(Object)

Unity.MARS.Providers.GeoLocationModule:Unity.XRTools.ModuleLoader.IModule.LoadModule()

Unity.XRTools.ModuleLoader.ModuleLoaderCore:LoadModulesWithTypes(List`1)

Unity.XRTools.ModuleLoader.ModuleLoaderCore:LoadModules()

Unity.XRTools.ModuleLoader.ModuleLoaderCore:ReloadModules()

Unity.MARS.MARSSession:Awake()


[Subsystems] Subsystem ARKit-Input already instantiated, returning reference

[Subsystems] Subsystem ARKit-Meshing already instantiated, returning reference

No active UnityEngine.XR.ARSubsystems.XRFaceSubsystem is available. Please ensure that a valid loader configuration exists in the XR project settings.

UnityEngine.DebugLogHandler:Internal_Log(LogType, LogOption, String, Object)

UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[])

UnityEngine.Logger:LogFormat(LogType, String, Object[])

UnityEngine.Debug:LogWarningFormat(String, Object[])

UnityEngine.XR.ARFoundation.SubsystemLifecycleManager`3:GetActiveSubsystemInstance()

UnityEngine.XR.ARFoundation.SubsystemLifecycleManager`3:EnsureSubsystemInstanceSet()

UnityEngine.XR.ARFoundation.SubsystemLifecycleManager`3:OnEnable()

UnityEngine.XR.ARFoundation.ARTrackableManager`5:OnEnable()

UnityEngine.GameObject:Internal_AddComponentWithType(Type)

UnityEngine.GameObject:AddComponent(Type)

UnityEngine.GameObject:AddComponent()

Unity.MARS.Providers.ARFoundation.ARFoundationFaceTrackingProvider:Unity.XRTools.ModuleLoader.IFunctionalityProvider.LoadProvider()

Unity.XRTools.ModuleLoader.FunctionalityIsland:AddProvider(Type, IFunctionalityProvider)

Unity.XRTools.ModuleLoader.FunctionalityIsland:prepareProviders(HashSet`1, List`1, Boolean)

Unity.XRTools.ModuleLoader.FunctionalityIsland:InjectFunctionality(List`1, List`1)

Unity.MARS.Settings.MARSSceneModule:Unity.XRTools.ModuleLoader.IModuleBehaviorCallbacks.OnBehaviorAwake()

Unity.XRTools.ModuleLoader.ModuleLoaderCore:OnBehaviorAwake()

Unity.MARS.MARSSession:Awake()


Configuration Descriptor 0x1fb8e52d0 (rank 2): Rotation and Orientation, Plane Tracking, Light Estimation (Ambient Intensity), Light Estimation (Ambient Color), Raycast

Configuration Descriptor 0x1fb8e5118 (rank 0): World Facing Camera, Rotation and Orientation, Plane Tracking, Image Tracking, Object Tracking, Environment Probes, 2D Body Tracking, People Occlusion Stencil, People Occlusion Depth, Collaboration, Auto-Focus, Light Estimation (Ambient Intensity), Light Estimation (Ambient Color), Raycast, Meshing, Mesh Classification, Environment Depth, Environment Depth Temporal Smoothing

Configuration Descriptor 0x1fb8e53c0 (rank 1): World Facing Camera, Rotation Only, 2D Body Tracking, People Occlusion Stencil, People Occlusion Depth, Auto-Focus, Light Estimation (Ambient Intensity), Light Estimation (Ambient Color), Environment Depth, Environment Depth Temporal Smoothing

Configuration Descriptor 0x1fb8e4a60 (rank 1): World Facing Camera, Rotation Only, Image Tracking, 2D Body Tracking, People Occlusion Stencil, People Occlusion Depth, Auto-Focus, Light Estimation (Ambient Intensity), Light Estimation (Ambient Color), Environment Depth, Environment Depth Temporal Smoothing

Configuration Descriptor 0x1fb8e61f8 (rank -1): World Facing Camera, Rotation and Orientation, Plane Tracking, Image Tracking, Environment Probes, 2D Body Tracking, 3D Body Tracking, 3D Body Scale Estimation, Auto-Focus, Light Estimation (Ambient Intensity), Light Estimation (Ambient Color), Raycast

UnityEngine.DebugLogHandler:Internal_Log(LogType, LogOption, String, Object)

UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[])

UnityEngine.Logger:Log(LogType, Object)

UnityEngine.Debug:Log(Object)

UnityEngine.XR.ARSubsystems.XRSessionSubsystem:smile:etermineConfiguration(Feature)

UnityEngine.XR.ARSubsystems.XRSessionSubsystem:Update(XRSessionUpdateParams)

UnityEngine.XR.ARFoundation.ARSession:Update()


Using session configuration 0x1fb8e5118

    Requested Features: User Facing Camera, Rotation and Orientation, Auto-Focus

    Supported Features: Rotation and Orientation, Auto-Focus

    Requested features not satisfied: User Facing Camera

UnityEngine.DebugLogHandler:Internal_Log(LogType, LogOption, String, Object)

UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[])

UnityEngine.Logger:Log(LogType, Object)

UnityEngine.Debug:Log(Object)

UnityEngine.XR.ARSubsystems.XRSessionSubsystem:smile:ebugPrintConfigurationChange(Configuration, Feature)

UnityEngine.XR.ARSubsystems.XRSessionSubsystem:Update(XRSessionUpdateParams)

UnityEngine.XR.ARFoundation.ARSession:Update()


2022-04-05 11:38:31.454304-0500 mirror[2254:341142] UnityARKit: Updating ARSession configuration with <ARWorldTrackingConfiguration: 0x282ae9e00 worldAlignment=Gravity lightEstimation=Disabled frameSemantics=None videoFormat=<ARVideoFormat: 0x283c61d60 imageResolution=(1920, 1440) framesPerSecond=(60) captureDeviceType=AVCaptureDeviceTypeBuiltInWideAngleCamera captureDevicePosition=(1)> autoFocus=Enabled environmentTexturing=None wantsHDREnvironmentTextures=Enabled planeDetection=None collaboration=Disabled userFaceTracking=Disabled sceneReconstruction=None appClipCodeTracking=Disabled>

-> applicationWillResignActive()

WARNING -> applicationDidReceiveMemoryWarning()

-> applicationDidBecomeActive()

Found 3 interfaces on host iPhone: 0) 10.93.2.11 1) 162.165.5.241 2) 192.168.100.13

Multi-casting "[IP] 10.93.2.11 [Port] 55000 [Flags] 2 [Guid] 171254066 [EditorId] 485543332 [Version] 1048832 [Id] iPhonePlayer(8,iPhone):56000 [Debug] 0 [PackageName] iPhonePlayer [ProjectName] mirror" to [225.0.0.222:54997]...

-> applicationWillResignActive()

-> applicationDidBecomeActive()

Found 3 interfaces on host iPhone: 0) 10.93.2.11 1) 162.165.5.241 2) 192.168.100.13

Multi-casting "[IP] 10.93.2.11 [Port] 55000 [Flags] 2 [Guid] 171254066 [EditorId] 485543332 [Version] 1048832 [Id] iPhonePlayer(8,iPhone):56000 [Debug] 0 [PackageName] iPhonePlayer [ProjectName] mirror" to [225.0.0.222:54997]...

Bug confirmed, but I think its AR Foundation / ARKit unity packages. I downgraded to AR Foundation package version 4.0.2 and ARKit package version 4.0.1. and everything works now: I face tracking, and I am also able to access the front facing camera. BOTH face tracking and front facing camera is NOT operational in the newest recommended package versions.
@todds_unity

Did you turn on face tracking in the ARKit settings in Project settings? I always forget that, and if you don’t you will only get the rear camera.

1 Like

@CiaranWills I can’t find that setting, see screenshot. Am I looking at the wrong place?

But it shouldn’t matter anyway, when I am using the newest recommended package versions Face Tracking and Front Facing camera do not work. When I downgrade to AR Foundation package version 4.0.2 and ARKit package version 4.0.1 everything works without changing any settings.

You need to install the ARKit Face Tracking package in the Package Manager.

@CiaranWills Yes ARKit Face Tracking package is installed. The newest recommended version does not work but downgrading to ARKit Face Tracking package version 4.0.1 does work.

Ok that’s weird; with 4.2.2 you should see it like this:

Which version of XR Plug-in Management do you have (I have 4.2.1 here, in Unity 2021.2.19f1)?