Camera feed black when spinning up AR

Unity Engine
,
1

Hi, I have a project where we first start in just regular 3D mode and then the user can switch to using AR. Testing this on the iPhone and using the XRGeneralSettings.Instance.Manager.StartSubsystems code on the Unity XRManagement website: End-user documentation | XR Plugin Management | 3.2.17

When we hit the button to switch into AR mode on an iPhone w/ ARKit the screen is just black though. Do we need to initialize something else? Also, there are two cameras in the scene, one for the regular 3D scene and then the AR camera. Before switching modes we enable/disable the appropriate cameras. Setup is:

Unity 2020.1.13
ARKit/ARFoundation 4.1.1
macOS Big Sur
iPhone 12 Pro

2

Could you please tell what render pipeline do you use?

Have you tried this samples repo?
https://github.com/Unity-Technologies/arfoundation-samples

3

Yes, using the Builtin render pipeline currently. Will move to URP soon but not yet. Read about all the hoops you need to jump through when you do that but not currently using it. Yes, have used the sample repos and built out to device many times. This project also used to be an AR only project and was working fine. Now the we have the switching functionality it doesn’t work. Per the documentation I have “Initialize XR at Startup” off. Then when you hit the AR button we use the code described above with StartSubsystems. It looks like it loads correctly and there doesn’t appear to be any errors:

Initializing XR...
UnityEngine.DebugLogHandler:Internal_Log(LogType, LogOption, String, Object)
UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[ ])
UnityEngine.Logger:Log(LogType, Object)
UnityEngine.Debug:Log(Object)
<StartXR>d__11:MoveNext()
UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
UnityEngine.MonoBehaviour:StartCoroutineManaged2(IEnumerator)
UnityEngine.MonoBehaviour:StartCoroutine(IEnumerator)
ARController:<Awake>b__10_0(Boolean)
UnityEngine.Events.UnityAction`1:Invoke(T0)
Menu:ToggleAR(Boolean)
UnityEngine.Events.UnityAction`1:Invoke(T0)
UnityEngine.Events.InvokableCall`1:Invoke(T1)
UnityEngine.Events.UnityEvent`1:Invoke(T0)
UnityEngine.UI.Toggle:Set(Boolean, Boolean)
UnityEngine.UI.Toggle:set_isOn(Boolean)
UnityEngine.UI.Toggle:InternalToggle()
UnityEngine.UI.Toggle:OnPointerClick(PointerEventData)
UnityEngine.EventSystems.ExecuteEvents:Execute(IPointerClickHandler, BaseEventData)
UnityEngine.EventSystems.EventFunction`1:Invoke(T1, BaseEventData)
UnityEngine.EventSystems.ExecuteEvents:Execute(GameObject, BaseEventData, EventFunction`1)
UnityEngine.XR.Interaction.Toolkit.UI.UIInputModule:ProcessMouseButton(ButtonDeltaState, PointerEventData)
UnityEngine.XR.Interaction.Toolkit.UI.UIInputModule:ProcessTouch(TouchModel&)
UnityEngine.XR.Interaction.Toolkit.UI.XRUIInputModule:ProcessTouches()
UnityEngine.XR.Interaction.Toolkit.UI.XRUIInputModule:smile:oProcess()
UnityEngine.XR.Interaction.Toolkit.UI.UIInputModule:Process()
UnityEngine.EventSystems.EventSystem:Update()

(Filename: ./Runtime/Export/Debug/Debug.bindings.h Line: 35)

2021-03-01 19:29:48.745238-0800 planeshifter[25850:20163742] 
// 2D joint skeleton
enum JointIndices
{
Invalid = -1,
Head = 0, // parent: Neck1 [1]
Neck1 = 1, // parent: Root [16]
RightShoulder1 = 2, // parent: Neck1 [1]
RightForearm = 3, // parent: RightShoulder1 [2]
RightHand = 4, // parent: RightForearm [3]
LeftShoulder1 = 5, // parent: Neck1 [1]
LeftForearm = 6, // parent: LeftShoulder1 [5]
LeftHand = 7, // parent: LeftForearm [6]
RightUpLeg = 8, // parent: Root [16]
RightLeg = 9, // parent: RightUpLeg [8]
RightFoot = 10, // parent: RightLeg [9]
LeftUpLeg = 11, // parent: Root [16]
LeftLeg = 12, // parent: LeftUpLeg [11]
LeftFoot = 13, // parent: LeftLeg [12]
RightEye = 14, // parent: Head [0]
LeftEye = 15, // parent: Head [0]
Root = 16, // parent: <none> [-1]
}
2021-03-01 19:29:48.751863-0800 planeshifter[25850:20163742] 
// 3D joint skeleton
enum JointIndices
{
Invalid = -1,
Root = 0, // parent: <none> [-1]
Hips = 1, // parent: Root [0]
LeftUpLeg = 2, // parent: Hips [1]
LeftLeg = 3, // parent: LeftUpLeg [2]
LeftFoot = 4, // parent: LeftLeg [3]
LeftToes = 5, // parent: LeftFoot [4]
LeftToesEnd = 6, // parent: LeftToes [5]
RightUpLeg = 7, // parent: Hips [1]
RightLeg = 8, // parent: RightUpLeg [7]
RightFoot = 9, // parent: RightLeg [8]
RightToes = 10, // parent: RightFoot [9]
RightToesEnd = 11, // parent: RightToes [10]
Spine1 = 12, // parent: Hips [1]
Spine2 = 13, // parent: Spine1 [12]
Spine3 = 14, // parent: Spine2 [13]
Spine4 = 15, // parent: Spine3 [14]
Spine5 = 16, // parent: Spine4 [15]
Spine6 = 17, // parent: Spine5 [16]
Spine7 = 18, // parent: Spine6 [17]
LeftShoulder1 = 19, // parent: Spine7 [18]
LeftArm = 20, // parent: LeftShoulder1 [19]
LeftForearm = 21, // parent: LeftArm [20]
LeftHand = 22, // parent: LeftForearm [21]
LeftHandIndexStart = 23, // parent: LeftHand [22]
LeftHandIndex1 = 24, // parent: LeftHandIndexStart [23]
LeftHandIndex2 = 25, // parent: LeftHandIndex1 [24]
LeftHandIndex3 = 26, // parent: LeftHandIndex2 [25]
LeftHandIndexEnd = 27, // parent: LeftHandIndex3 [26]
LeftHandMidStart = 28, // parent: LeftHand [22]
LeftHandMid1 = 29, // parent: LeftHandMidStart [28]
LeftHandMid2 = 30, // parent: LeftHandMid1 [29]
LeftHandMid3 = 31, // parent: LeftHandMid2 [30]
LeftHandMidEnd = 32, // parent: LeftHandMid3 [31]
LeftHandPinkyStart = 33, // parent: LeftHand [22]
LeftHandPinky1 = 34, // parent: LeftHandPinkyStart [33]
LeftHandPinky2 = 35, // parent: LeftHandPinky1 [34]
LeftHandPinky3 = 36, // parent: LeftHandPinky2 [35]
LeftHandPinkyEnd = 37, // parent: LeftHandPinky3 [36]
LeftHandRingStart = 38, // parent: LeftHand [22]
LeftHandRing1 = 39, // parent: LeftHandRingStart [38]
LeftHandRing2 = 40, // parent: LeftHandRing1 [39]
LeftHandRing3 = 41, // parent: LeftHandRing2 [40]
LeftHandRingEnd = 42, // parent: LeftHandRing3 [41]
LeftHandThumbStart = 43, // parent: LeftHand [22]
LeftHandThumb1 = 44, // parent: LeftHandThumbStart [43]
LeftHandThumb2 = 45, // parent: LeftHandThumb1 [44]
LeftHandThumbEnd = 46, // parent: LeftHandThumb2 [45]
Neck1 = 47, // parent: Spine7 [18]
Neck2 = 48, // parent: Neck1 [47]
Neck3 = 49, // parent: Neck2 [48]
Neck4 = 50, // parent: Neck3 [49]
Head = 51, // parent: Neck4 [50]
Jaw = 52, // parent: Head [51]
Chin = 53, // parent: Jaw [52]
LeftEye = 54, // parent: Head [51]
LeftEyeLowerLid = 55, // parent: LeftEye [54]
LeftEyeUpperLid = 56, // parent: LeftEye [54]
LeftEyeball = 57, // parent: LeftEye [54]
Nose = 58, // parent: Head [51]
RightEye = 59, // parent: Head [51]
RightEyeLowerLid = 60, // parent: RightEye [59]
RightEyeUpperLid = 61, // parent: RightEye [59]
RightEyeball = 62, // parent: RightEye [59]
RightShoulder1 = 63, // parent: Spine7 [18]
RightArm = 64, // parent: RightShoulder1 [63]
RightForearm = 65, // parent: RightArm [64]
RightHand = 66, // parent: RightForearm [65]
RightHandIndexStart = 67, // parent: RightHand [66]
RightHandIndex1 = 68, // parent: RightHandIndexStart [67]
RightHandIndex2 = 69, // parent: RightHandIndex1 [68]
RightHandIndex3 = 70, // parent: RightHandIndex2 [69]
RightHandIndexEnd = 71, // parent: RightHandIndex3 [70]
RightHandMidStart = 72, // parent: RightHand [66]
RightHandMid1 = 73, // parent: RightHandMidStart [72]
RightHandMid2 = 74, // parent: RightHandMid1 [73]
RightHandMid3 = 75, // parent: RightHandMid2 [74]
RightHandMidEnd = 76, // parent: RightHandMid3 [75]
RightHandPinkyStart = 77, // parent: RightHand [66]
RightHandPinky1 = 78, // parent: RightHandPinkyStart [77]
RightHandPinky2 = 79, // parent: RightHandPinky1 [78]
RightHandPinky3 = 80, // parent: RightHandPinky2 [79]
RightHandPinkyEnd = 81, // parent: RightHandPinky3 [80]
RightHandRingStart = 82, // parent: RightHand [66]
RightHandRing1 = 83, // parent: RightHandRingStart [82]
RightHandRing2 = 84, // parent: RightHandRing1 [83]
RightHandRing3 = 85, // parent: RightHandRing2 [84]
RightHandRingEnd = 86, // parent: RightHandRing3 [85]
RightHandThumbStart = 87, // parent: RightHand [66]
RightHandThumb1 = 88, // parent: RightHandThumbStart [87]
RightHandThumb2 = 89, // parent: RightHandThumb1 [88]
RightHandThumbEnd = 90, // parent: RightHandThumb2 [89]
}
[Subsystems] Loading plugin UnityARKit for subsystem ARKit-Input...
[Subsystems] UnityARKit successfully registered Provider for ARKit-Input
[Subsystems] UnityARKit successfully registered Provider for ARKit-Input
[Subsystems] UnityARKit successfully registered Provider for ARKit-Meshing
[Subsystems] Loading plugin UnityARKit for subsystem ARKit-Meshing...
Starting XR...
UnityEngine.DebugLogHandler:Internal_Log(LogType, LogOption, String, Object)
UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[ ])
UnityEngine.Logger:Log(LogType, Object)
UnityEngine.Debug:Log(Object)
<StartXR>d__11:MoveNext()
UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
4

Interestingly, I deleted the app and rebuilt and reinstalled it and it’s not asking for permission to use the camera either. Which it should. So why is the camera not being activated? Again, I am using two different cameras when switching between AR and regular 3D mode but each is activated/deactivated appropriately. I activate the AR camera before I turn on all the subsystems. Stumped.

1 Like
5

@KyryloKuzyk any ideas? Would appreciate your input. Many thanks!

6

OK I found out what is going on. Sort of. Again, I have been following these instructions to manage the lifecycle of AR and turn AR on and off:

https://docs.unity3d.com/Packages/com.unity.xr.management@3.2/manual/EndUser.html

According to those instructions you are supposed to:

  • Access the Project Settings window (menu: Edit > Project Settings).
  • Select the XR Plug-in Management tab on the left.
  • Disable the Initialize on start option for each platform you support.
  • At runtime, call the following methods on XRGeneralSettings.Instance.Manager to add/create, remove, and reorder the Loaders from your scripts

When you disable the Init on Start option in Project Settings->XR Management and then build to device (in this case an iPhone) and use the code in the above link to InitializeLoader(Async) and StartSubsytems, it just doesn’t work. The camera feed comes up black even though in Xcode you can see the systems were initialized. Ticking Init on Start in Project Settings fixes the whole situation. But then that has all the subsystems fired up from the beginning. In a fresh install of the app it doesn’t even ask for camera permission if Init on Start is not checked in Project Settings and you fire up AR in the linked code. So something is not working correctly here.

Furthermore, when turning it off with StopSubsystems this doesn’t appear to work either. After it says subsystems are stopped, anchors are still being updated in the background constantly by ARFoundation.ARTrackableManager. Shouldn’t all of this be stopped if the subsystems are all stopped? How can anchors be updating if tracking is theoretically stopped?

Am I fundamentally misunderstanding all this?
Do we also need to disable the AR Session node? And AR Session Origin also?
Do we need to turn off each component like Plane Manager and Anchor Manager too?
Does anyone have a full and complete example of stopping and starting AR cleanly?
Can someone from Unity chime in here?

I scoured the forums and elsewhere and seen all kinds of “solutions” but nothing that really addresses this completely. Please help! Thanks.

7

Team had a bit of guidance to share:

So, you should not be using XRGeneralSettings.Instance.Manager.StartSubsystems to start AR.

The AR managers in AR Foundation will start the subsystems as needed. You should be enabling the AR managers.

8

Thanks @TreyK-47 ! Sooo… are you guys going to be changing the documentation? What is Start/StopSubsystems for then? And is it enough to just stop ARSession and that will stop everything else or do we need to disable every manager? Many thanks.

2 Likes
9

The Start/StopSubsystems documentation is part of the XR Management package.

AR Foundation documentation does not mention Start/StopSubsystems.

Todd