MARS scenes don't work the second time I load them

alb-lab · May 16, 2021, 6:40pm

Hi everybody!
I’m working on a very simple MARS app.
There are 3 scenes:
1 - A simple menu to choose one of the other 2 scenes
2 - An AR scene with one ImageMarker that triggers the display of some prefabs
3 - An AR scene with face tracking, to show some prefab around the face of the user

The app works well on Android phones but on iOS only the first executed AR scene works well. If I return to the menu and choose the other scene (or the same scene) it doesn’t work anymore.

In other words consider the following sequence:

Execute the app
In the menu choose scene 2 or scene 3, it works well
Return to the menu (there’s a button for this action)
Choose scene 2 or scene 3, it doesn’t work.

Any suggestion ?
Thank you in advance.

alb-lab · May 16, 2021, 6:44pm

This is the log output.
As you can see the sequence of scenes is:
Menu → Scene2 → Menu → Scene2

2021-05-16 19:47:21.174711+0200 TerraIgnota[2118:1344572] Built from '2020.2/staging' branch, Version '2020.2.3f1 (8ff31bc5bf5b)', Build type 'Release', Scripting Backend 'il2cpp'
2021-05-16 19:47:21.175947+0200 TerraIgnota[2118:1344572] MemoryManager: Using 'Default' Allocator.
-> applicationDidFinishLaunching()
-> applicationDidBecomeActive()
[Subsystems] Discovering subsystems at path /private/var/containers/Bundle/Application/9B6FCD4F-FFE1-4EFD-BD0F-A296311F0376/TerraIgnota.app/Data/UnitySubsystems
[Subsystems] No descriptors matched for    examples in UnitySubsystems/UnityARKit/UnitySubsystemsManifest.json.
[Subsystems] 1 'inputs' descriptors matched in UnitySubsystems/UnityARKit/UnitySubsystemsManifest.json
[Subsystems] No descriptors matched for    displays in UnitySubsystems/UnityARKit/UnitySubsystemsManifest.json.
[Subsystems] 1 'meshings' descriptors matched in UnitySubsystems/UnityARKit/UnitySubsystemsManifest.json
GfxDevice: creating device client; threaded=1
Initializing Metal device caps: Apple A12Z GPU
Initialize engine version: 2020.2.3f1 (8ff31bc5bf5b)
XRGeneral Settings awakening...
UnityEngine.XR.Management.XRGeneralSettings:Awake()

2021-05-16 19:47:22.489470+0200 TerraIgnota[2118:1344572]
        // 2D joint skeleton
        enum JointIndices
        {
                Invalid = -1,
                Head = 0, // parent: Neck1 [1]
                Neck1 = 1, // parent: Root [16]
                RightShoulder1 = 2, // parent: Neck1 [1]
                RightForearm = 3, // parent: RightShoulder1 [2]
                RightHand = 4, // parent: RightForearm [3]
                LeftShoulder1 = 5, // parent: Neck1 [1]
                LeftForearm = 6, // parent: LeftShoulder1 [5]
                LeftHand = 7, // parent: LeftForearm [6]
                RightUpLeg = 8, // parent: Root [16]
                RightLeg = 9, // parent: RightUpLeg [8]
                RightFoot = 10, // parent: RightLeg [9]
                LeftUpLeg = 11, // parent: Root [16]
                LeftLeg = 12, // parent: LeftUpLeg [11]
                LeftFoot = 13, // parent: LeftLeg [12]
                RightEye = 14, // parent: Head [0]
                LeftEye = 15, // parent: Head [0]
                Root = 16, // parent: <none> [-1]
        }
2021-05-16 19:47:22.494524+0200 TerraIgnota[2118:1344572]
        // 3D joint skeleton
        enum JointIndices
        {
                Invalid = -1,
                Root = 0, // parent: <none> [-1]
                Hips = 1, // parent: Root [0]
                LeftUpLeg = 2, // parent: Hips [1]
                LeftLeg = 3, // parent: LeftUpLeg [2]
                LeftFoot = 4, // parent: LeftLeg [3]
                LeftToes = 5, // parent: LeftFoot [4]
                LeftToesEnd = 6, // parent: LeftToes [5]
                RightUpLeg = 7, // parent: Hips [1]
                RightLeg = 8, // parent: RightUpLeg [7]
                RightFoot = 9, // parent: RightLeg [8]
                RightToes = 10, // parent: RightFoot [9]
                RightToesEnd = 11, // parent: RightToes [10]
                Spine1 = 12, // parent: Hips [1]
                Spine2 = 13, // parent: Spine1 [12]
                Spine3 = 14, // parent: Spine2 [13]
                Spine4 = 15, // parent: Spine3 [14]
                Spine5 = 16, // parent: Spine4 [15]
                Spine6 = 17, // parent: Spine5 [16]
                Spine7 = 18, // parent: Spine6 [17]
                LeftShoulder1 = 19, // parent: Spine7 [18]
                LeftArm = 20, // parent: LeftShoulder1 [19]
                LeftForearm = 21, // parent: LeftArm [20]
                LeftHand = 22, // parent: LeftForearm [21]
                LeftHandIndexStart = 23, // parent: LeftHand [22]
                LeftHandIndex1 = 24, // parent: LeftHandIndexStart [23]
                LeftHandIndex2 = 25, // parent: LeftHandIndex1 [24]
                LeftHandIndex3 = 26, // parent: LeftHandIndex2 [25]
                LeftHandIndexEnd = 27, // parent: LeftHandIndex3 [26]
                LeftHandMidStart = 28, // parent: LeftHand [22]
                LeftHandMid1 = 29, // parent: LeftHandMidStart [28]
                LeftHandMid2 = 30, // parent: LeftHandMid1 [29]
                LeftHandMid3 = 31, // parent: LeftHandMid2 [30]
                LeftHandMidEnd = 32, // parent: LeftHandMid3 [31]
                LeftHandPinkyStart = 33, // parent: LeftHand [22]
                LeftHandPinky1 = 34, // parent: LeftHandPinkyStart [33]
                LeftHandPinky2 = 35, // parent: LeftHandPinky1 [34]
                LeftHandPinky3 = 36, // parent: LeftHandPinky2 [35]
                LeftHandPinkyEnd = 37, // parent: LeftHandPinky3 [36]
                LeftHandRingStart = 38, // parent: LeftHand [22]
                LeftHandRing1 = 39, // parent: LeftHandRingStart [38]
                LeftHandRing2 = 40, // parent: LeftHandRing1 [39]
                LeftHandRing3 = 41, // parent: LeftHandRing2 [40]
                LeftHandRingEnd = 42, // parent: LeftHandRing3 [41]
                LeftHandThumbStart = 43, // parent: LeftHand [22]
                LeftHandThumb1 = 44, // parent: LeftHandThumbStart [43]
                LeftHandThumb2 = 45, // parent: LeftHandThumb1 [44]
                LeftHandThumbEnd = 46, // parent: LeftHandThumb2 [45]
                Neck1 = 47, // parent: Spine7 [18]
                Neck2 = 48, // parent: Neck1 [47]
                Neck3 = 49, // parent: Neck2 [48]
                Neck4 = 50, // parent: Neck3 [49]
                Head = 51, // parent: Neck4 [50]
                Jaw = 52, // parent: Head [51]
                Chin = 53, // parent: Jaw [52]
                LeftEye = 54, // parent: Head [51]
                LeftEyeLowerLid = 55, // parent: LeftEye [54]

                LeftEyeUpperLid = 56, // parent: LeftEye [54]
                LeftEyeball = 57, // parent: LeftEye [54]
                Nose = 58, // parent: Head [51]
                RightEye = 59, // parent: Head [51]
                RightEyeLowerLid = 60, // parent: RightEye [59]
                RightEyeUpperLid = 61, // parent: RightEye [59]
                RightEyeball = 62, // parent: RightEye [59]
                RightShoulder1 = 63, // parent: Spine7 [18]
                RightArm = 64, // parent: RightShoulder1 [63]
                RightForearm = 65, // parent: RightArm [64]
                RightHand = 66, // parent: RightForearm [65]
                RightHandIndexStart = 67, // parent: RightHand [66]
                RightHandIndex1 = 68, // parent: RightHandIndexStart [67]
                RightHandIndex2 = 69, // parent: RightHandIndex1 [68]
                RightHandIndex3 = 70, // parent: RightHandIndex2 [69]
                RightHandIndexEnd = 71, // parent: RightHandIndex3 [70]
                RightHandMidStart = 72, // parent: RightHand [66]
                RightHandMid1 = 73, // parent: RightHandMidStart [72]
                RightHandMid2 = 74, // parent: RightHandMid1 [73]
                RightHandMid3 = 75, // parent: RightHandMid2 [74]
                RightHandMidEnd = 76, // parent: RightHandMid3 [75]
                RightHandPinkyStart = 77, // parent: RightHand [66]
                RightHandPinky1 = 78, // parent: RightHandPinkyStart [77]
                RightHandPinky2 = 79, // parent: RightHandPinky1 [78]
                RightHandPinky3 = 80, // parent: RightHandPinky2 [79]
                RightHandPinkyEnd = 81, // parent: RightHandPinky3 [80]
                RightHandRingStart = 82, // parent: RightHand [66]
                RightHandRing1 = 83, // parent: RightHandRingStart [82]
                RightHandRing2 = 84, // parent: RightHandRing1 [83]
                RightHandRing3 = 85, // parent: RightHandRing2 [84]
                RightHandRingEnd = 86, // parent: RightHandRing3 [85]
                RightHandThumbStart = 87, // parent: RightHand [66]
                RightHandThumb1 = 88, // parent: RightHandThumbStart [87]
                RightHandThumb2 = 89, // parent: RightHandThumb1 [88]
                RightHandThumbEnd = 90, // parent: RightHandThumb2 [89]
        }
[Subsystems] Loading plugin UnityARKit for subsystem ARKit-Input...
[Subsystems] UnityARKit successfully registered Provider for ARKit-Input
[Subsystems] UnityARKit successfully registered Provider for ARKit-Input
[Subsystems] UnityARKit successfully registered Provider for ARKit-Meshing
[Subsystems] Loading plugin UnityARKit for subsystem ARKit-Meshing...
2021-05-16 19:47:22.849763+0200 TerraIgnota[2118:1344572] Unbalanced calls to begin/end appearance transitions for <UnityViewControllerStoryboard: 0x13bf31b70>.
UnloadTime: 3.597875 ms
Unloading 6 Unused Serialized files (Serialized files now loaded: 0)
UnloadTime: 0.697000 ms

Unloading 8 unused Assets to reduce memory usage. Loaded Objects now: 18375.
Total: 5.530167 ms (FindLiveObjects: 0.551458 ms CreateObjectMapping: 0.045917 ms MarkObjects: 4.879792 ms    DeleteObjects: 0.052458 ms)


----> LOAD SCENE 2 <----


Unloading 7 Unused Serialized files (Serialized files now loaded: 0)
UnloadTime: 11.567958 ms
MARS GeoLocation is not configured to request location service permission for this project. See 'GeoLocationModule' asset.
Unity.MARS.Providers.GeoLocationModule:Unity.XRTools.ModuleLoader.IModule.LoadModule()
Unity.XRTools.ModuleLoader.ModuleLoaderCore:LoadModulesWithTypes(List`1)
Unity.MARS.MARSSession:Awake()


Unloading 60 unused Assets to reduce memory usage. Loaded Objects now: 129196.
Total: 10.376042 ms (FindLiveObjects: 4.189208 ms CreateObjectMapping: 0.104667 ms MarkObjects: 5.307167 ms    DeleteObjects: 0.774208 ms)

2021-05-16 19:47:56.239567+0200 TerraIgnota[2118:1344572] UnityARKit: Updating ARSession configuration with <ARWorldTrackingConfiguration: 0x280869c30 worldAlignment=Gravity lightEstimation=Disabled frameSemantics=None videoFormat=<ARVideoFormat: 0x281e481e0 imageResolution=(1920, 1440) framesPerSecond=(60)> autoFocus=Enabled environmentTexturing=None wantsHDREnvironmentTextures=Enabled planeDetection=None collaboration=Disabled userFaceTracking=Disabled sceneReconstruction=None detectionImages=[<ARReferenceImage: 0x281b22580 name="TerraIgnotaImgMarker_6013055B-BD03-CD4E-869C-DFE943A91076" physicalSize=(1.000, 1.000)>] maximumNumberOfTrackedImages=4 automaticImageScaleEstimation=Disabled appClipCodeTracking=Disabled>


----> LOAD MENU SCENE <----


UnityEngine.Events.UnityAction:Invoke()
UnityEngine.Events.UnityEvent:Invoke()
UnityEngine.EventSystems.EventFunction`1:Invoke(T1, BaseEventData)
UnityEngine.EventSystems.ExecuteEvents:Execute(GameObject, BaseEventData, EventFunction`1)
UnityEngine.EventSystems.StandaloneInputModule:ProcessTouchPress(PointerEventData, Boolean, Boolean)
UnityEngine.EventSystems.StandaloneInputModule:ProcessTouchEvents()
UnityEngine.EventSystems.StandaloneInputModule:Process()

Unloading 5 Unused Serialized files (Serialized files now loaded: 0)
UnloadTime: 124.216667 ms

Unloading 112 unused Assets to reduce memory usage. Loaded Objects now: 18469.
Total: 7.918667 ms (FindLiveObjects: 0.518625 ms CreateObjectMapping: 0.043500 ms MarkObjects: 5.665208 ms    DeleteObjects: 1.690750 ms)


----> LOAD SCENE 2 <----

Unloading 7 Unused Serialized files (Serialized files now loaded: 0)
UnloadTime: 12.749250 ms
MARS GeoLocation is not configured to request location service permission for this project. See 'GeoLocationModule' asset.
Unity.MARS.Providers.GeoLocationModule:Unity.XRTools.ModuleLoader.IModule.LoadModule()
Unity.XRTools.ModuleLoader.ModuleLoaderCore:LoadModulesWithTypes(List`1)
Unity.MARS.MARSSession:Awake()

2021-05-16 19:48:28.843521+0200 TerraIgnota[2118:1344572] UnityARKit: Updating ARSession configuration with <ARWorldTrackingConfiguration: 0x280806f70 worldAlignment=Gravity lightEstimation=Disabled frameSemantics=None videoFormat=<ARVideoFormat: 0x281e481e0 imageResolution=(1920, 1440) framesPerSecond=(60)> autoFocus=Enabled environmentTexturing=None wantsHDREnvironmentTextures=Enabled planeDetection=None collaboration=Disabled userFaceTracking=Disabled sceneReconstruction=None detectionImages=[<ARReferenceImage: 0x281b22580 name="TerraIgnotaImgMarker_6013055B-BD03-CD4E-869C-DFE943A91076" physicalSize=(1.000, 1.000)>] maximumNumberOfTrackedImages=4 automaticImageScaleEstimation=Disabled appClipCodeTracking=Disabled>

Unloading 61 unused Assets to reduce memory usage. Loaded Objects now: 129196.
Total: 9.701417 ms (FindLiveObjects: 3.863833 ms CreateObjectMapping: 0.090167 ms MarkObjects: 5.294500 ms    DeleteObjects: 0.452292 ms)

-> applicationWillResignActive()
-> applicationDidEnterBackground()

CiaranWills · May 17, 2021, 4:20pm

This is an issue that has come up before for iOS; we’re looking into it but a workaround is to reset the session when changing scenes:

ARFoundationSessionProvider.currentSession?.Reset();

alb-lab · May 17, 2021, 6:58pm

CiaranWills:

This is an issue that has come up before for iOS; we’re looking into it but a workaround is to reset the session when changing scenes:
ARFoundationSessionProvider.currentSession?.Reset();

Thank you very much for you reply !
With this solution the situation improved.

Now I can enter and exit the scene 2 (the one with the image marker) over and over without any fault. That’s good.

But if I enter the scene 3 (the one with face tracking) it only works for the first time. From the second time on, it starts to drift the location of my 3d objects. This problem affects also the objects of scene 2.

It seems that the face tracking scene corrupts in some way all subsequent AR tracking and adds some sort of displacement to the position of the head or of the image marker.

Any other idea to solve this ?

alb-lab · May 23, 2021, 3:42pm

Can’t find a way to make this work.

Any help will be very appreciated !

jmunozarUTech · May 23, 2021, 10:54pm

Hello @alb-lab we are looking into this.

alb-lab · May 24, 2021, 4:49pm

Thank you!
From your reply I suppose you’ve been able to replicate the problem.

My client wants the iOS version now… The app is very basic without any fancy feature other than different scenes.

G_Abb · May 25, 2021, 4:41pm

Hi everybody!

I’m Gianluca, I’m working with Alberto on an augmented reality application developed with Mars.
The work we are developing is an artistic application for a photography exhibition in Italy.
The exhibition is open to the public and the application is already available on google store.
It runs very well on android but the client is waiting for a Ios release
Unfortunately as Alberto described with IOS “It seems that the face tracking scene corrupts in some way all subsequent AR tracking and adds some sort of displacement to the position of the head or of the image marker” so we are really stuck on this.
In one way or another we need to solve this problem.

On https://www.terraignota.site take a quick look at the project and the application we are working on.

I was wondering is it possible to have support on these kinds of issues or maybe I should sign up for Unity Core Support?

Mars is amazing and I hope to hear from you soon!
Thanks guys in advance!

jmunozarUTech · May 25, 2021, 4:50pm

hello @G_Abb , @alb-lab ,

We have managed to reproduce this issue and we are working on a fix with urgency, will update when we either have a workaround or a fix for you guys

alb-lab · May 25, 2021, 6:55pm

Thank you so much @jmunozarUTech for taking care of this problem with urgency ! We really need a fix or a workaround.

CiaranWills · May 25, 2021, 9:35pm

In the face tracking scene can you try setting the Requested Max Face Count on the MARS Session to 1 and let me know if that improves things any?

There seem to be a couple of intersecting bugs here that I am still getting to the bottom of…

alb-lab · May 25, 2021, 10:57pm

Just tried changing that parameter from -1 to 1 and unfortunately there’s no difference.

CiaranWills · May 25, 2021, 11:39pm

Also, if you aren’t already, call AR Foundation’s LoaderUtility Deinitialize() and Initialize() when switching scenes. I’ve seen issues when switching between the front and rear cameras if you don’t do this.

alb-lab · May 26, 2021, 9:15pm

I tried adding the calls to Initialize() and Deinitialize() and after a bit of trial and error everything works well now!

It’s still not clear to me if the problem was due to the change of scene or to the change of front/back camera.

Anyway, thank you so much for your support !

CiaranWills · May 26, 2021, 9:33pm

Awesome - glad it’s working or you!

We’ll have MARS handle this internally in a future version, but for now the workaround is to call it yourself when switching scenes.

G_Abb · May 27, 2021, 12:23pm

Thank you so much guys!
The application works very well now also on IOS!

▀█▀ █░█ ▄▀█ █▄░█ █▄▀
░█░ █▀█ █▀█ █░▀█ █░█

█▄█ █▀█ █░█
░█░ █▄█ █▄█