Some crashes on Unity 4.6.4 for iOS - 64bits

Hi there

We have an app, built for 64bits with IL2CPP for iOS with Fabric (formerly called Crashlytics) and we are receiving some crashed on AppWillTerminate with all version of Unity (we started working on it on 4.6.3). It’s quite strange, because to it seems to happen once the app is killed from background by OS due to memory pressure

Do you know anything about this? It’s quite weird

Regards

Here is the raw of the crash

Thread : Crashed: com.apple.main-thread
0 libsystem_kernel.dylib 0x3853ddf0 __pthread_kill + 8
1 libsystem_pthread.dylib 0x385bccc7 pthread_kill + 62
2 libsystem_c.dylib 0x384d9909 abort + 76
3 Scooby 0x01865270 GC_stop_world (darwin_stop_world.c:569)
4 Scooby 0x018644f4 GC_stopped_mark (alloc.c:610)
5 Scooby 0x0186407c GC_try_to_collect_inner (alloc.c:460)
6 Scooby 0x01860a6c GC_collect_or_expand (alloc.c:1260)
7 Scooby 0x01860d64 GC_allocobj (alloc.c:1349)
8 Scooby 0x0185f544 GC_generic_malloc_inner (malloc.c:133)
9 Scooby 0x01860e34 GC_generic_malloc (malloc.c:182)
10 Scooby 0x01861870 GC_malloc (misc.c:381)
11 Scooby 0x018b7b10 il2cpp::vm::Object::Clone(Il2CppObject*) (Object.cpp:52)
12 Scooby 0x00b104f8 MulticastDelegate_RemoveImpl_m1569 (GeneratedVirtualInvokers.h:20)
13 Scooby 0x00b11ef0 Delegate_Remove_m1466 (Bulk_mscorlib_0.cpp:61557)
14 Scooby 0x001e6168 EventManager_remove_OnDarkBackgroundOut_m3980 (il2cpp-codegen.h:331)
15 Scooby 0x002fedd0 TutorialEvent_OnDestroy_m4589 (il2cpp-codegen.h:331)
16 Scooby 0x00ce2ce0 RuntimeInvoker_Void_t54(MethodInfo*, void*, void**) (GeneratedInvokers.cpp:12)
17 Scooby 0x018bd048 il2cpp::vm::Runtime::Invoke(MethodInfo*, void*, void**, Il2CppObject**) (Runtime.cpp:307)
18 Scooby 0x0130a344 ScriptingInvocation::Invoke(ScriptingException**, bool) (ScriptingInvocation.cpp:128)
19 Scooby 0x0130a8dc ScriptingInvocationNoArgs::InvokeChecked() (ScriptingInvocationNoArgs.cpp:83)
20 Scooby 0x012fe21c MonoBehaviour::CallMethodInactive(ScriptingMethodIl2Cpp) (MonoBehaviour.cpp:370)
21 Scooby 0x01114664 Unity::GameObject::WillDestroyGameObject() (stl_iterator.h:752)
22 Scooby 0x012a8264 PreDestroyRecursive(Unity::GameObject&, unsigned long*) (stl_vector.h:400)
23 Scooby 0x012a84bc DestroyGameObjectHierarchy(Unity::GameObject&) (GameObject.h:424)
24 Scooby 0x012a8904 DestroyObjectHighLevel(Object*, bool) (GameObjectUtility.cpp:1447)
25 Scooby 0x012c7bd8 CleanupAllObjects(bool) (BaseObject.h:845)
26 Scooby 0x012c8430 CleanupEngine() (SaveAndLoadHelper.cpp:450)
27 Scooby 0x012b004c PlayerCleanup(bool, bool) (Player.cpp:1162)
28 Scooby 0x01005378 UnityCleanup (LibEntryPoint.mm:235)
29 UIKit 0x2d1d18b1 -[UIApplication _terminateWithStatus:] + 192
30 UIKit 0x2d3b8349 -[UIApplication _handleApplicationDectivationWithScene:shouldForceExit:transitionContext:completion:] + 2416
31 UIKit 0x2d3b38e7 -[UIApplication workspaceShouldExit:] + 166
32 FrontBoardServices 0x305c2ec9 __31-[FBSSerialQueue performAsync:]_block_invoke_2 + 16
33 CoreFoundation 0x29ac1fd5 CFRUNLOOP_IS_CALLING_OUT_TO_A_BLOCK + 12
34 CoreFoundation 0x29ac1299 __CFRunLoopDoBlocks + 216
35 CoreFoundation 0x29abfa37 __CFRunLoopRun + 790
36 CoreFoundation 0x29a0c201 CFRunLoopRunSpecific + 476
37 CoreFoundation 0x29a0c013 CFRunLoopRunInMode + 106
38 GraphicsServices 0x311e8201 GSEventRunModal + 136
39 UIKit 0x2d1b0a59 UIApplicationMain + 1440
40 Scooby 0x000b09e4 main (main.mm:45)

We’ve not seen a crash like this before, from what I can gather. Is this something reproducible, or does it occur intermittently?

Thanks Josh for the time. Yes, it is reproducible if in debug, you start opening others app until WatchDog tries to close yours.

This crash is from our crash tracker tool, crashlytics.

In method “TutorialEvent_OnDestroy” we only desubscribe from an event, but this couldn’t be the error, this line is executed everytime we change scene.

very strange

Thanks for the information. Can you submit a bug report with a project that causes this crash? That will give us the best chance to correct it.

Is there an update for this issue? We are having the exact same problem.

Well have you tried the lasted version of Unity to see if the problem still exists?
4.6.6 even patch releases, considering that each new release has had a tone of fixes for iOS 64bit

@Mwumwu

We’ve not yet corrected this issue, as we don’t have a bug report for it yet with a reproducible case. Can you submit a bug report and include a project that causes this? If so, we would love to have a look so that we can correct it. Thanks.

Looking at my crashes in XCode from my live app, I’m seeing a VERY similar crash stack in my app as well. I’ve been unable to repro it locally so far. Like the OP, this looks to be triggered by desubscribing to an event, but in my case, I’m desubscribing in OnDisable (not OnDestroy like the OP).

My app was built with Unity 4.6.6p1

@Callabrator88

Thanks for the information. If you are able to reproduce it locally, please let us know, and we will investigate it.

Just to let you know, our latest game update built on 4.6.6p4 still have this issue. But no STR on our side either.

I have soft launched a game a few days ago, with about 100 users right now. We got 9 crashes, apparently from the same device / user, all apparently starting after the game is killed because of memory pressure.

8 of the crashes have a very similar stack, of an exception triggered during GameObject destroy that calls our Log Handler which then tries to allocate memory (and then fail) while reporting the issue to our server.

And here’s the remaining one again at cleanup time (OnDestroy → save → Json Serialization → allocation → crash )

App is IL2CPP built 4.6.5f1 based. Device exhibiting issue is running IOS 8.3.

@jerome-lacoste

Do you know if it is possible that the app is close to the maximum memory available, and might actually be crashing because it runs out of memory?

@JoshPeterson the affected device is an ipad Air (A1474). It has double the amount of RAM as my test device (ipad2) and I have never managed to crash the app on the ipad2 in normal use.

I will try further with swapping the app out etc.

In all cases, it looks like I should be able to take some preventive measures by reducing the amount of work that happens during this shutdown phase. The OnApplicationQuit() seems to be sent too late (Unity - Manual: Order of execution for event functions). Is there a way for me to programatically check that Unity is shutting down ? Maybe Application.isPlaying is false ?

@jerome-lacoste

That probably indicates it is not actually running out of memory so there is likely a bug somewhere here that we will need to track down. If you are able to get to this something that is reproducible (even intermittently), we would love to have a bug report on it.

The OnApplicationQuit event is usually the place to check, but since it is too late, you might be able to use the AppDomain.DomainUnload event. I’m unsure if that event is actually before or after OnApplicationQuit though. It might be worth a try at least.

@jerome-lacoste

It looks like OnApplicationPause (Unity - Scripting API: MonoBehaviour.OnApplicationPause(bool)) might be a possibility as well, if you have not already tried it.

The first thing I noticed was the stack

  1. UnityCleanup()
  2. triggers object Destroy ?
  3. something goes wrong, Exception triggered
  4. Log Handler kicks in (the one from the Perf module)
  5. because of its implementation, it allocates objects before starting the coroutine
  6. crash (probably because memory constrained so allocation fail ?!)

I have been trying to create a reproducible case in the past few hours. I managed to create some oddities but no crash. Without knowing more of what’s happening inside the hood, I cannot turn thee symptoms as bugs to report yet.

I have 2 issues:

  • I didnt’ manage to get the UnityCleanup() call to be triggered upon memory pressure. At least I don’t know where to place my breakpoints in order to see it happening. If I put a breakpoint on a OnDestroy() somewhere, then if the app if killed while in the background or foreground because of memory pressure, calls to OnDestroy() don’t seem to happen. So my point #2 is maybe incorrectly understood:

What does the above stack mean ? Does this mean that the Destroy() method of a GameObject threw an exception ? Or is the exception coming internally form Unity ?

  • Any tip on how to trigger a UnityCleanup() call ?

@jerome-lacoste

I’ve been talking with members of our iOS team about this as well, and we have a few suggestions.

First, you can call UnityCleanup directly from managed code if you want to do so to test something. It is exposed in UnityInterface.h.

The section of the stack trace you mentioned does indeed indicate the Destroy is throwing a managed exception, then the LogException call ends up calling LogCallbackImplementation, which in turn attempts to submit something via WWW. I wonder if you can track down the actual exception in Destroy by sending this user a version which does not submit via WWW, but instead writes something to disk that can be retrieved later.

Also, is there some chance that the user is manually killing the app? Or iOS team suggested that might be the cause.

@JoshPeterson

I’ll investigate some more, although travelling all day tomorrow, then partially in holidays.

Some thoughts:

  • I will try calling UnityCleanup
  • I will try killing the app manually
  • I don’t know who the user is, nor can I send a special build right now. The LogCallbackImplementation is yours :slight_smile: (the Perf service one).

I will probably try modifying the UnityAppController.mm so that I can track the applicationWillTerminate() callback. That way I shut down the LogCallbackImplementation when this Unity is shutting down.

@jerome-lacoste

Ahh, thanks, obviously I don’t know everything going on in the Unity code :).

I think your approach sounds good, please let me know what you determine, thanks!

Seemingly same issue here. On 4.6.9f1, Xcode 7, IL2PCC, happening on wide range of iOS devices, iPhone / iPad, 32 / 64 bit and OS versions 8.4-9.1.

Will investigate further soon but here’s trace:

1 libsystem_kernel.dylib __pthread_kill + 8
2 libsystem_pthread.dylib pthread_kill + 108
3 libsystem_c.dylib abort + 136
4 [APPNAME] GC_stop_world + 172
5 [APPNAME] GC_stopped_mark + 80
6 [APPNAME] GC_try_to_collect_inner + 292
7 [APPNAME] GC_collect_or_expand + 216
8 [APPNAME] GC_allocobj + 228
9 [APPNAME] GC_generic_malloc_inner + 496
10 [APPNAME] GC_generic_malloc + 112
11 [APPNAME] GC_malloc + 232
12 [APPNAME] il2cpp::vm::Object::Clone(Il2CppObject*) + 40
13 [APPNAME] MulticastDelegate_RemoveImpl_m29023 (GeneratedVirtualInvokers.h:33)
14 [APPNAME] UIAnchor_OnDestroy_m3555 (Bulk_Assembly-CSharp_2.cpp:1994)
15 [APPNAME] RuntimeInvoker_Void_t2283(MethodInfo const*, void*, void**) (Il2CppInvokerTable.cpp:828)
16 [APPNAME] il2cpp::vm::Runtime::Invoke(MethodInfo const*, void*, void**, Il2CppObject**) + 64
17 [APPNAME] ScriptingInvocation::Invoke(ScriptingException**, bool) + 52
18 [APPNAME] ScriptingInvocationNoArgs::InvokeChecked() + 56
19 [APPNAME] MonoBehaviour::CallMethodInactive(ScriptingMethodIl2Cpp) + 72
20 [APPNAME] Unity::GameObject::WillDestroyGameObject() + 80
21 [APPNAME] PreDestroyRecursive(Unity::GameObject&, unsigned long*) + 112
22 [APPNAME] PreDestroyRecursive(Unity::GameObject&, unsigned long*) + 244
23 [APPNAME] PreDestroyRecursive(Unity::GameObject&, unsigned long*) + 244
24 [APPNAME] PreDestroyRecursive(Unity::GameObject&, unsigned long*) + 244
25 [APPNAME] PreDestroyRecursive(Unity::GameObject&, unsigned long*) + 244
26 [APPNAME] PreDestroyRecursive(Unity::GameObject&, unsigned long*) + 244
27 [APPNAME] PreDestroyRecursive(Unity::GameObject&, unsigned long*) + 244
28 [APPNAME] PreDestroyRecursive(Unity::GameObject&, unsigned long*) + 244
29 [APPNAME] DestroyGameObjectHierarchy(Unity::GameObject&) + 40
30 [APPNAME] DestroyObjectHighLevel(Object*, bool) + 896
31 [APPNAME] CleanupAllObjects(bool) + 276
32 [APPNAME] CleanupEngine() + 40
33 [APPNAME] PlayerCleanup(bool, bool) + 60
34 [APPNAME] UnityCleanup + 16
35 UIKit -[UIApplication _terminateWithStatus:] + 276
36 UIKit __102-[UIApplication _handleApplicationDeactivationWithScene:shouldForceExit:transitionContext:completion:]_block_invoke1976 + 312
37 UIKit _runAfterCACommitDeferredBlocks + 288
38 UIKit _cleanUpAfterCAFlushAndRunDeferredBlocks + 88
39 UIKit _afterCACommitHandler + 92
40 CoreFoundation CFRUNLOOP_IS_CALLING_OUT_TO_AN_OBSERVER_CALLBACK_FUNCTION + 28
41 CoreFoundation __CFRunLoopDoObservers + 368
42 CoreFoundation __CFRunLoopRun + 924
43 CoreFoundation CFRunLoopRunSpecific + 380
44 GraphicsServices GSEventRunModal + 176
45 UIKit UIApplicationMain + 200
46 [APPNAME] main (main.mm:37)
47 libdyld.dylib start + 0