Making sense of an il2cpp crash

Unity Engine
,
1

Hello,

Our app’s most frequent crash has a rather cryptic stack trace, as follows…

CrashedCheckBellowForHintsWhy() (CrashReporter.mm:97)
UnhandledExceptionEventHandler_Invoke_m689053609 (Bulk_mscorlib_8.cpp:9214)
RuntimeInvoker_Void_t2779279689_Il2CppObject_Il2CppObject(MethodInfo const*, void*, void**) (Il2CppInvokerTable.cpp:951)
il2cpp::vm::Runtime::CallUnhandledExceptionDelegate(Il2CppDomain*, Il2CppDelegate*, Il2CppObject*) (Runtime.cpp:350)
il2cpp::vm::Runtime::UnhandledException(Il2CppObject*) (Runtime.cpp:433)
ScriptingInvocation::Invoke(ScriptingException**, bool) (ScriptingInvocation.cpp:202)
MonoBehaviour::InvokeMethodOrCoroutineChecked(ScriptingMethodIl2Cpp, ScriptingObject*, ScriptingException**) (MonoBehaviour.cpp:839)
MonoBehaviour::InvokeMethodOrCoroutineChecked(ScriptingMethodIl2Cpp, ScriptingObject*) (MonoBehaviour.cpp:859)
Scripting::SendScriptingMessage(Unity::GameObject&, char const*, ScriptingObject*) (Scripting.cpp:254)
WebScripting::ProcessSendMessages() (WebScripting.cpp:286)
PlayerLoop(bool, bool, IHookEvent*) (Player.cpp:1833)
UnityPlayerLoopImpl(bool) (LibEntryPoint.mm:239)
UnityRepaint (UnityAppController+Rendering.mm:236)
-[UnityAppController(Rendering) repaintDisplayLink] (UnityAppController+Rendering.mm:52)
CA::smile:isplay::smile:isplayLinkItem::dispatch() + 36
CA::smile:isplay::smile:isplayLink::dispatch_items(unsigned long long, unsigned long long, unsigned long long) + 420
IODispatchCalloutFromCFMessage + 368
__CFMachPortPerform + 176
CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE1_PERFORM_FUNCTION + 52
__CFRunLoopDoSource1 + 432
__CFRunLoopRun + 1796
CFRunLoopRunSpecific + 380
GSEventRunModal + 176
UIApplicationMain + 200
main (main.mm:32)
0x0 + 0

I’m assuming that “t2779279689” is the name of a C# type that has been converted to C++. However, I have no idea how I can link the C++ type back to the original C# type, or discover the offending method call.

If it matters, this crash is reported by HockeyApp. It appears to be sporadic, and I have not seen it occur while I’ve been attached to Xcode (meaning it’s hard for me to get much more information).

Does anyone have any suggestions on where I could look to wring more information out of this stack trace?

Another note, the crash above was from a build running Unity 5.2.3p4. I’ve also encountered what I believe is the same crash on 5.2.4f1. The only line that’s different is the third line, and it’s…

RuntimeInvoker_Void_t_224166001_0_Object_t_Object_t(MethodInfo const*, void*, void**) (Il2CppInvokerTable.cpp:945)

Thanks for any help you can provide!

2

@BJO

Sure, I can point out a few things. First, “t2779279689” means “type 2779279689” where type is a hash we generated based on the fully qualified type name in managed code. This should not be too important though, as we reserve the right to change the way we name things in generated C++ code in the future. However, you can always map a given function or type back to its managed equivalent by using the generated C++ code.

In the Xcode project, you can search for any method or type (say UnhandledExceptionEventHandler_Invoke_m689053609 in this case). The implementation of that method in C++ will have a comment just above it that listed to full managed name for that method.

The method RuntimeInvoker_Void_t2779279689_Il2CppObject_Il2CppObject happens to not be generated from managed code, but instead is a generated helper method that we use. So you won’t be able to track down where that method is in C#, for example.

That is all pretty general information. As to this specific case, note that the call stack you see here is not the call stack where the crash happened. On iOS, if the “Script Call Optimization” option in the player settings has a value of “Fast but no Exceptions” then any exception in managed code that escapes out of the project’s script code and back to the engine will cause exactly what you see here.

Unity will trigger the AppDomain.UnhandledException event. You can register a specific handler for this event, but Unity registers a general handler for it. This handler logs the information about the call stack from the exception for the Unity CrashReport API, then it terminates the application. So you could use that CrashReport API or you could register your own handler for the UnhandledException event and log the call stack elsewhere.

However, I thought that HockeyApp actually does register this event handler and record the actual manage call stack for the exception. Maybe there is a different configuration option in HockeyApp to enable this, I’m not sure.

Also, the difference in the third line between Unity versions probably does not matter. We’ve just changed the way we name things in generated C++ code.

1 Like
3

@JoshPeterson we have a similar problem with an update we released last week. We’re finding a lot of devices are having this crash but we have no idea what’s causing it.

What can we do as developers in general?

Due to this I’ve had a quick look at the Crash Reporting API. If that is implemented would it give better information than this?

I could not seem to get it working at all though. Perhaps you can respond to my post here.

I also see that the Performance Reporting tool is integrated with Unity 5.4, so once we upgrade I will enable it.

2745498--197916--Screen Shot 2016-08-09 at 12.01.02.png
2745498--197916--Screen Shot 2016-08-09 at 12.01.02.png805×432 54.9 KB

4

@andymads

I’ve not used the Unity Crash Reporting API myself, so I can’t offer too much practical advice on it. Make sure you select the “Enable CrashReport API” option in the iOS player settings as described here: https://docs.unity3d.com/Manual/class-PlayerSettingsiOS.html. I’ve pinged our iOS team to respond to the other thread you posted about how to use the crash reporting API.

You may also want to try to use the AppDomain.UnhandledException event as mentioned above. That should give you a managed code stack trace which might help locate the cause of the issue. Also, is this something you can reproduce locally? If so, catching the crash in Xcode is probably best. You can use an exception break point in Xcode to track down the cause of the issue if it is a C++ or managed exception (See this post for details: http://blogs.unity3d.com/2015/05/20/il2cpp-internals-debugging-tips-for-generated-code/).

Finally, if this is not occurring locally, you can use a number of different crash reporting systems to help track down the cause of crashes on end user devices. Most of them work with Unity 5.3 and earlier. I’ve not used them myself, but I’ve heard of good results others have had.

5

@JoshPeterson

Thanks for the tips.

Yes, I did enable CrashReport API.

Any idea how to actually force a crash so I can test the CrashReporting API and AppDomain.UnhandledException?

We cannot reproduce locally, it’s end-user devices, except it’s been seen on one device at our other office. I had the crash logs from that device sent to me but they only give me the callstack that I already posted above.

6

@andymads

You can throw a managed exception from your script code. That should trigger the UnhandledException event for testing.

7

@andymads

I just replied on your other thread, but we have a custom preview build that will allow you to capture native crashes like this and see them in our Game Performance dashboard. It will capture both the native stack trace and the managed stack that the exception originated from. Please let me know if you’re interested in giving it a try.

You get a report that looks like this (notice how similar the native stack is to yours):

8

Can you give an example?

9

@andymads

Sure. You can throw a managed exception from the Start method, for example:

public class Throws : MonoBehaviour
{
    void Start()
    {
       throw new System.Exception("Test exception.");
    }
}
10

That’s exactly what I was doing, however I was doing it from a button OnClick handler. Now I moved it to Start it worked.

However, now I’ve simulated a couple of crashes, the CrashReporting API is itself crashing I think, when I try to use it.

This is using Mono and not IL2CPP btw.

* Assertion at aot-runtime.c:1745, condition `!is_wrapper’ not met
2745923--197932--Screen Shot 2016-08-09 at 17.20.49.png

11

This is shutting the gate after the horse has bolted for us, but I would be interested in getting this stuff into our future builds. Presumably though it’s Unity 5.4 only?

12

@andymads

This looks like a bug in the crash reporting API, as you mentioned! Can you get anything useful from it with IL2CPP instead?

13

Yes, works ok with IL2CPP, but couldn’t get the AppDomain.UnhandledException event to work.

14

We currently have preview builds based off of 5.3.5p8 and 5.4.0f3. If you can reproduce the crash in-house, these would give you the managed stack trace. Otherwise, you’d have to ship a new build of your game using the preview build.

15

I think the problem there might be that the crash report API is also registered for AppDomain.UnhandledException, and it gets the callback first and triggers the crash, so your callback handler never gets called. I’m speculating a bit on that though. :slight_smile:

16

That’s ok. I was just testing really to see what’s possible. Am interested in using the Game Performance dashboard going forward if it will give us better info on what’s going wrong in end-user builds. I’m currently on 5.3.6, you don’t have a preview build for that?

17

That build is a couple weeks old and 5.3.6 wasn’t out yet when that build was made. I can update it and spin up a new build if you’re sure you want to try it. Would you want 5.3.6f1 or 5.3.6p2?

18

Don’t do anything yet, I need to speak to my boss. I’ll come back to you.

19

@thomasn_unity ok can we please get the preview build for 5.3.6f1. We are going to update our app next week and it would be good to try the Game Performance functionality out with that.

What do I need to do?

20

@JoshPeterson First off, thanks for your original reply to this post. I’ve been meaning to thank you for you response since, well, since you originally posted it.

@andymads Skimming through this discussion, it seems like your questions might have diverged from my original question. However, in case it’s helpful, here’s the conclusion I came to…

As the second line alludes to, “UnhandledExceptionEventHandler”, this crash seems to be the result of a managed exception being thrown that is never caught. What was confusing me is that in this situation, two crash reports were generated. One report would have the stack trace above. The other would have a meaningful c# stack trace. What was even more confusing is that not all c#-originating crashes seems to trigger the stack trace posted above. Ultimately, I concluded that crash reports like the one above can be ignored, as they always correspond to a second, meaningful report. One potentially important footnote is that HockeyApp is used for our crash reporting. I have not tried Unity’s performance reporting.