Huge jump on tgkill (signal 6) crashes after moving to IL2CPP

Unity Engine
,
1

We have seen about 500% increase on this crash after moving from 2017.4 to 2018.4 and switching using IL2CPP. Unfortunately stack trace doesn’t tell much.

Pre-IL2CPP crash that occurred rarely:

Caused by java.lang.Error: signal 6 (SIGABRT), code -6 (?), fault addr --------
Build fingerprint: 'samsung/j6ltedx/j6lte:8.0.0/R16NW/J600GDXU3ARK5:user/release-keys'
Revision: '2'
pid: 8907, tid: 9048, name: UnityMain  >>> net.test.text <<<
    r0 00000000  r1 00002358  r2 00000006  r3 00000008
    r4 000022cb  r5 00002358  r6 cb97ec80  r7 0000010c
    r8 00000056  r9 cd8d6e00  sl cb97ee90  fp cb97ecf4
    ip 00000000  sp cb97ec70  lr f59e44d7  pc f5a1580c  cpsr 00000056

       at libc.tgkill + 12(tgkill:12)
       at libc.abort + 54(abort:54)
       at libc.__libc_fatal + 24(__libc_fatal:24)
       at libc.__pthread_internal_find(long) + 88(__pthread_internal_find:88)
       at libc.pthread_gettid_np + 2(pthread_gettid_np:2)
       at libc.pthread_kill + 18(pthread_kill:18)
       at libmono.0023fef0()
       at libmono.mono_thread_suspend_all_other_threads + 1424(mono_thread_suspend_all_other_threads:1424)
       at libmono.mono_unity_jit_cleanup + 28(mono_unity_jit_cleanup:28)
       at libunity.0072fba0()
       at libunity.00705764()
       at libunity.002df31c()
       at libunity.002e2148()
       at base.00116113()

Post IL2CPP tgkill crashes that increased by a huge margin:

Caused by java.lang.Error: signal 6 (SIGABRT), code -6 (?), fault addr --------
Build fingerprint: 'samsung/grandppltedx/grandpplte:6.0.1/MMB29T/G532GDXU1ASA5:user/release-keys'
Revision: '5'
pid: 3179, tid: 3466, name: Thread-6859  >>> net.test.text <<<
    r0 00000000  r1 00000d8a  r2 00000006  r3 833ae978
    r4 833ae980  r5 833ae930  r6 00000000  r7 0000010c
    r8 000004d6  r9 b433de44  sl 00000008  fp b4323820
    ip 00000006  sp 833ae7d0  lr b6cdfde5  pc b6ce21d4  cpsr 833ae4e0

       at libc.tgkill + 12(tgkill:12)
       at libc.pthread_kill + 32(pthread_kill:32)
       at libc.raise + 10(raise:10)
       at libc.__libc_android_abort + 34(__libc_android_abort:34)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)

Any ideas?

2

Something seems wrong about this call stack. Nothing in libmono should be running for an IL2CPP build. Are you sure this is from an IL2CPP build of the app?

3

My bad, the first log was from pre-IL2CPP version, it was happening rarely back then, but the second one is the one that increased this crash report by 500% after IL2CPP swap, it’s the one I’m concerned about right now.

4

Oops, sorry, I did not notice that before. I wonder, is it possible ti symbolicate the send call stack? It looks like something is wrong there, as all of those frames are probably not in abort.

5

I used to do some symbolication on Mono stacks but I don’t know how to symbolicate IL2CPP stacks, would you guide me to it?

6

This article is a bit older, but I believe that it still applies: https://support.unity3d.com/hc/en-us/articles/115000292166-Symbolicate-Android-crash

1 Like
7

I have used that article to symbolicate mono crashes before, but it doesn’t tell about IL2CPP symbolication sadly.

I have tried

ndk-stack -sym /Applications/Unity/Hub/Editor/2018.4.3f1/PlaybackEngines/AndroidPlayer/Variations/ -dump myCrashDump.txt

but didn’t work.

8

Hey,

out of curiosity, the logs you’re showing, where are they from? Because it seems like there was already an attempt to symbolicate the stacktrace, and it didn’t go well.

I was hopping to see a format like this

2019-05-17 12:00:58.823 30759-30803/? E/CRASH: signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 00000094
2019-05-17 12:00:58.823 30759-30803/? E/CRASH: *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
2019-05-17 12:00:58.823 30759-30803/? E/CRASH: Build type 'Release', Scripting Backend 'mono', CPU 'armeabi-v7a'
2019-05-17 12:00:58.823 30759-30803/? E/CRASH: Build fingerprint: 'OnePlus/OnePlus3/OnePlus3:8.0.0/OPR1.170623.032/1812060016:user/release-keys'
2019-05-17 12:00:58.823 30759-30803/? E/CRASH: Revision: '0'
2019-05-17 12:00:58.823 30759-30803/? E/CRASH: pid: 30759, tid: 30803, name: Thread-88  >>> .............. <<<
2019-05-17 12:00:58.823 30759-30803/? E/CRASH:     r0 00000000  r1 ce3da200  r2 f3f00040  r3 00000080
2019-05-17 12:00:58.823 30759-30803/? E/CRASH:     r4 ca45ed50  r5 ca45e7f8  r6 ca45e7f8  r7 ca45ecf8
2019-05-17 12:00:58.823 30759-30803/? E/CRASH:     r8 dd8bca10  r9 dd8bca1c  sl ca45e918  fp dd8bca18
2019-05-17 12:00:58.823 30759-30803/? E/CRASH:     ip f28efd5c  sp ca45e7f0  lr cd86524c  pc cd5633fc  cpsr 00007853
2019-05-17 12:00:58.823 30759-30803/? E/CRASH: backtrace:
2019-05-17 12:00:58.830 30759-30803/? E/CRASH:     #00  pc 002983fc  /data/app/......../lib/arm/libunity.so
2019-05-17 12:00:58.831 30759-30803/? E/CRASH:     #01  pc 002b3b3c  /data/app/......../lib/arm/libunity.so
2019-05-17 12:00:58.831 30759-30803/? E/CRASH:     #02  pc 002b6ee4  /data/app/......../lib/arm/libunity.so
2019-05-17 12:00:58.831 30759-30803/? E/CRASH:     #03  pc 002b6b00  /data/app/......../lib/arm/libunity.so
2019-05-17 12:00:58.831 30759-30803/? E/CRASH:     #04  pc 002b6a40  /data/app/......../lib/arm/libunity.so
2019-05-17 12:00:58.831 30759-30803/? E/CRASH:     #05  pc 002b6750  /data/app/......../lib/arm/libunity.so
2019-05-17 12:00:58.831 30759-30803/? E/CRASH:     #06  pc 0029a280  /data/app/......../lib/arm/libunity.so
2019-05-17 12:00:58.831 30759-30803/? E/CRASH:     #07  pc 00a01a60  /data/app/......../lib/arm/libunity.so
2019-05-17 12:00:58.831 30759-30803/? E/CRASH:     #08  pc 0064abe0  /data/app/......../lib/arm/libunity.so
2019-05-17 12:00:58.831 30759-30803/? E/CRASH:     #09  pc b3078ea4  <unknown/absolute>
9

Hi,

Both logs are from the Fabric Crashlytics, there is nothing we do manually to enhance/symbolicate as far as I know.

Honestly, I have no idea what would have alter these crash dumps, are you sure they are not what they are supposed to be like?

10

May that’s how Fabric Crashlytics prints them, if you would try to crash your app locally, and check the logcat, the information should look like the one I posted above.

Maybe you have a logcat from your crash ?

11

Unfortunately I couldn’t manage to reproduce this issue in any way so I don’t have access to any logcat details.

12

Can you maybe somehow instruct Fabric Crashlytics to give you raw stacktrace?

13

Maybe I can find a way to attach the logcats to exceptions that are caught by Fabric, I will look into that.

Meanwhile this attached log is all the information it provides to us about an occurrence of this crash. Maybe that helps you somehow?

4805321–459719–stacktrace.txt (48.8 KB)

14 
       at libc.tgkill + 12(tgkill:12)
       at libc.pthread_kill + 32(pthread_kill:32)
       at libc.raise + 10(raise:10)
       at libc.__libc_android_abort + 34(__libc_android_abort:34)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)
       at libc.abort + 4(abort:4)

looks incorrect, either it’s a memory corruption, or the stacktrace was resolved incorrectly

15

Terrible news, so we have actually nothing to help us to resolve this at all. What else would you suggest me to do to get some information about this crash?

16

The crash says, it crashes on Samsung Galaxy J6 with Android 8.0

Did you try testing on that specific phone?

17

Yeah, I have tried it on J6 and J7 but sadly couldn’t produce the problem. This problem is not specific to those phones either, see the screenshots attached.

4805816--459815--Screen Shot 2019-07-31 at 1.47.30 PM.png
4805816--459818--Screen Shot 2019-07-31 at 1.47.42 PM.png

18

I see, that’s too bad…that makes things very difficult to fix. The only thing I can suggest, is to try to update to latest 2018 LTS, maybe it will contain stability fix to your problem.

19

Thanks for your time anyway, I couldn’t find anything related to this issue on 2018.4 and 2018.5 releases and to be honest kind of scared to upgrade, so far 2018 LTS upgrades introduced nothing but new crashes/bugs for us, but I guess I’ll give it a try.

20

I’m finally able to reproduce the problem, and did it on development mode however log is exactly the same.

08-09 14:47:46.956 14765 15279 F art     : art/runtime/thread.cc:1238] Native thread exited without calling DetachCurrentThread: Thread[26,tid=15279,Native,Thread*=0x6539c200,peer=0x138d20a0,"Thread-12925"]
08-09 14:47:46.956 14765 15279 F art     : art/runtime/runtime.cc:368] Runtime aborting...
08-09 14:47:46.956 14765 15279 F art     : art/runtime/runtime.cc:368]
08-09 14:47:46.956 14765 15279 E CRASH   : signal 6 (SIGABRT), code -6 (?), fault addr --------
********** Crash dump: **********
Build fingerprint: 'samsung/kltejv/klte:6.0.1/MMB29M/G900FQJVS1CSB1:user/release-keys'
pid: 14765, tid: 15279, name: Thread-12925  >>> my.awesome.app <<<
Stack frame #00  pc 00041ff8  /system/lib/libc.so (tgkill+12)
Stack frame #01  pc 0003fc05  /system/lib/libc.so (pthread_kill+32)
Stack frame #02  pc 0001c38b  /system/lib/libc.so (raise+10)
Stack frame #03  pc 00019609  /system/lib/libc.so (__libc_android_abort+34)
Stack frame #04  pc 0001755c  /system/lib/libc.so (abort+4)
Stack frame #05  pc 0001755c  /system/lib/libc.so (abort+4)
Stack frame #06  pc 0001755c  /system/lib/libc.so (abort+4)
Stack frame #07  pc 0001755c  /system/lib/libc.so (abort+4)
Stack frame #08  pc 0001755c  /system/lib/libc.so (abort+4)
Stack frame #09  pc 0001755c  /system/lib/libc.so (abort+4)
Stack frame #10  pc 0001755c  /system/lib/libc.so (abort+4)
Stack frame #11  pc 0001755c  /system/lib/libc.so (abort+4)
Stack frame #12  pc 0001755c  /system/lib/libc.so (abort+4)
Stack frame #13  pc 0001755c  /system/lib/libc.so (abort+4)
Stack frame #14  pc 0001755c  /system/lib/libc.so (abort+4)
Stack frame #15  pc 0001755c  /system/lib/libc.so (abort+4)
Stack frame #16  pc 0001755c  /system/lib/libc.so (abort+4)
Stack frame #17  pc 0001755c  /system/lib/libc.so (abort+4)
Stack frame #18  pc 0001755c  /system/lib/libc.so (abort+4)
Stack frame #19  pc 0001755c  /system/lib/libc.so (abort+4)
Stack frame #20  pc 0001755c  /system/lib/libc.so (abort+4)
Stack frame #21  pc 0001755c  /system/lib/libc.so (abort+4)
Stack frame #22  pc 0001755c  /system/lib/libc.so (abort+4)
Stack frame #23  pc 0001755c  /system/lib/libc.so (abort+4)
Stack frame #24  pc 0001755c  /system/lib/libc.so (abort+4)
Stack frame #25  pc 0001755c  /system/lib/libc.so (abort+4)
Stack frame #26  pc 0001755c  /system/lib/libc.so (abort+4)
Stack frame #27  pc 0001755c  /system/lib/libc.so (abort+4)
Stack frame #28  pc 0001755c  /system/lib/libc.so (abort+4)
Stack frame #29  pc 0001755c  /system/lib/libc.so (abort+4)
Stack frame #30  pc 0001755c  /system/lib/libc.so (abort+4)
Stack frame #31  pc 0001755c  /system/lib/libc.so (abort+4)

Seems like a threading problem, but no idea how to progress based on this log, it actually says not much besides the new “Native thread exited without calling DetachCurrentThread:” part.

I also have bunch load of;

08-09 14:49:18.656 14765 14765 E CRASH   : other thread is trapped; signum = 6
08-09 14:49:18.656 14765 14765 E CRASH   : main thread is trapped; signum = 6
08-09 14:49:18.656 14765 14765 E CRASH   : other thread is trapped; signum = 6
08-09 14:49:18.656 14765 14765 E CRASH   : main thread is trapped; signum = 6
08-09 14:49:18.656 14765 14765 E CRASH   : other thread is trapped; signum = 6
08-09 14:49:18.666 14765 14765 E CRASH   : main thread is trapped; signum = 6
08-09 14:49:18.666 14765 14765 E CRASH   : other thread is trapped; signum = 6
08-09 14:49:18.666 14765 14765 E CRASH   : main thread is trapped; signum = 6
08-09 14:49:18.666 14765 14765 E CRASH   : other thread is trapped; signum = 6
08-09 14:49:18.666 14765 14765 E CRASH   : main thread is trapped; signum = 6
08-09 14:49:18.666 14765 14765 E CRASH   : other thread is trapped; signum = 6

How I can debug what’s going wrong during runtime from this, any ideas?