Unity 2017.1 - TLS 1.2 still not working with .NET 4.6 (673677)

Unity Engine
41

Hi all
I have exact problem this is my log
Unity3d 2018.2.4f1, Mono .NET 4x .NET Standart 2.0, Android 4.4.2 GT-N7100, custom WebSocket with System.Net.Security.SslStream

08-21 13:24:39.466: E/dalvikvm(23605): VM aborting
08-21 13:24:39.466: E/CRASH(23605): signal 6 (SIGABRT), code -6 (?), fault addr --------
08-21 13:24:39.466: E/CRASH(23605): *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
08-21 13:24:39.466: E/CRASH(23605): Build fingerprint: 'samsung/t03gxx/t03g:4.4.2/KOT49H/N7100XXUFNE1:user/release-keys'
08-21 13:24:39.466: E/CRASH(23605): Revision: '11'
08-21 13:24:39.466: E/CRASH(23605): pid: 23605, tid: 23672, name: Thread Pool Wor  >>> com.project.app <<<
08-21 13:24:39.466: E/CRASH(23605):     r0 00000000  r1 00005c78  r2 00000006  r3 00000000
08-21 13:24:39.466: E/CRASH(23605):     r4 00000006  r5 00000016  r6 00005c78  r7 0000010c
08-21 13:24:39.466: E/CRASH(23605):     r8 70489640  r9 70489658  sl 7048963c  fp 704895e8
08-21 13:24:39.466: E/CRASH(23605):     ip 70489638  sp 70489308  lr 4019f10d  pc 401ae18c  cpsr 70488f98
08-21 13:24:39.466: E/CRASH(23605): backtrace:
08-21 13:24:39.576: E/CRASH(23605):     #00  pc 0002218c  /system/lib/libc.so (tgkill+12)
08-21 13:24:39.576: E/CRASH(23605):     #01  pc 00013109  /system/lib/libc.so (pthread_kill+48)
08-21 13:24:39.576: E/CRASH(23605):     #02  pc 0001331d  /system/lib/libc.so (raise+10)
08-21 13:24:39.576: E/CRASH(23605):     #03  pc 00012053  /system/lib/libc.so
08-21 13:24:39.576: E/CRASH(23605):     #04  pc 00021a40  /system/lib/libc.so (abort+4)
08-21 13:24:39.576: E/CRASH(23605):     #05  pc 00046aab  /system/lib/libdvm.so (dvmAbort+78)
08-21 13:24:39.576: E/CRASH(23605):     #06  pc 00055855  /system/lib/libdvm.so (dvmDetachCurrentThread()+548)
08-21 13:24:39.576: E/CRASH(23605):     #07  pc 00049e8b  /system/lib/libdvm.so
08-21 13:24:39.576: E/CRASH(23605):     #08  pc 0003abc1  /system/lib/libdvm.so
08-21 13:24:39.576: E/CRASH(23605):     #09  pc 00b72984  /data/app-lib/com.project.app-2/libunity.so
08-21 13:24:39.576: E/CRASH(23605):     #10  il 00000057  at (wrapper managed-to-native) object.wrapper_native_0x617b8830 (Mono.Unity.UnityTls/unitytls_x509list_ref,byte*,Mono.Unity.size_t,Mono.Unity.UnityTls/unitytls_x509verify_callback,void*,Mono.Unity.UnityTls/unitytls_errorstate*) <0x00057>
08-21 13:24:39.576: E/CRASH(23605):     #11  il 00000117  at (wrapper delegate-invoke) <Module>.invoke_UnityTls/unitytls_x509verify_result_UnityTls/unitytls_x509list_ref_byte*_size_t_UnityTls/unitytls_x509verify_callback_void*_UnityTls/unitytls_errorstate* (Mono.Unity.UnityTls/unitytls_x509list_ref,byte*,Mono.Unity.size_t,Mono.Unity.UnityTls/unitytls_x509verify_callback,void*,Mono.Unity.UnityTls/unitytls_errorstate*) <0x00117>
08-21 13:24:39.576: E/CRASH(23605):     #12  il 00000503  at Mono.Unity.UnityTlsProvider.ValidateCertificate (Mono.Security.Interface.ICertificateValidator2,string,bool,System.Security.Cryptography.X509Certificates.X509CertificateCollection,bool,System.Security.Cryptography.X509Certificates.X509Chain&,Mono.Security.Interface.MonoSslPolicyErrors&,int&) [0x00188] in <7505e1ded68144da95fe0bd520bab178>:0
08-21 13:24:39.576: E/CRASH(23605):     #13  il 0000030b  at Mono.Net.Security.ChainValidationHelper.ValidateChain (string,bool,System.Security.Cryptography.X509Certificates.X509Certificate,System.Security.Cryptography.X509Certificates.X509Chain&,System.Security.Cryptography.X509Certificates.X509CertificateCollection,System.Net.Security.SslPolicyErrors) [0x000e6] in <7505e1ded68144da95fe0bd520bab178>:0
08-21 13:24:39.576: E/CRASH(23605):     #14  il 00000083  at Mono.Net.Security.ChainValidationHelper.ValidateChain (string,bool,System.Security.Cryptography.X509Certificates.X509Certificate,System.Security.Cryptography.X509Certificates.X509Chain,System.Security.Cryptography.X509Certificates.X509CertificateCollection,System.Net.Security.SslPolicyErrors) [0x00009] in <7505e1ded68144da95fe0bd520bab178>:0
08-21 13:24:39.576: E/CRASH(23605):     #15  il 0000007b  at Mono.Net.Security.ChainValidationHelper.ValidateCertificate (string,bool,System.Security.Cryptography.X509Certificates.X509CertificateCollection) [0x00017] in <7505e1ded68144da95fe0bd520bab178>:0
08-21 13:24:39.576: E/CRASH(23605):     #16  il 00000033  at Mono.Net.Security.MobileTlsContext.ValidateCertificate (System.Security.Cryptography.X509Certificates.X509CertificateCollection) [0x00012] in <7505e1ded68144da95fe0bd520bab178>:0
08-21 13:24:39.576: E/CRASH(23605):     #17  il 0000008f  at Mono.Unity.UnityTlsContext.VerifyCallback (Mono.Unity.UnityTls/unitytls_x509list_ref,Mono.Unity.UnityTls/unitytls_errorstate*) [0x0001a] in <7505e1ded68144da95fe0bd520bab178>:0
08-21 13:24:39.576: E/CRASH(23605):     #18  il 00000083  at Mono.Unity.UnityTlsContext.VerifyCallback (void*,Mono.Unity.UnityTls/unitytls_x509list_ref,Mono.Unity.UnityTls/unitytls_errorstate*) [0x00013] in <7505e1ded68144da95fe0bd520bab178>:0
08-21 13:24:39.576: E/CRASH(23605):     #19  il 000000d7  at (wrapper native-to-managed) Mono.Unity.UnityTlsContext.VerifyCallback (void*,Mono.Unity.UnityTls/unitytls_x509list_ref,Mono.Unity.UnityTls/unitytls_errorstate*) <0x000d7>
08-21 13:24:39.576: E/CRASH(23605):     #20  il 0000003b  at (wrapper managed-to-native) object.wrapper_native_0x61062e14 (Mono.Unity.UnityTls/unitytls_tlsctx*,Mono.Unity.UnityTls/unitytls_errorstate*) <0x0003b>
08-21 13:24:39.576: E/CRASH(23605):     #21  il 00000097  at Mono.Unity.UnityTlsContext.ProcessHandshake () [0x0001c] in <7505e1ded68144da95fe0bd520bab178>:0
08-21 13:24:39.576: E/CRASH(23605):     #22  il 000000fb  at Mono.Net.Security.MobileAuthenticatedStream.ProcessHandshake (Mono.Net.Security.AsyncOperationStatus) [0x0003e] in <7505e1ded68144da95fe0bd520bab178>:0
08-21 13:24:39.576: E/CRASH(23605):     #23  il 00000073  at (wrapper remoting-invoke-with-check) Mono.Net.Security.MobileAuthenticatedStream.ProcessHandshake (Mono.Net.Security.AsyncOperationStatus) <0x00073>
08-21 13:24:39.576: E/CRASH(23605):     #24  il 00000027  at Mono.Net.Security.AsyncHandshakeRequest.Run (Mono.Net.Security.AsyncOperationStatus) [0x00006] in <7505e1ded68144da95fe0bd520bab178>:0
08-21 13:24:39.576: E/CRASH(23605):     #25  il 000002b7  at Mono.Net.Security.AsyncProtocolRequest/<ProcessOperation>d__24.MoveNext () [0x000ff] in <7505e1ded68144da95fe0bd520bab178>:0
08-21 13:24:39.576: E/CRASH(23605):     #26  il 00000087  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.InvokeMoveNext (object) [0x00000] in <dd5ad072bbb44eaa9c17089b0641164a>:0
08-21 13:24:39.576: E/CRASH(23605):     #27  il 0000019f  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00071] in <dd5ad072bbb44eaa9c17089b0641164a>:0
08-21 13:24:39.581: E/CRASH(23605):     #28  il 0000002b  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00000] in <dd5ad072bbb44eaa9c17089b0641164a>:0
08-21 13:24:39.581: E/CRASH(23605):     #29  il 000000ab  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.Run () [0x00024] in <dd5ad072bbb44eaa9c17089b0641164a>:0
08-21 13:24:39.581: E/CRASH(23605):     #30  il 0000007b  at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action,bool,System.Threading.Tasks.Task&) [0x0001c] in <dd5ad072bbb44eaa9c17089b0641164a>:0
08-21 13:24:39.581: E/CRASH(23605):     #31  il 00000127  at System.Threading.Tasks.Task.FinishContinuations () [0x00068] in <dd5ad072bbb44eaa9c17089b0641164a>:0
08-21 13:24:39.581: E/CRASH(23605):     #32  il 0000009f  at System.Threading.Tasks.Task.FinishStageThree () [0x0003c] in <dd5ad072bbb44eaa9c17089b0641164a>:0
08-21 13:24:39.581: E/CRASH(23605):     #33  il 000000f3  at System.Threading.Tasks.Task`1<System.Nullable`1<int>>.TrySetResult (System.Nullable`1<int>) [0x0004f] in <dd5ad072bbb44eaa9c17089b0641164a>:0
08-21 13:24:39.581: E/CRASH(23605):     #34  il 0000007f  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<System.Nullable`1<int>>.SetResult (System.Nullable`1<int>) [0x0003e] in <dd5ad072bbb44eaa9c17089b0641164a>:0
08-21 13:24:39.581: E/CRASH(23605):     #35  il 0000043f  at Mono.Net.Security.AsyncProtocolRequest/<InnerRead>d__25.MoveNext () [0x00169] in <7505e1ded68144da95fe0bd520bab178>:0
08-21 13:24:39.581: E/CRASH(23605):     #36  il 00000087  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.InvokeMoveNext (object) [0x00000] in <dd5ad072bbb44eaa9c17089b0641164a>:0
08-21 13:24:39.581: E/CRASH(23605):     #37  il 0000019f  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00071] in <dd5ad072bbb44eaa9c17089b0641164a>:0
08-21 13:24:39.581: E/CRASH(23605):     #38  il 0000002b  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00000] in <dd5ad072bbb44eaa9c17089b0641164a>:0
08-21 13:24:39.581: E/CRASH(23605):     #39  il 000000ab  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.Run () [0x00024] in <dd5ad072bbb44eaa9c17089b0641164a>:0
08-21 13:24:39.581: E/CRASH(23605):     #40  il 0000007b  at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action,bool,System.Threading.Tasks.Task&) [0x0001c] in <dd5ad072bbb44eaa9c17089b0641164a>:0
08-21 13:24:39.581: E/CRASH(23605):     #41  il 00000127  at System.Threading.Tasks.Task.FinishContinuations () [0x00068] in <dd5ad072bbb44eaa9c17089b0641164a>:0
08-21 13:24:39.581: E/CRASH(23605):     #42  il 0000009f  at System.Threading.Tasks.Task.FinishStageThree () [0x0003c] in <dd5ad072bbb44eaa9c17089b0641164a>:0
08-21 13:24:39.581: E/CRASH(23605):     #43  il 000000df  at System.Threading.Tasks.Task`1<int>.TrySetResult (int) [0x0004f] in <dd5ad072bbb44eaa9c17089b0641164a>:0
08-21 13:24:39.581: E/CRASH(23605):     #44  il 0000006f  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<int>.SetResult (int) [0x0003e] in <dd5ad072bbb44eaa9c17089b0641164a>:0
08-21 13:24:39.581: E/CRASH(23605):     #45  il 000006f3  at Mono.Net.Security.MobileAuthenticatedStream/<InnerRead>d__66.MoveNext () [0x00181] in <7505e1ded68144da95fe0bd520bab178>:0
08-21 13:24:39.581: E/CRASH(23605):     #46  il 00000087  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.InvokeMoveNext (object) [0x00000] in <dd5ad072bbb44eaa9c17089b0641164a>:0
08-21 13:24:39.581: E/CRASH(23605):     #47  il 0000019f  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00071] in <dd5ad072bbb44eaa9c17089b0641164a>:0
08-21 13:24:39.581: E/CRASH(23605):     #48  il 0000002b  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00000] in <dd5ad072bbb44eaa9c17089b0641164a>:0
08-21 13:24:39.581: E/CRASH(23605):     #49  il 000000ab  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.Run () [0x00024] in <dd5ad072bbb44eaa9c17089b0641164a>:0
08-21 13:24:39.581: E/CRASH(23605):     #50  il 0000007b  at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action,bool,System.Threading.Tasks.Task&) [0x0001c] in <dd5ad072bbb44eaa9c17089b0641164a>:0
08-21 13:24:39.581: E/CRASH(23605):     #51  il 00000127  at System.Threading.Tasks.Task.FinishContinuations () [0x00068] in <dd5ad072bbb44eaa9c17089b0641164a>:0
08-21 13:24:39.581: E/CRASH(23605):     #52  il 0000009f  at System.Threading.Tasks.Task.FinishStageThree () [0x0003c] in <dd5ad072bbb44eaa9c17089b0641164a>:0
08-21 13:24:39.581: E/CRASH(23605):     #53  il 00000173  at System.Threading.Tasks.Task.FinishStageTwo () [0x000d4] in <dd5ad072bbb44eaa9c17089b0641164a>:0
08-21 13:24:39.581: E/CRASH(23605):     #54  il 000000cf  at System.Threading.Tasks.Task.Finish (bool) [0x00036] in <dd5ad072bbb44eaa9c17089b0641164a>:0
08-21 13:24:39.581: E/CRASH(23605):     #55  il 0000014f  at System.Threading.Tasks.Task.ExecuteWithThreadLocal (System.Threading.Tasks.Task&) [0x00067] in <dd5ad072bbb44eaa9c17089b0641164a>:0
08-21 13:24:39.581: E/CRASH(23605):     #56  il 000000ef  at System.Threading.Tasks.Task.ExecuteEntry (bool) [0x00058] in <dd5ad072bbb44eaa9c17089b0641164a>:0
08-21 13:24:39.581: E/CRASH(23605):     #57  il 00000017  at System.Threading.Tasks.Task.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem () [0x00000] in <dd5ad072bbb44eaa9c17089b0641164a>:0
08-21 13:24:39.586: E/CRASH(23605):     #58  il 00000217  at System.Threading.ThreadPoolWorkQueue.Dispatch () [0x00074] in <dd5ad072bbb44eaa9c17089b0641164a>:0
08-21 13:24:39.586: E/CRASH(23605):     #59  il 00000037  at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback () [0x00000] in <dd5ad072bbb44eaa9c17089b0641164a>:0
08-21 13:24:39.586: E/CRASH(23605):     #60  il 0000006f  at (wrapper runtime-invoke) <Module>.runtime_invoke_bool (object,intptr,intptr,intptr) <0x0006f>
42

@andreasreich I already moved on from this issue. The problem does not happen when I use self-signed certificates. Only happens with Letā€™s Encrypt ones which is weird. My game unfortunately has another problem where during switching to .Net4.0, Unity crashes. For the sake of moving on, I am rolling with self-signed ones for now until further need. Hope new bit of info helps.

43

@EduardMatveev Thanks for reporting this issue! Somethingā€™s clearly amiss on our side here, there should never be a a crash in native with all of this. Can you please create a bug with a repro case (if possible) so I can look into it? Thank you!

44

@cryptoforge I hope you reported the .Net4 switching crash already :). Weā€™re trying to make .Net4 the new default asap and for a proper judgement on when weā€™re ready for it we need all the reports we can get.

Outside of .Net4 we donā€™t provide any guarantees on TLS support. Some things work, some donā€™t. You get whatever ā€œold monoā€ had in store for it and we never tested it thoroughly.

45

@andreasreich Case 1073395

1 Like
46

Thanks, Iā€™ll look at it asap!

47

@andreasreich Iā€™m excited that Iā€™m now able to connect and exchange data with SslStream using TLS 1.2, using version 2018.2.6f1 on Windows. However, I found an odd issue that others may not encounter. When the other end of an SslStream shuts down (orderly or not), ReadAsync never completes its operation and the task runs indefinitely. It should instead complete its Task and reading Result should return 0. In regular Mono 5.x SslStream does not seem to exhibit this problem (Windows, Linux). I have tried working around this to no avail. It is probably trivial to test, but I can put together a project and submit a bug report if youā€™d like.

I noticed this when doing my initial tests. Needing a simple TLS 1.2 server, I just used my web server and performed an HTTP/1.0 request. The server is expected to shut down the SSL connection in this case, and I was able to verify that the server did a shutdown and closed the socket. SslStream reads the response, but never handles the shutdown, and the underlying socket shows data available. Since there is no way to check what the underlying socket is receiving without disrupting the stream, and because a read is required on the socket to determine itā€™s closed, there is no reasonable workaround that I can find.

Thanks!

1 Like
48

@AlleywayDave thanks for very much for the feedback! Somebody else already reported this bug, it is going to be fixed in 2018.2.7 (missed 2018.2.6 just by a bit :()
Would be great if you could confirm the fix once it is out, just in case

49

Actually I did have a lot of trouble creating a minimal repro case that would hit the bug reliably (in the report it happened randomly). You sound like you were able to get it reliably, Iā€™d really like to have this as a regression test in Unity. If it is no trouble for you, could you give me code/a project to that?

1 Like
50

@andreasreich Yes, Iā€™ve put one together, but I will make a few improvements to the code and submit it.

51

@andreasreich Case 1076788

Thanks!

1 Like
52

Thank you! Turns out this is a different issue than the one I fixed. So far it looks like an issue in Mono where reads on a closed NetworkStream block, but Iā€™m not done investigating yet.

53

Shame on me for blaming Mono so quickly! Was a mistake on our side in the Mono integration of the Tls backend. Fix is on the way.

1 Like
54

I think I have stumbled across the same problem as @EduardMatveev . During my tests, the crash only occured on androids 4.4 and below. Androids 5 and above work as expected.

I tried searching for the case number to see what the current status was but was unable anything. Could anyone point me to it or update me on the current status?

Thanks!

55

There was some delay on this one since the issue doesnā€™t reproduce on all Android devices, but the Mobile team has it successfully isolated now and is working on a fix.

56

@andreasreich Just downloaded 2018.2.8f1 with your fix and it continues to exhibit the problem of not detecting the other end of the SslStream has shut down. Read and ReadAsync should eventually return a 0 result on a shutdown. The test project I sent you should log ā€œClient: Remote end has closed.ā€ followed by ā€œClient: Done.ā€ but it still hangs.

In the code UnityTlsContext.cs, it makes me wonder if Parent.InternalRead is actually blocking, or saying it would block, or something, because I donā€™t see anything wrong in the fix. It looks like the 0 would get returned, but the read never gets the 0 result, or any result in fact, which is curious.

[ EDIT: SEE BELOW - I wrapped the await ReadAsync() in a try block and found that it is throwing an exception on a graceful close. ]

57

Not sure if it affects anything but do you still want to raise UNITYTLS_USER_READ_FAILED on a gracefully closed connection?

I just tested against a Linux openssl s_server client also by setting the host/port on my ā€œTLS Clientā€ object, and the client doesnā€™t get the graceful shutdown.

58

@andreasreich Actually it looks like ReadASync does throw an exception on a graceful close, and a different exception if data has been successfully written but the remote end has shut down, both of which it shouldnā€™t do. The exception when a shutdown happens is ā€œA call to SSPI failed, see inner exception.ā€, where the exception appears to be an abort of an internal MobileAuthenticatedStream async MoveNext(). Could this be related to the raised UNITYTLS_USER_READ_FAILED error upon a graceful shutdown that I mentioned above? A gracefully closed connection is not an error state, and if any treatment of this as an error causes the underlying task to throw exceptions, it could be causing this exception as a result in an unexpected way. For the situation where a Write()/WriteAsync has been called but the remote end has shut down with that data in the out buffer, ReadAsync should still continue to read data and give a 0 result on the stream if the remote shutdown was graceful, and should still not throw an exception based on write state (if that is indeed happening). Only a further write should cause an error, or an erroneous read after the 0 result has already been received.

59

@anderasreich Iā€™ve also determined that ShutdownAsync() does not shut down the SSL stream cleanly, which makes testing confusing, and I have tested this against OpenSSL with Unity acting as client and server. However the above issue continues against OpenSSL if I just open SslStream and shutdown the remote end of the connection. An exception is thrown, whether it be ā€œA call to SSPI failed, see inner exception.ā€, or the other mystery exception, ā€œUnable to read data from the transport connection: An established connection was aborted by the software in your host machine.ā€, even though data has been received before that shutdown occurred and the shutdown was graceful.

All of the above happens on SslStream built either from a TcpClient or from a NetworkStream attached to a Socket.

If a Wait() is performed on async code on shutdown a ReadAsync it freezes the editor and I have encountered asynchronous tasks staying open and logging after stopping execution.

I think SslStream needs more extensive testing against other tools for reads and writes on shutdowns and graceful/non-graceful disconnections, and the SSL shutdown itself. Undoubtedly there is a lot of complexity to creating this layer. If I can assist with more project examples let me know.

My tests are all Windows so far.

60

Sorry for the delayed reply, my EMail notifications didnā€™t do their job.

@gianfanatee
The issue is fixed in 2018.3 (not sure if it is in the first beta already) and is currently backported 2018.2
I donā€™t think the case is visibly public.

@AlleywayDave
Sorry for all the extra effort you put in there, but the fix did not land in 2018.2 yet. In fact there was apparently some delay and the fix did not even land in 2018.3b1 yet :(. Iā€™m chasing up when it lands and will double check that your proejct works as expected now - quite sure though that I tried with the fix already and it was fine.
Was there something in the release notes that indicated a fix?