Ubuntu Headless Build - TLS Handshake fails

Hey there!

We’ve been using Ubuntu docker containers to host our game servers for a while now and have never had issues with having this server connect to another via Secure Websockets. We’ve upgraded from Unity 5.6 up to 2018.1 without issue.

However, after upgrading to Unity 2018.2, we first noticed an issue calling HTTPS requests through the UnityWebRequest. We were getting error codes 0 with the description Unknown error. Weirdly enough, installing curl in my container ended up resolving this issue. Now I’m stuck with the error described earlier. We’re using Websocket-sharp and never had any issue connecting to our server until we updated to 2018.2. The websocket connector still works in 2018.2 and windows but fails on Linux with a TLS handshake error when trying to connect.

I was wondering if it could be another issue of a missing dependency or that the Linux player is somehow wrongly wrapping the TLS encryption process? I’ve tried with Ubuntu 16.04 and 18.04.

Cheers!

Hi! Unity should come with everything bundled, I don’t think you’re missing anything.

In 2018.2 we got implemented a new TLS backend which we can use in Mono (.Net4 only) and UnityWebRequest. One of the consequences of that is that we can now use platform specific certificate stores on every individual platforms where before we would have shipped a “hardcoded store”. On Linux we’re using the store in /etc/ssl/certs/ca-certificates.crt

So since it works on Windows it does sound a bit like your certificate is trusted by one OS and not by the other. I have not encountered something like that yet but this is also the first report of a TLS handshake with the new system working on one desktop platform but not the other. So maybe something wrong on our side after all.
AlsoI am a bit troubled that you get “Unknown error” with UnityWebRequest, this should be more specific. Can you please file a bugreport with a repro case? Thank you!

Hey Andreas!

Thanks for the quick response! We already filed a bug for the Unknown Error issue. It seems like the error is no longer Unknown Error in 2018.2.5f1 though. But I still need to install curl to get HTTPS working.

I ran a couple tests, like adding our certificate to the list of trusted certificates, accepting any certificates and whatnot but it still doesn’t work. Weirdly enough, in a smaller project, I’m unable to reproduce the issue, so I’m beginning to wonder if I might be doing something wrong.

I have the full stack trace of the issue happening. We’re on Unity 2018.2.5f1 with the 4.x scripting runtime :

WebSocketSharp.WebSocketException: An error has occurred during a TLS handshake. ---> System.AggregateException: One or more errors occurred. ---> System.Security.Authentication.AuthenticationException: A call to SSPI failed, see inner exception. ---> System.ArgumentException: GCHandle value belongs to a different domain
 at System.Runtime.InteropServices.GCHandle.op_Explicit (System.IntPtr value) [0x00025] in <f2e6809acb14476a81f399aeb800f8f2>:0 
 at Mono.Unity.UnityTlsContext.WriteCallback (System.Void* userData, System.Byte* data, Mono.Unity.size_t bufferLen, Mono.Unity.UnityTls unitytls_errorstate* errorState) [0x00006] in <60233c34ef3e4e5f892759dbfc5ae27d>:0 
 at (wrapper managed-to-native) System.Object.wrapper_native_0x8ab3370(Mono.Unity.UnityTls/unitytls_tlsctx*,Mono.Unity.UnityTls/unitytls_errorstate*)
 at Mono.Unity.UnityTlsContext.ProcessHandshake () [0x0001c] in <60233c34ef3e4e5f892759dbfc5ae27d>:0 
 at Mono.Net.Security.MobileAuthenticatedStream.ProcessHandshake (Mono.Net.Security.AsyncOperationStatus status) [0x0003e] in <60233c34ef3e4e5f892759dbfc5ae27d>:0 
 at (wrapper remoting-invoke-with-check) Mono.Net.Security.MobileAuthenticatedStream.ProcessHandshake(Mono.Net.Security.AsyncOperationStatus)
 at Mono.Net.Security.AsyncHandshakeRequest.Run (Mono.Net.Security.AsyncOperationStatus status) [0x00006] in <60233c34ef3e4e5f892759dbfc5ae27d>:0 
 at Mono.Net.Security.AsyncProtocolRequest <ProcessOperation>d__24.MoveNext () [0x000ff] in <60233c34ef3e4e5f892759dbfc5ae27d>:0 
--- End of stack trace from previous location where exception was thrown ---
 at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <f2e6809acb14476a81f399aeb800f8f2>:0 
 at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0003e] in <f2e6809acb14476a81f399aeb800f8f2>:0 
 at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x00028] in <f2e6809acb14476a81f399aeb800f8f2>:0 
 at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x00008] in <f2e6809acb14476a81f399aeb800f8f2>:0 
 at System.Runtime.CompilerServices.ConfiguredTaskAwaitable ConfiguredTaskAwaiter.GetResult () [0x00000] in <f2e6809acb14476a81f399aeb800f8f2>:0 
 at Mono.Net.Security.AsyncProtocolRequest <StartOperation>d__23.MoveNext () [0x0008b] in <60233c34ef3e4e5f892759dbfc5ae27d>:0 
 --- End of inner exception stack trace ---
 at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <f2e6809acb14476a81f399aeb800f8f2>:0 
 at Mono.Net.Security.MobileAuthenticatedStream <ProcessAuthentication>d__47.MoveNext () [0x00254] in <60233c34ef3e4e5f892759dbfc5ae27d>:0 
 --- End of inner exception stack trace ---
 at System.Threading.Tasks.Task.ThrowIfExceptional (System.Boolean includeTaskCanceledExceptions) [0x00011] in <f2e6809acb14476a81f399aeb800f8f2>:0 
 at System.Threading.Tasks.Task.Wait (System.Int32 millisecondsTimeout, System.Threading.CancellationToken cancellationToken) [0x00043] in <f2e6809acb14476a81f399aeb800f8f2>:0 
 at System.Threading.Tasks.Task.Wait () [0x00000] in <f2e6809acb14476a81f399aeb800f8f2>:0 
 at Mono.Net.Security.MobileAuthenticatedStream.AuthenticateAsClient (System.String targetHost, System.Security.Cryptography.X509Certificates.X509CertificateCollection clientCertificates, System.Security.Authentication.SslProtocols enabledSslProtocols, System.Boolean checkCertificateRevocation) [0x0000d] in <60233c34ef3e4e5f892759dbfc5ae27d>:0 
 at (wrapper remoting-invoke-with-check) Mono.Net.Security.MobileAuthenticatedStream.AuthenticateAsClient(string,System.Security.Cryptography.X509Certificates.X509CertificateCollection,System.Security.Authentication.SslProtocols,bool)
 at Mono.Net.Security.MobileAuthenticatedStream.AuthenticateAsClient (System.String targetHost) [0x0000d] in <60233c34ef3e4e5f892759dbfc5ae27d>:0 
 at System.Net.Security.SslStream.AuthenticateAsClient (System.String targetHost) [0x00006] in <60233c34ef3e4e5f892759dbfc5ae27d>:0 
 at WebSocketSharp.WebSocket.setClientStream () [0x000de] in <2f2e2c67bdfd47ac97ec567117e0069d>:0 
 --- End of inner exception stack trace ---
 at WebSocketSharp.WebSocket.setClientStream () [0x00101] in <2f2e2c67bdfd47ac97ec567117e0069d>:0 
at WebSocketSharp.WebSocket.doHandshake () [0x00001] in <2f2e2c67bdfd47ac97ec567117e0069d>:0
at WebSocketSharp.WebSocket.connect ()

Oh apparently we’re using the system’s curl on Linux, not our own. Wasn’t aware, the configuration for curl (or whether we use it at all) is different on every platform. I’ll confirm that later to be sure though.

The inner most exception hints that this is not a verification issue hat all:

System.ArgumentException: GCHandle value belongs to a different domain
at System.Runtime.InteropServices.GCHandle.op_Explicit (System.IntPtr value) [0x00025] in <f2e6809acb14476a81f399aeb800f8f2>:0

As far as I know this should only happen if a disposed SSLStream was accessed. Looking at the current implementation of WebSocket.SetClientStream this can’t be the case since it creates a new SSLStream every time. The only interesting thing in there I can think of is that if there is an exception, there is no explicit Dispose call to the SSLStream, so it would be cleaned up whenever the GC feels like it, potentially leading to issues.
Another possibility is that Mono disposes the SslContext the GCHandle points to due to some other error, but then we should see that error instead.
It’s the first time I have an issue with GCHandle here, so really hard to diagnose without a repro project :frowning:

Does this issue happen on the first connection attempt? Do you have multiple connects in parallel in this project?

@ChainsawesomeJS any more info you can provide?

@andreasreich We also got a problem with Linux Standalone builds for Unity 2018.2. We didn’t see the exact same errors as above, but it looks a lot like the same problem. When we run the app all external (web) calls fail with this error :

<color=Red>Webcallback failed!   ==>> </color>Request (GET) => (<URL>/HealthCheck)
Headers:
Accept: application/json

Exception: System.Net.WebException: Error: SecureChannelFailure (One or more errors occurred.) ---> System.AggregateException: One or more errors occurred. ---> System.Security.Authentication.AuthenticationException: A call to SSPI failed, see inner exception. ---> System.ArgumentException: GCHandle value belongs to a different domain
  at System.Runtime.InteropServices.GCHandle.op_Explicit (System.IntPtr value) [0x00025] in <f2e6809acb14476a81f399aeb800f8f2>:0
  at Mono.Unity.UnityTlsContext.WriteCallback (System.Void* userData, System.Byte* data, Mono.Unity.size_t bufferLen, Mono.Unity.UnityTls+unitytls_errorstate* errorState) [0x00006] in <60233c34ef3e4e5f892759dbfc5ae27d>:0
  at (wrapper managed-to-native) System.Object.wrapper_native_0x8b49270(Mono.Unity.UnityTls/unitytls_tlsctx*,Mono.Unity.UnityTls/unitytls_errorstate*)
  at Mono.Unity.UnityTlsContext.ProcessHandshake () [0x0001c] in <60233c34ef3e4e5f892759dbfc5ae27d>:0
  at Mono.Net.Security.MobileAuthenticatedStream.ProcessHandshake (Mono.Net.Security.AsyncOperationStatus status) [0x0003e] in <60233c34ef3e4e5f892759dbfc5ae27d>:0
  at (wrapper remoting-invoke-with-check) Mono.Net.Security.MobileAuthenticatedStream.ProcessHandshake(Mono.Net.Security.AsyncOperationStatus)
  at Mono.Net.Security.AsyncHandshakeRequest.Run (Mono.Net.Security.AsyncOperationStatus status) [0x00006] in <60233c34ef3e4e5f892759dbfc5ae27d>:0
  at Mono.Net.Security.AsyncProtocolRequest+<ProcessOperation>d__24.MoveNext () [0x000ff] in <60233c34ef3e4e5f892759dbfc5ae27d>:0
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <f2e6809acb14476a81f399aeb800f8f2>:0
  at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0003e] in <f2e6809acb14476a81f399aeb800f8f2>:0
  at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x00028] in <f2e6809acb14476a81f399aeb800f8f2>:0
  at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x00008] in <f2e6809acb14476a81f399aeb800f8f2>:0
  at System.Runtime.CompilerServices.ConfiguredTaskAwaitable+ConfiguredTaskAwaiter.GetResult () [0x00000] in <f2e6809acb14476a81f399aeb800f8f2>:0
  at Mono.Net.Security.AsyncProtocolRequest+<StartOperation>d__23.MoveNext () [0x0008b] in <60233c34ef3e4e5f892759dbfc5ae27d>:0
   --- End of inner exception stack trace ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <f2e6809acb14476a81f399aeb800f8f2>:0
  at Mono.Net.Security.MobileAuthenticatedStream+<ProcessAuthentication>d__47.MoveNext () [0x00254] in <60233c34ef3e4e5f892759dbfc5ae27d>:0
   --- End of inner exception stack trace ---
  at System.Threading.Tasks.Task.ThrowIfExceptional (System.Boolean includeTaskCanceledExceptions) [0x00011] in <f2e6809acb14476a81f399aeb800f8f2>:0
  at System.Threading.Tasks.Task.Wait (System.Int32 millisecondsTimeout, System.Threading.CancellationToken cancellationToken) [0x00043] in <f2e6809acb14476a81f399aeb800f8f2>:0
  at System.Threading.Tasks.Task.Wait () [0x00000] in <f2e6809acb14476a81f399aeb800f8f2>:0
  at Mono.Net.Security.MobileAuthenticatedStream.AuthenticateAsClient (System.String targetHost, System.Security.Cryptography.X509Certificates.X509CertificateCollection clientCertificates, System.Security.Authentication.SslProtocols enabledSslProtocols, System.Boolean checkCertificateRevocation) [0x0000d] in <60233c34ef3e4e5f892759dbfc5ae27d>:0
  at Mono.Net.Security.MonoTlsStream.CreateStream (System.Byte[] buffer) [0x0007b] in <60233c34ef3e4e5f892759dbfc5ae27d>:0
  at System.Net.WebConnection.CreateStream (System.Net.HttpWebRequest request) [0x00073] in <60233c34ef3e4e5f892759dbfc5ae27d>:0
   --- End of inner exception stack trace ---
  at System.Net.HttpWebRequest.EndGetResponse (System.IAsyncResult asyncResult) [0x00058] in <60233c34ef3e4e5f892759dbfc5ae27d>:0
  at System.Net.HttpWebRequest.GetResponse () [0x0000e] in <60233c34ef3e4e5f892759dbfc5ae27d>:0
  at CI.HttpClient.Core.HttpBase.HandleStringResponseRead (System.Action`1[T] responseCallback) [0x00000] in <2a0ce68d4a9d476bb4d55a791bdf1068>:0
  at CI.HttpClient.Core.HttpGet.GetString (System.Action`1[T] responseCallback) [0x00007] in <2a0ce68d4a9d476bb4d55a791bdf1068>:0
Exception Body:

This was working in Unity 2017.4. We are working on updating the project from Unity 2017.4.4f1 to Unity 2018.2.6f1

Do you know why this happens?

Yes, clearly the same issue. It’s clear to me why it pops up starting with 2018.2 (because of the new UnityTls backend and more importantly its mono sided binding), but I’m still in the dark on how and why.

Does this happen only in the Linux standalone? Do you have several connections in parallel?
Would it be possible for you to file a bug with a repro case for this? Thank you

I already submitted a bug report on this number : 1082650 . QA also already reproduced this on there end, and have contacted development to look into this error.

Thank you so much for your help

1 Like

We are having what sounds like a similar issue, with our players reporting that certain distros of Linux are unable to correctly communicate with various HTTPS URLs that our game tries to reach. It doesn’t affect all Linux versions, just some. I don’t have detailed error logs yet about the issue however. The issue mentioned here - Unity Issue Tracker - [x86 Linux Player] WebRequest returns error: ArgumentException: GCHandle value belongs to a different domain says fixed in 2019.1, has this fix been backported to 2018.2? We are using 2018.2.14. Our 2018.1.7 builds do not have this issue.

Thanks for the feedback. I forgot to update this thread. The issue should be should be fixed respectively starting with:
2019.1.0a6, 2018.3.0b8, 2018.2.14f

Officially we only support Ubuntu. Apart from other potential Unity issues I heard that on other distros it can happen that we either fail to access the system certificate store or that we don’t find all the root certificates there that are needed.

Do you have the GCHandle issue described in this thread in particular or is it a “regular” authentication issue?

The issue persists even with 2018.2.15, so it doesn’t look like the same issue. Fedora is the main distro that has been tested. The UnityWebRequest class error message is just “Unknown error” with no real information.

Getting the same error on 2018.2.16f1 Unity Linux headless build.

Unknown error… running on an amazon EC2 instance.

Also have the same issue,
2018.2.16f1 Linux headless build,
it is working on Windows, iOS and android platform

Unity only officially supports Ubuntu Linux, so it is looking (and failing to find) the certificate store where it would expect it to be. You can work around on Fedora by creating a symbolic link:

mkdir -p /etc/ssl/certs && ln -s /etc/pki/tls/certs/ca-bundle.crt /etc/ssl/certs/ca-certificates.crt

1 Like

Thanks a lot for this! It worked.

This is all new to me(Http requests / linux) , could I get some clarification on what to do with this info?

I’m currently experiencing Unity web requests (v 2018.2.11 f 1) replying unknown error with error code 0 when running on Amazon Linux. I’m assuming this is potentially the same certificate problem?

Is it possible to resolve this issue by avoiding UnityWebRequest and using .Net Http support instead?

I switched from UnityWebRequest to using System.Net.WebClient to send the request instead.

This results in a certificate error, even on windows.

I was however able to override the ServerCertificateValidationCallback to get the request to work on both windows and linux. I’m assuming this is not a safe or valid solution though?

What’s the best way to send http requests on linux?

hello any update on this ?
I am using CentOS 7 clients and https urls form a client - server game and on centos with unity 2019.1.0f2 i could not connect webrequests anymore. Was working fine on old unity 2018. Also i build only for 64bits.

Don’t know if it will work for your situation but I managed to get it working on Amazon Linux by running this shell script on the server instance:

sudo mkdir -p /etc/ssl/certs
sudo ln -s /etc/pki/tls/certs/ca-bundle.crt /etc/ssl/certs/ca-certificates.crt