Issues Setting Up Encrypted Websocket Connection with Unity Transport

Unity Engine
, , ,
1

Hello,

I have been developing a multiplayer game for several months, it is built with WebGL and hosted on a website. The server is a linux server that I deploy via AWS GameLift. I recently added SSL to my site, and naturally began to get MixedContent errors in the console since my UnityTransport was not using secure websockets.

I ticked on Use Encryption within the UnityTransport component on my NetworkManager hoping it would be seamless integration but sadly I have spent countless hours trying to fix my whole setup due to the lack of support for wss.

I came across this thread where @simon-lemay-unity mentioned the following:

Set up a reverse proxy that will accept WSS traffic, pop the encryption, and forward it to your server unencrypted (and then do the reverse for all traffic coming from the server). Clients will think theyā€™re connecting to a secure server, but your actual server will not need to mess with certificates and such. Unfortunately, thatā€™s not exactly a simple solution. Youā€™ll need to spin up a machine at some cloud provider to act as the reverse proxy. And youā€™ll still need to set up certificates on that machine (some cloud providers might make this easier though).

I have done exactly this. I built a node.js reverse proxy server that is hosted on a subdomain of my main website. Both my main website and the subdomain have SSL certificates. I have set up my nginx configuration for my subdomain such that when the index route / is hit, it routes traffic to the proxy server running on the same machine at port 8080:

server {
    http2 on;
    listen 443 ssl;
    listen [::]:443 ssl;
    server_name subdomain.mysite.com;

    ssl_certificate ...
    ssl_certificate_key ...


...

location / {

        proxy_pass http://127.0.0.1:8080;
        proxy_http_version 1.1;

        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_set_header Host $host;
    }

I am able to successfully hit this proxy server which is hosted on the same machine as my backend api ā€“ both from my local dev machine and the machine itself. I can hit it successfully with a variety of websocket tools including websocat, browser console, curl, etc. I cannot however hit this proxy server with Unity despite trying everything imaginable. It seems the UnityTransport doesnā€™t allow hostnames but rather only ip addresses, so I have to connect like this:

UnityTransport transport = NetworkManager.Singleton.GetComponent<UnityTransport>();

transport.UseWebSockets = true;
transport.UseEncryption = true;
transport.SetClientSecrets("subdomain.mysite.com");  //using the real domain
transport.SetConnectionData("000.000.000.000", 443); //using the real IP address
...
NetworkManager.Singleton.StartClient();

However time and time again I get Failed to connect to server. I have tried with the hostname instead of the IP, I have tried specifying the caCert from LetsEncrypt (r3, r6). I even put a direct log route in the nginx config to see if I could even hit that route from unity but I canā€™t. I can hit it via anything else, but not Unity. Verified several times that I have the right IP and port.

In summary, I cannot for the life of me get Unity to connect to the proxy server (to then establish an unencrypted websocket connection with the gameserver) after calling NetworkManager.Singleton.StartClient(). Iā€™ve had direct unencrypted connections to GameLift-hosted server instances working for months, but as soon as I ticked encryption on, everything went to hell :slight_smile: . I think it has to do with the fact that I must supply an IP address instead of the hostname, to which the SSL cert is bound. If you try to supply the hostname as the transport connection address, youā€™ll get invalid endpoint errors when calling StartClient.

Is there an established way to connect the UnityTransport to a proxy server that is hosted on a website with SSL?

Package Versions:

Netcode for Gameobjects: 2.2
Unity Transport: 2.4
2

I should mention Iā€™m mainly trying to connect from the Unity editorā€¦ I am not sure if there is significantly different behaviour between the editor and WebGL build when it comes to connecting with TLS.

Nothing seems to work :skull_and_crossbones:

3

Iā€™m not sure what is going wrong here because you appear to be configuring everything correctly. I would probably try from a WebGL build just to rule out a difference in the TLS handling in the editor. Iā€™d also look at the traffic with a tool with Wireshark (or tcpdump on your Linux server) to see where exactly things are falling apart (e.g. does the TLS session manages to be established but then the connection to the game server fails or is TLS going wrong from the get-go).

However in your specific situation you might be able to do away with the reverse proxy entirely. AWS GameLift appears to support providing TLS certificates to its servers. If this is actually the case, you could use those certificates and configure the Unity game server to use encryption directly (through UnityTransport.SetServerSecrets). No need for a reverse proxy at all. My recommendation to use one is really in the context of Unity Multiplay, which currently canā€™t provide certificates to game servers.

4

Wow, I am so grateful that you pointed out this option that I have seemingly been overlooking all this time!

image

I see the ā€œGenerate TLS certificateā€ option now in fleet creation setup. Thank you, and apologies as netcode (in general) is relatively new territory for me.

I will move forward with this approach immediately, but if you end up having some time to reply to this and just confirm my understanding of the following, it would be very much appreciated (in case I run into a roadblock in the meantime):

  • I now need no TLS termination via nginx, and can establish a direct secure websocket connection to my linux server hosted on my AWS fleet.
  • Prior to calling NetworkManager.Singleton.StartServer() on my server instance, I will need to call SetServerSecrets on the unity transport, passing in the server certificate, and the server private key. These will be tied to my fleet, and Iā€™ll access them when the server spins up by calling GetInstanceCertificate which is noted on the page you linked from the AWS SDK.
  • I will still need to call SetClientSecrets, on the UnityTransport object, as I believe this is needed any time Use Encryption is ticked on. For the serverCommonName, I am assuming I provide the domain of my website that my game is hosted on? For caCertificate I am assuming I provide the LetsEncrypt certificate for my website as well? The method parameters for SetClientSecrets I find less intuitive.

For the last point above, I feel like I read somewhere that if in WebGL, I wonā€™t need to supply a certificate cause it uses the browser store. However, while testing in the Editor, I think this get a bit more hairy and Iā€™ll have to work through this certificate mess :slight_smile: .

Thank you again so much for pointing me to this. If I get it working in the coming days, I will report back with the full solution and mark as resolved.

5

I now need no TLS termination via nginx, and can establish a direct secure websocket connection to my linux server hosted on my AWS fleet.

Yes, that is correct.

Prior to calling NetworkManager.Singleton.StartServer() on my server instance, I will need to call SetServerSecrets on the unity transport, passing in the server certificate, and the server private key. These will be tied to my fleet, and Iā€™ll access them when the server spins up by calling GetInstanceCertificate which is noted on the page you linked from the AWS SDK.

Yes, that is also correct. I donā€™t know in what format the AWS SDK will provide the certificate and key, but know that SetServerSecrets expects those in the PEM format. Unfortunately the version of .NET weā€™re on doesnā€™t have built-in support for PEM, so if you need to convert from another format youā€™ll have to do so manually.

I will still need to call SetClientSecrets, on the UnityTransport object, as I believe this is needed any time Use Encryption is ticked on. For the serverCommonName, I am assuming I provide the domain of my website that my game is hosted on? For caCertificate I am assuming I provide the LetsEncrypt certificate for my website as well? The method parameters for SetClientSecrets I find less intuitive.

For serverCommonName you need to provide the domain name of the game server itself, not the website hosting the game that your users access. Iā€™m not familiar with the AWS SDK so not sure if they provide a domain name for each fleet/server. If they only provide an IP address, put that in the serverCommonName. Itā€™s possible theyā€™re being fancy and providing certificates for IP addresses.

For caCertificate you donā€™t provide anything. Presumably the certificates provided by AWS are authenticated through a proper certificate authority and thus will be validated automatically by the browser/OS. That is also the case in the editor (if no CA certificate is provided we just ask the OS).

6

Hello, Iā€™m helping @nickostan make these changes.

I have attempted to apply the suggestions above, but note that Iā€™m using GetComputeCertificate as it seems to be the new name for GetInstanceCertificate in the latest AWS Gamelift SDK.

Iā€™m running into this error Iā€™m seeing in the Firefox console:

Firefox canā€™t establish a connection to the server at wss://blah-domain-name

I see a similar error on chrome as well.

After enabling TLS certificates on the Gamelift fleet, I believe the SetClientSecrets parameter serverCommonName needs to be the DnsName property from the GameSession object that weā€™re passing back to the client. That is what Iā€™m using (I did try using the IpAddress there too but without success). However in the description of DnsName it states:

When connecting to a game session that is running on a TLS-enabled fleet, you must use the DNS name, not the IP address.

Right now, when calling Unity Transportā€™s SetConnectionData method, I am using the GameSession objectā€™s IpAddress. I did try resolving the DnsName manually and it does point to the same IpAddress, but Iā€™m wondering if the reason why the wss connection cannot be established is because Iā€™m attempting to connect with the ip address when the DnsName property description states not to.

Is my reasoning for why the browser canā€™t connect via wss plausible? If so, is there a way around this using Unity Transport? From what I can tell, establishing connections using domain names is not supported.

Note that Iā€™m doing all this on my development machine and so Iā€™m connected to my unity client through my browser at localhost which is unsecured. I was going to try using a self-signed cert for localhost to see if the fact that Iā€™m trying to establish a secure connection within an unsecured browser context is part of the problem, but I figured Iā€™d ask about DnsName first.

Hopefully that all makes sense. Thank you!

7

Iā€™m not familiar with AWS GameLift, but assuming that this DnsName property is indeed the hostname on which to reach the game server, you only need to pass it as the serverCommonName parameter of SetClientSecrets on your client. On WebGL builds, that will actually override the IP address from SetConnectionData and weā€™ll make a connection straight to the domain name, not the IP address. Passing the correct IP address to SetConnectionData (on top of passing the correct domain name to SetClientSecrets) is only required in the editor or in non-WebGL builds, where weā€™re dealing with low-level sockets and thus need the IP address.

If youā€™re still observing connection failures in that situation, Iā€™d first check if the certificate and its key are properly passed to SetServerSecrets on the server side, inspecting them to make sure they are in the PEM format. If thatā€™s the case, youā€™d need to look at what exactly is going wrong with the TLS handshake. Often the browser will give you pertinent information. You can look at the network tab of the developer tools for the details of the WebSocket connection. Or alternatively you could just put the address/port of the game server in the address bar of the browser. Obviously that will fail to establish a WebSocket connection (youā€™ll normally get a 400 HTTP code), but the TLS handshake takes place before we get there so if something is wrong the browser will tell you what.

8

Oh and one thing I forgot to mention above, while we do ignore the IP address from SetConnectionData in WebGL builds, we donā€™t ignore the port. So make sure to at least set that right. In general Iā€™d still recommend passing the correct IP address if you have that handy, since then it will automatically work in the editor too.

9

Thank you @simon-lemay-unity for the insight and tips!

I finally got this working! It was a silly mistake, but I overlooked one crucial aspect when connecting to our Gamelift fleet: setting up a port range in our Gamelift fleet settings to allow for inbound traffic to access the instance running our unity server process. Aws Gamelift was rejecting the wss request from the unity client because the port we were connecting to was not open!

A final recap for future readers: the process discussed here is exactly what needed to be done to communicate over wss with the caveat of needing to use the DnsName GameSession property instead of the IpAddress property when calling the SetClientSecrets function.

10

I also encountered the same problem, according to the document Create a secure client and server | Unity Multiplayer Still unable to connect to dedicated servers, constantly reporting errors, WebSocket connection to ā€™ wss://xx.xxx.xxx.xxx:xxxx/ ā€˜failed. It seems that wss cannot be used for dedicated servers direct connection.ā€™ @simon-lemay-unity

11

Those instructions have you create a self-signed certificate. Is that what you did? Such certificates will not work on browsers, which require certificates signed by an official certificate authority. Youā€™ll need to obtain official certificates for your domain. If youā€™re using AWS GameLift (as above), then their SDK provides those directly. If youā€™re hosting the game on your own server, youā€™ll need to use a service like Letā€™s Encrypt to get a certificate.