Unity Relay with Multiplay Hosting

Hi all,

I’ve been working on supporting secure Websockets (WSS) for a N4E-based application. This is to ensure the app works in restrictive firewall environments.

We have Unity Relay working for client-hosted sessions using WSS. Now I am trying to do the same for a dedicated server mode.

The advice I’ve been able to find online is, either implement a reverse proxy with proper certificates or use Unity Relay in front of the dedicated server.

I didn’t have much luck with the reverse proxy, so now I am trying to get our server to connect as a host on Relay. Only, the client connections time out. This is confusing, as I am using near-identical code as we use for client-hosted sessions, which we confirm to be working well.

Is there anything I need to know about the Multiplay Hosting context, like network restrictions or any specific steps I need to carry out to make it work with dedicated servers?

On the server, we perform boot processes and start hosting. High-level, this is what happens next:

// 1. Allocate Relay server and obtain Join Code
var allocation = await _relayService.CreateAllocation(_lifecycleSettings.MaxPlayers);
string joinCode = await _relayService.GetJoinCode(allocation.AllocationId);

// 2. Register Relay network driver constructor
var serverRelayData = new RelayServerData(allocation, "wss");
var driverConstructor = new RelayNetworkDriverConstructor(serverData: serverRelayData);
NetworkStreamReceiveSystem.DriverConstructor = driverConstructor;

// ... Under the hood: RelayNetworkDriverConstructor.CreateServerDriver
var settings = DefaultDriverBuilder.GetNetworkSettings();
settings = settings.WithRelayParameters(ref _relayServerData);
DefaultDriverBuilder.RegisterServerWebSocketDriver(world, ref driverStore, netDebug, settings);

// 3. Set up server world and transition to scene
var serverWorld = ClientServerBootstrap.CreateServerWorld("ServerWorld");
await _ourSceneSwitchingLogic.LoadSessionSceneAsync();

// 4. Start listening for client connections
var endpoint = NetworkEndpoint.AnyIpv4.WithPort(serverRelayData.Endpoint.Port);
var listenRequest = serverWorld .EntityManager.CreateEntity(typeof(NetworkStreamRequestListen));
serverWorld.EntityManager.SetComponentData(listenRequest, new NetworkStreamRequestListen { Endpoint = endpoint });

On the client, when joining a server, we do something like:

// 1. Join the allocation using code
var joinAllocation = await _relayService.JoinAllocation("Join code from server");

// 2. Register Relay network driver constructor
var clientRelayData = new RelayServerData(joinAllocation, "wss");
var driverConstructor = new RelayNetworkDriverConstructor(clientData: clientRelayData);
NetworkStreamReceiveSystem.DriverConstructor = driverConstructor;

// ... Under the hood: RelayNetworkDriverConstructor.CreateClientDriver
var settings = DefaultDriverBuilder.GetNetworkSettings();
settings = settings.WithRelayParameters(ref _relayClientData);
DefaultDriverBuilder.RegisterServerWebSocketDriver(world, ref driverStore, netDebug, settings);

// 3. Set up client world and transition to scene
var clientWorld = ClientServerBootstrap.CreateClientWorld("ClientWorld");
await _ourSceneSwitchingLogic.LoadSessionSceneAsync();

// 4. Connect to Relay server
var connectRequest = clientWorld .EntityManager.CreateEntity(typeof(NetworkStreamRequestConnect));
clientWorld .EntityManager.SetComponentData(connectRequest, new NetworkStreamRequestConnect { Endpoint = clientRelayData .endpoint });

What happens next is nothing. No errors until the connection eventually times out.

I’ve confirmed the network driver setup to work for client-hosted sessions. Servers work when we do standard UDP connect as well.

My suspicion is something is going on in the Multiplay hosted context that’s different from client builds, but I am struggling to narrow down what.

Any ideas? Is there something I’m doing wrong?

Some additional context:

If I connect my client+server using the flow described above but with UDP instead of WSS on both sides, it works.

If I connect the server using UDP and the client using WSS, a connection is established but server logs the connect attempt as timed out during handshake:

[Server World][Connection] NetworkConnection[id0,v1] timed out after 5000ms (threshold:5000ms, state:Handshake)!

According to Unity Relay docs:

When using Unity Relay, then cross-play support comes for free, without you having to do anything to enable it. You could have a host connecting to the Relay server with DTLS, a client connecting with WebSocket, and another connecting with UDP, and both clients will be able to communicate with the host without any issue.

So I’m confused as to why this problem should occur.

The cross-play capabilities of Relay are based on the assumption that each peer consider the end-to-end transport as unreliable. Each packet can be wrapped for Relay in one protocol, unwrapped then rewrapped in another protocol to its final destination. Peers don’t have to be aware of what protocol each other is using.

When configured for WebSockets, Netcode for Entities makes optimizations on its Unity Transport pipelines by using Null stages where you’d normally have Reliability and Sequencing stages. The nuance here is that a Direct client-to-server WebSocket connection is not equivalent to a Relay connection over WebSocket.

You want each side to be using Unity Transport pipelines suited for unreliable transport (eg. UDP) , enable Relay, and then independently decide which of the Relay protocols will be used on each side (UDP, DTLS, WSS).

3 Likes

Thank you, appreciate the extra context!

Making sure that the same pipelines are used on the client connecting via WSS resolved the issue.

We got some pointers from Unity support as well on how to configure this. For future reference, instead of using Netcode’s DefaultDriverBuilder class for creating the client driver, we can create it manually with the required pipelines:

var driverInstance = new NetworkDriverStore.NetworkDriverInstance
{
    simulatorEnabled = false,
    driver = NetworkDriver.Create(new WebSocketNetworkInterface(), settings)
};

// use the same pipelines in WSS mode as UTP/DTLS use
driverInstance.unreliablePipeline = driverInstance.driver.CreatePipeline(typeof(NullPipelineStage));
driverInstance.reliablePipeline = driverInstance.driver.CreatePipeline(typeof(ReliableSequencedPipelineStage));
driverInstance.unreliableFragmentedPipeline = driverInstance.driver.CreatePipeline(typeof(FragmentationPipelineStage));
driverStore.RegisterDriver(TransportType.Socket, driverInstance);

This works in a scenario where the server connects to Relay via DTLS (using DefaultDriverBuilder.RegisterServerDriver) and the client connects via WSS.