Client-hosted server model?

Hey,
What are the plans to support a client-hosted server model?

In my specific case, I have a lot of procedural data that is needed by both client and the server worlds. This causes duplicate data on a hosting client i cannot afford.

Kind regards,

Both server-hosted and client-hosted games are supported. What exactly do you mean by “client-hosted server model”?

Cannot afford because of … ?

Procedural data, by its nature, should be able to be replicated on every client merely by sending a minimal amount of data to start with, such as a random seed number and maybe some extra setting values (size, count, etc). Therefore this does not affect traffic in any meaningful way if engineered appropriately.

The amount of data kept in memory or on disk (on the server side) does not affect current hosting prices. For the server you get a fixed amount of memory (8 GiB) and disk space (50 GiB) that’s included in the monthly Game Server Hosting plan.

Client-hosted games do not incur costs but the (necessary) Relay service does.

Hey, thanks for the reply.

In my case, I have procedural terrain / polygons, these need to exist in both worlds since the data is needed for both gameplay logic (server) and rendering/interaction (client). While you’re right that the procedural generation itself only needs minimal seed data, the actual generated data (vertices, heights, etc.) ends up being processed/stored twice when running as a client-host - once in the Server World and once in the Client World.

This duplicate memory/work is what I’m trying to avoid. Aka 1 world, which acts as a Server-Client.

Im wondering if there are any planned features or recommended patterns
to handle this kind of scenario where we want to share data between the client and server worlds when they’re running on the same machine.

1 Like

I guess i can try it like this:

 [WorldSystemFilter(WorldSystemFilterFlags.ClientSimulation | WorldSystemFilterFlags.LocalSimulation)]
     public partial struct MyClientSystem : ISystem {}

And in the bootstrap if its a server/with local client, I’ll add the LocalSimulation systems to the ServerWorld also.

if (clientHost) {
   systems = DefaultWorldInitialization.GetAllSystems(WorldSystemFilterFlags.LocalSimulation | WorldSystemFilterFlags.Presentation);
   DefaultWorldInitialization.AddSystemsToRootLevelSystemGroups(serverWorld, systems);
}

Not sure how this will work with SendRpcCommandRequests though.

I guess this mainly depends on how you set up that data. The raw data could be in any struct you define, and you reference the same struct in both worlds. I can‘t guarantee it but I would be surprised if it weren‘t possible to have data shared between worlds, particularly if you ensure the data is not written to from both worlds in parallel.

If anyone takes the time to read this, thank you in advance.

Aim: Have a Local Client that can function within the ServerWorld
Test Case: Share camera position of all connected clients / local client via ghost entities.

Step 1: Server World Setup

  • Setup ServerWorld to include WorldSystemFilterFlags.LocalSimulation and WorldSystemFilterFlags.Presentation
  • Don’t spawn ClientWorld if PlayType.ClientAndServer

Step 2: Local Client Network Connection
Create NetworkConnection entity mimic for local client with networkId=0:

    public partial struct LocalConnectionEntitySingleton  : IComponentData{
        public Entity Entity;
        public static implicit operator Entity(LocalConnectionEntitySingleton element) => element.Entity;
    }
    
    [WorldSystemFilter(WorldSystemFilterFlags.LocalSimulation)]
    [BurstCompile]
    public partial struct LocalConnectionSystem : ISystem{
        [BurstCompile]
        public void OnCreate(ref SystemState state) {
            Entity networkConnectionLocal = state.EntityManager.CreateEntity();
            state.EntityManager.AddComponentData(networkConnectionLocal, new NetworkId() {
                Value = 0
            });
            state.EntityManager.AddComponent<NetworkStreamInGame>(networkConnectionLocal);
            state.EntityManager.CreateSingleton(new LocalConnectionEntitySingleton() {
                Entity = networkConnectionLocal
            });
        }
    }

This is obviously very naive since it’s not really a NetworkConnection like the ones that spawn for external clients.
I tried to create a new NetworkDriver and register it to the store but that was too complex.
Perhaps a LocalNetwork component could be added that gets handled differently for RPCs/ghosts.

Step 3: RPC System
Create a system that runs on both LocalSimulation (ServerWorld) and ClientSimulation (ClientWorld):

    [WorldSystemFilter(WorldSystemFilterFlags.ClientSimulation | WorldSystemFilterFlags.LocalSimulation)]
    [UpdateInGroup(typeof(CameraSystemGroup))]
    public partial struct ClientCameraLocationRpcProviderSystem : ISystem {

        public void OnCreate(ref SystemState state) {
            if (state.World.IsClient()) {
                state.RequireForUpdate<NetworkStreamConnection>();
            }
            state.RequireForUpdate<EndSimulationEntityCommandBufferSystem.Singleton>();
        }

        public void OnUpdate(ref SystemState state) {
            if (state.World.IsClient()) {
                NetworkStreamConnection     connectionState = SystemAPI.GetSingleton<NetworkStreamConnection>();
                if (connectionState.CurrentState != ConnectionState.State.Connected) {
                    return;
                }
            }
            
            foreach (RefRO<CameraControl> cameraControl in SystemAPI.Query<RefRO<CameraControl>>().WithChangeFilter<CameraControl>()) {
           
                EntityCommandBuffer ecb             = SystemAPI.GetSingleton<EndSimulationEntityCommandBufferSystem.Singleton>().CreateCommandBuffer(state.WorldUnmanaged);
                Entity              rpcEntity       = ecb.CreateEntity();
            
                ecb.AddComponent(rpcEntity, new CameraLocationRPC() {
                    Position = cameraControl.ValueRO.CurrentPosition
                });
                if (state.World.IsClient()) {
                    ecb.AddComponent(rpcEntity, new SendRpcCommandRequest());
                }
                else {
                    if (SystemAPI.TryGetSingleton(out LocalConnectionEntitySingleton singleton)) {
                        ecb.AddComponent(rpcEntity, new ReceiveRpcCommandRequest() {
                            SourceConnection =  singleton
                        });
                    }
                }
            }
        }
    }

There are some ways in which i could remove if (state.World.IsClient()) checks, i could always send a
SendRpcCommandRequest and have a system that converts it to a ReceiveRpcCommandRequest before it gets picked up. idk.

Steps 4/5: Ghost Management and RPC Handling
Single system handling both ghost instantiation (prefabs created with GhostPrefabCreation) and RPC processing:

  public struct CameraLocationReference : IComponentData {

        public Entity CameraLocationEntity;

    };
    
    
    [WorldSystemFilter(WorldSystemFilterFlags.ServerSimulation)]
    [UpdateInGroup(typeof(CameraSystemGroup), OrderFirst = true)]
    public partial struct CameraLocationHandlerSystem : ISystem {

        private ComponentLookup<CameraLocation> _cameraLocationLookup;
        private ComponentLookup<CameraLocationReference> _cameraLocationReferenceLookup;
        
        public void OnCreate(ref SystemState state) {
            state.RequireForUpdate<LocalConnectionEntitySingleton>();
            state.RequireForUpdate<CameraLocationPrefabSingleton>();
            state.RequireForUpdate<EndSimulationEntityCommandBufferSystem.Singleton>();
            
            _cameraLocationLookup = SystemAPI.GetComponentLookup<CameraLocation>(false);
            _cameraLocationReferenceLookup = SystemAPI.GetComponentLookup<CameraLocationReference>(true);
        }

        public void OnUpdate(ref SystemState state) {
            _cameraLocationReferenceLookup.Update(ref state);
            _cameraLocationLookup.Update(ref state);

            EntityCommandBuffer ecb = SystemAPI.GetSingleton<EndSimulationEntityCommandBufferSystem.Singleton>().CreateCommandBuffer(state.WorldUnmanaged);
            foreach ((RefRO<NetworkStreamInGame> _, RefRO<NetworkId> id, Entity entity) in 
                     SystemAPI.Query<RefRO<NetworkStreamInGame>, RefRO<NetworkId>>().WithNone<CameraLocationReference>().WithEntityAccess()) {
                CameraLocationPrefabSingleton locationPrefabSingleton = SystemAPI.GetSingleton<CameraLocationPrefabSingleton>();
                Entity                        instance             = ecb.Instantiate(locationPrefabSingleton.Entity);
                ecb.SetComponent(instance, new GhostOwner() {
                    NetworkId = id.ValueRO.Value
                });
                ecb.AddComponent<CameraLocationReference>(entity, new CameraLocationReference() {
                    CameraLocationEntity = instance
                });
                Debug.Log($"creating camera location entity for client: {id.ValueRO.Value}]");
            }

            foreach ((RefRO<ReceiveRpcCommandRequest> receiveRpcCommandRequest, RefRO<CameraLocationRPC> cameraLocationRPC, Entity entity) in 
                     SystemAPI.Query<RefRO<ReceiveRpcCommandRequest>, RefRO<CameraLocationRPC>>().WithEntityAccess()) {
                Entity                  sourceConnection        = receiveRpcCommandRequest.ValueRO.SourceConnection;
                CameraLocationReference cameraLocationReference = _cameraLocationReferenceLookup[sourceConnection];
                RefRW<CameraLocation>   cameraLocation          = _cameraLocationLookup.GetRefRW(cameraLocationReference.CameraLocationEntity);
                cameraLocation.ValueRW.Position = cameraLocationRPC.ValueRO.Position;
                ecb.DestroyEntity(entity);
            }
        }

Effect: Achieved local client request functionality in the ServerWorld without needing a separate ClientWorld.

While this workaround functions, having an official way of doing this would be nice.
If anyone from the Unity team could comment on whether this is a planned feature or if there’s a better approach, it would be greatly appreciated.

I got something similar to work with ICommandData too.

1 Like

This is not a problem for Netcode to solve. This is a design problem in your product. If your proc gen functions are deterministic then only the seed is ever needed to be shared to replicate it correctly on other instances of the same application. If you are concerned about Client-As-Server architecture having to generate an instance of data to work on as well as a client to generate the same data then you simply have to start looking at options and choose a solution.

For example if you’re running a separate headless application as the server, then connecting to it as a client application on the same machine - you’re clearly not going to be able to share any datasets for processing between them without significant (and moderately silly) effort.

The typical approach to Client-As-Server is to not have a headless server running and simply use the Client application as the server so that you can leverage sharing data within the application. If that’s your target structure then there is ample information about approaching designs that way and many nuances you’ll have to tackle for your project specifically.

My last post is under review from the automated spam system… Maybe that will clear things up.

However,
Hi LaneFox
I think there might be some misunderstanding. My question isn’t about procedural generation or sharing data between headless servers and clients.

My concern is specifically about the Netcode for Entities architecture when running in a single process (Client-As-Server as you mentioned). Even in this scenario, the current design requires spawning separate Client and Server worlds, which seems unnecessary for local play.

What I’m trying to achieve is:

  1. Architecture where local client logic can run directly in the server world
  2. Clean handling of RPCs and Ghost entities for local clients and remote clients (unified system)
  3. Proper support for local client networking without spawning a separate client world

I respectfully disagree with that statement

Netcode for Entities already solves many architectural problems that could be argued are “design problems” rather tha networking problems.

If the problem is sharing data among the server and the client world, you can opt for many solution, given the fact both world live in the same application.
You can definitively share the data itself (there is no problem accessing readonly blobs or other things).

That being said, we have some plan to have a hosting model that have both client and server in the same world. However, that break the nice data and logic encapsulation the world separation provide (and nice code split and other subtle problems).

Indeed the single world with client/server systems is way more performant (both memory and cpu) but lose some major advantages:

  • cleaner architecture
  • closer the actual client + server model
  • less to test. True, you always need to test real build with single client but indeed the odds that you have code path that include server logic in client systems is close to 0.
  • more flexible
2 Likes

I was also sceptical about not having option for hosted-client world in netcode for ecs but, If your game doesn’t run at reasonable framerate when host is running server and client world at the same time. Your game won’t be able to run for remote clients smoothly. Because at minimum, they’ll have 30ms of latency which will cause prediction loop to run multiple times (loop count depends on your tick rate). So, in fact, being host and local-client is more lightweight than being remote client only.

If your problem is sharing readonly data between worlds, you have many solutions as others stated above.