Adding GhostComponents to a ghost entity at runtime

I’m working on a project that requires adding GhostComponents to entities at runtime. When I tried this it did not work. I could add the GhostComponent to the entity on the client and the server but then if I change the value of the data on the server it does not update on the client. I understand that this is expected behavior. Is there any way around it?

I’ve thought of having a buffer where each element of the buffer includes a type (maybe serialized to an int) and an entity. Then when you would normally attach a GhostComponent to an entity, you instead spawn an entity from a prefab with that component and “attach” that entity to the main entity via the buffer. If you want to access the data you simply search the buffer for the element with the correct type and run GetComponent on the entity. This gives us all the benefits of a GhostComponent (delta compression mainly) but is a lot harder to work with.

Please let me know if you have any thoughts or if there are any workarounds. Thanks!

At the moment the it is not possible to add component at runtime to a ghost entity and there are any easy work around.

Adding child entities does not means they get replicated. The structure of a ghost is pre-calculated during baking and processed at runtime once to extract a lot of metadata information to serialize the entity.
If you are speaking about making them ghost, then yes they will be. But then you will probably need to make them a group (to be sent together), that will make everything really slow. The solution does not scale well, and it may not work in practice.

The problem, you see, is a little more complex and to properly solve it we need to re-design a lot of things.
We need to change the protocol to communicate which components needs to be added/remove, re-calculate a lot of pre-computed data that it is used to serialize and deserialise ghosts and more.

I’m not saying it can’t be done, but because the current implementation has been designed to be a little more static from the beginning (for many reason, structural changes are really bad), it is hard to fit.

you may try to add all components to the entity and use enable bits to enable/disable the active one. Enable bits are replicated.
However, the bandwidth cost can be quite large if the component are many and big (because they are all serialized, even though they are disabled, for many reasons). However, if the component is disabled, it is probably not updated, and delta compression should work very effectively.

1 Like

Ya. It’s not worth to spend time re-design a lot of things for the solution that offer bad performance by default. I think it’s better to focus on implement enable/disable component feature that able to sync across network efficiently at 1.0 release. So you bake all the required components/ghost component into ghost entity at authoring and also set which component should enable/disable by default then enable/disable component again at runtime based on game requirement.

Do u mean ghost component? I believe when enable/disable component feature is coming at 1.0 release this bandwidth cost issue should be completely solve right? Btw one more thing I would like to ask is that 128 ghost component limitation still there? From what I know 1.0 release will have some limitation removed that it will only count the ghost component that has been instantiated at runtime so still able to mitigate this limitation.

The 128 ghost component limitation is still there. Can be lifted via define to bring it up to 256 but that it.
We are working on a solution that would let us avoid that completely but requires some re-design to the safety system.
However, the ghost component limitation only apply to active ones (used in the current scene/prefab that are loaded).
That should be more than enough in general (128 is a pretty large number of replicated component types).

2 Likes

While we don’t currently use netcode and have our own network solution, I just checked and we have 164 replicate components and buffers.

So I can say in a published game 128 can definitely be surpassed and this would be a consideration for us in our next project where we are considering using netcode but expect to have a significantly larger scope.

The limit of 128 is per Prefab, or overall inside a scene (including sub-scenes) or in the project overall?

Do the existing unity components (like translation, rotation, and physics components) already count against that limit?

Is there a way to check how many components are already used up?

The 128 component limit is imposed on the currently in use components at any point in time. That means, the replicated component collected from the ghost prefabs that you are using while you are in game.

Translation, Rotation, PhysicsVelocity counts toward that limit, if you have ghost using them (and for sure you have).

In this sense, does not matter if you have 300 or 400 or more replicated component in your projects in total. Is just the currently used one that matter.

Now, for sure is possible to surpass this number. And it is possible to lift the limitation (see DynamicTypeList), by setting the NETCODE_COMPONENTS_256 define in your project.
That will give you up 256 component limit.

We are working on removing this limit, but that as I already said, require some rework and changes in the way we deal with the safety handles.

Has this limit been removed in 1.0.0-exp.8?

We currently are working on a networked game where we hit the 256-component limit.

no it has not been removed yet

Any ETA when will happen? Soon or still long time to go?

Do u mean the ghost component count is based on the ghosts found at GhostCollectionPrefab Component that process by GhostCollectionSystem?

The ETA can be a little long, because just expanding the count is not the solution. We need a different approach and that may requires some non trivial changes.

Yes, the count depend on the current loaded ghost prefab in the scene (sub-scene I this case). If you have just a bunch of it, the used count is not for sure 256, unless your prefabs does have a lot of components, in which case It is definitively possible.

How would you go about having a component be synchronized but not active on the instance at the start?

With

[GhostComponent]
[GhostEnabledBit]
public struct SomeComponent : IInputComponentData, IEnableableComponent
{
    [GhostField(Quantization = 1000)] public float2 Position;
}


public class SomeComponentAuthoring : MonoBehaviour
{
    [GhostComponent]
    [GhostEnabledBit]
    public struct WorldTargetComponent : IInputComponentData, IEnableableComponent
    {
        [GhostField(Quantization = 1000)] public float2 Position;
    }

    [SerializeField] private bool isActive;
    [SerializeField] private float2 position;

    public class Baker : Baker<SomeComponentAuthoring>
    {
        public override void Bake(SomeComponentAuthoring authoring)
        {
            var entity = GetEntity(TransformUsageFlags.None);
            AddComponent(
                entity,
                new SomeComponent
                {
                    Position = authoring.position,
                }
            );
            SetComponentEnabled<SomeComponent>(entity, authoring.isActive);
        }
    }
}

if I have isActive set to false in the authoring component the component is added to the ghost prefab entity but set to disabled, the GhostAuthoring is not picking up on the component (which kind of seems like a bug).

I’d like to share a workaround that may be useful to someone here:
In our Ability System, all numerical values are represented as Attributes (floats), and booleans are indexed in a bitmask.
These Attributes are stored in a Dynamic Buffer as key-value pairs and are automatically replicated upon changes, whether new values are added or existing attributes are modified or removed.
We introduced a feature that replicates Attributes as components. This is useful when users need direct access to specific attributes rather than the entire AttributesBuffer, or when they need to respond to certain attribute changes in some reactive systems.
This feature automatically manages the dynamic replication of components from the server to clients.

I’m not sure about you, but my previous response seems unclear even to me! :smile:
Regarding the Attributes Replication feature, it reads from the Attributes ‘replicated component buffer’ + the Tags ‘bitmask component’ and dynamically adds/removes/sets component at runtime. We initially store ComponentTypes related to attributes in a native container and use the internal API to set values in an unsafe manner using pointers.
We created extensions for the ECB models, and added new functions with public access to call their internal versions. Additionally, we included the assembly reference of the entities package to avoid forking the package itself.

I solved this issue by having a custom RPC wrappers that send messages to clients, with manually managing Differences and serialization

Would probably make it into a package later when i find time
I utilize Roslyn and look for ALL types that have ‘AttachTo’ (custom attribute) to automatically generate the necessary code, kind of like with the MLAPI / Mirror / netcode for GameGbjects

using Entities 1.0.10 Unity 2022.3.14

While not pretty, does allow for runtime adding of components (which i had due to my design to have Mods and each dll adds more functionality)
// pseudo code

[GhostComponent(OwnerSendType = SendToOwnerType.SendToOwner)]
[AttachTo(typeof(Player))]
public struct Shipyard : IComponentData
{
    public int dockedShips;
}

And then with the respective RPC receiver and Sender
// client side, On spawn - Send request
var messageArchetype = EntityManager.CreateArchetype(ComponentType.ReadWrite<Shipyard_DetailsRequest>(), ComponentType.ReadWrite<SendRpcCommandRequest>());
Entities.WithNone<RequestedDetails>().ForEach((Entity entity, GhostInstance instance) =>
{
   var message = new Shipyard_DetailsSubscription { target = entity };
   Send(ref message, ref buffer, ref messageArchetype);

   buffer.AddComponent<RequestedDetails>(entity);
}).Run()
buffer.Playback(EntityManager)

//serverside

// Catch the Message (keep in mind, MANY can be at the same time. If you need to process many - use buffers to store them
var messageArchetype = EntityManager.CreateArchetype(ComponentType.ReadWrite<Shipyard_DetailsResponse>(), ComponentType.ReadWrite<SendRpcCommandRequest>());
var lookup = this.GetComponentLookup<Shipyard>();
Entities.ForEach((Entity messageEntity, in Shipyard_DetailsSubscription cmd, in ReceiveRpcCommandRequest req) =>
{
   var response = new Shipyard_DetailsResponse{target = entity, data = lookup[cmd.target]}
   var player = req.SourceConnection;
   Send(ref response, ref player, ref buffer, ref messageArchetype);

   buffer.DestroyEntity(messageEntity);
}).Run()
buffer.Playback(EntityManager)


and then the last piece is to receive it on the Client side
// client side
// catch the messages and apply them to the desired entity
Entities.ForEach((Entity messageEntity, in Shipyard_DetailsResponse cmd, in ReceiveRpcCommandRequest req) =>
{
   buffer.SetComponent(cmd.target, cmd.data);
   buffer.DestroyEntity(messageEntity)
}).Run()
buffer.Playback(EntityManager)

Shipyard_DetailsResponse : IRpcCommand
{
    public Entity target;
    public Shipyard data;
}

Shipyard_DetailsRequest : IRpcCommand
{
    public Entity target;
}

Send methods are
[BurstCompile]
public static void Send(ref Shipyard_DetailsRequest message, ref Entity target, ref EntityCommandBuffer ecb, ref EntityArchetype entityArchetype)
{
    var commandEntity = ecb.CreateEntity(entityArchetype);
    ecb.SetComponent(commandEntity, message);
}

[BurstCompile]
public static void Send(ref Shipyard_DetailsResponse message, ref Entity player, ref EntityCommandBuffer ecb, ref EntityArchetype entityArchetype)
{
    var targetConnection = new SendRpcCommandRequest
    {
        TargetConnection = player
    };
    var commandEntity = ecb.CreateEntity(entityArchetype);
    ecb.SetComponent(commandEntity, message);
    ecb.SetComponent(commandEntity, targetConnection);
}

And then with some code-gen magic it goes generic
and with some ‘Change detection’ allows for a nice Subscriber - Publisher pattern
I got also buffers to be send in the same manner, with pagination. Tested with 50k (sending small pages of 128 items) so ~ 390 messages and it was ~(300-900ms) (No Burst enabled) on localhost until fully received, server was working fine, just the message queue was swamped
(bandwidth might be a bigger issue if you want to send 50k items every frame :smile:)
With Burst it took less than 10ms for the whole thing (6 frames) (6x server, 6x client), sending 8192 items per frame
in my test a message has ~12 + 4 * 128 bytes so 524b / message ~ 200kb total + (?) overhead from netcode

The way i have split things up is:
Components that need to be sent every frame - use the GhostComponent
Components that need to be sent ‘eventually’ - use RPC
Hopefully i do not run into a “Max number of RPC” issue :smile:
Edit: if (rpcIndex == ushort.MaxValue) => v .10 uses 65,535 max rpc, so no fear :smile:
Edit: added the send method; archetypes for completeness

2 Likes

I think this behavior should be better documented I’ve never seen this on the documentation. I was trying to add some tag components on the server side and they don’t get replicated on the clients.

Is the recommended fix is to still use enableable components?

2 Likes

Indeed, documentation is still a bit sparse. I think it’s found partially in the older documentation versions.

If you add a component at runtime it’s not propagated to the clients, it must be added during the baking so it can create a type - cache and look for changes

Using enableable Components works, but you would need to have them at baking time and be careful with running out of the max components to sync limit

The recommended way is to use enable able components to mimic adding/removing components and keep the archetype changes (and so combinations) as low as possible (ideally 0), as well as avoiding useless structural changes.
Dynamic Buffer approaches, like the one described by by @Opeth001 are also plenty valid.

We don’t allow new added component to be replicated on the fly right now and I agree that we need better docs for this,

2 Likes