RPC FixedList512Bytes<int> 0 on receive

Unity Engine
, ,
1

Hi,

I have been experimenting with FixedList512Bytes for sending a list of info in an RPC. However, when I receive my request the length is correct but the values are always 0.

For example:

   //A request from our client to move some units somewhere
    public struct MovementRequestRPC : IRpcCommand
    {
        public float3 DesiredLocation;
        //Note this has a hard limit of 127 entities! Stored as GhostComponent ghostid!
        public FixedList512Bytes<int> SelectedEntites;
    }

When breakpoint on send RPC, values are correct, when breakpoint receive RPC they are all 0.

How to sync a Dictionary between Server and Client?
2

FixedList512Byte is not a supported serialized type. We support FixedString but not FixedList.
You need to create a template that handle the fixed list and register it the code-gen system by using the
UserDefinedTemplates.
Read the docs manual, there is section in that regards. Or look at the Asteroids sample too.

In practice, create a class that implement the UserDefinedTemplates.RegisterTemplates partial method.

public static partial class UserDefinedTemplates
{
    static partial void RegisterTemplates(List<TypeRegistryEntry> templates, string defaultRootPath)
    {
                templates.Add(new TypeRegistryEntry
                {
                    Type = "Unity.Collections.FixedList512",
                    Quantized = false,
                    Smoothing = SmoothingAction.Clamp,
                    SupportCommand = true,
                    Composite = false,
                    Template = $"{TemplatesPath}DefaultTypes/GhostSnapshotValueFixedString32Bytes.cs"
                    TemplateOverride = "Path/To/MyTemplate.FixedList512.cs"
               }
}

And in your template file, you should write some fragments to handle the list. The DataStream don’t support writing and reading the fixed list but you can overcome the limitation.

The template looks like

namespace Generated
{
    public struct GhostSnapshotData
    {
        struct Snapshot
        {
            #region __GHOST_FIELD__
            public FixedList512 __GHOST_FIELD_NAME__;
            #endregion
        }

        public void SerializeCommand(ref DataStreamWriter writer, in IComponentData data, in IComponentData baseline, StreamCompressionModel compressionModel)
        {
            #region __COMMAND_WRITE__
            //The writer does not support WriteFixedList etc etc..
            writer.WriteShort(data.__COMMAND_FIELD_NAME__.Length);
            //brute force the serialiation would looks like or use the unsafe version and serialize all the bytes you need
            for (int i = 0; i < data.__COMMAND_FIELD_NAME__.Length; ++i)
                writer.WriteInt(data[i]);
            #endregion
        }

        public void DeserializeCommand(ref DataStreamReader reader, ref IComponentData data, in IComponentData baseline, StreamCompressionModel compressionModel)
        {
            #region __COMMAND_READ__
            data.__COMMAND_FIELD_NAME__.Lengh = reader.ReadShort();
            for (int i = 0; i < data.__COMMAND_FIELD_NAME__.Length; ++i)
                data[i] = reader.ReadInt();
            #endregion
        }
    }
}

This would only work for FixedList512 when used for RPC. If you need to have this type into ghost components or commands (discouraged in both case) you need add other fragments.

3

Another even simpler solution is to write a custom serializer for that RPC (that is probably fitting best your use case).

Something like

[BurstCompile]
public struct MovementRequestRPC : IComponentData, IRpcCommandSerializer<MovementRequestRPC>
{
    public float3 DesiredLocation;
    //Note this has a hard limit of 127 entities! Stored as GhostComponent ghostid!
    public FixedList512Bytes<int> SelectedEntites;


    public void Serialize(ref DataStreamWriter writer, in RpcSerializerState state, in RpcLevelLoaded data)
    {
        writer.WriteFloat(DesiredLocation.x);
        writer.WriteFloat(DesiredLocation.y);
        writer.WriteFloat(DesiredLocation.z);
        writer.WriteShort(SelectedEntities.Length);
        for(int i=0;i<SelectedEntities.Length;++i)
            writer.WriteInt(SelectedEntities[i]);
    }

    public void Deserialize(ref DataStreamReader reader, in RpcDeserializerState state, ref RpcLevelLoaded data)
    {
        DesiredLocation.x = writer.ReadFloat();
        DesiredLocation.y = writer.ReadFloat();
        DesiredLocation.z = writer.ReadFloat();
        SelectedEntities.Length = writer.ReadShort();
        for(int i=0;i<SelectedEntities.Length;++i)
            SelectedEntities[i] = writer.ReadInt();
    }

    [BurstCompile(DisableDirectCall = true)]
    [MonoPInvokeCallback(typeof(RpcExecutor.ExecuteDelegate))]
    private static void InvokeExecute(ref RpcExecutor.Parameters parameters)
    {
        RpcExecutor.ExecuteCreateRequestComponent<MovementRequestRPC, MovementRequestRPC>(ref parameters);
    }

    static readonly PortableFunctionPointer<RpcExecutor.ExecuteDelegate> InvokeExecuteFunctionPointer = new PortableFunctionPointer<RpcExecutor.ExecuteDelegate>(InvokeExecute);
    public PortableFunctionPointer<RpcExecutor.ExecuteDelegate> CompileExecute()
    {
        return InvokeExecuteFunctionPointer;
    }
}

class MovementRequestRPCCommandRequestSystem : RpcCommandRequestSystem<MovementRequestRPC, MovementRequestRPC>
{
    [BurstCompile]
    protected struct SendRpc : IJobEntityBatch
    {
        public SendRpcData data;
        public void Execute(ArchetypeChunk chunk, int orderIndex)
        {
            data.Execute(chunk, orderIndex);
        }
    }
    protected override void OnUpdate()
    {
        var sendJob = new SendRpc{data = InitJobData()};
        ScheduleJobData(sendJob);
    }
}
}

Check also the RPC documentation for more details

4

Thanks for your reply. I ended up using the fixed list 512 byte with the int param as I will require this for a number of different RPC’s (movement/attack etc).

I spent a significant amount of time getting this working but got there in the end. The main issue was that my template was not working and seemed to do nothing (not picked up by the autogenerator) despite running ForceCodeGeneration. I still have no idea why it happened but at some point it seems that it just got picked up (I don’t think I even changed anything!)

My UserTemplates file:

namespace Unity.NetCode.Generators
{
public static partial class UserDefinedTemplates
{
public const string TemplatesPath = "Packages/com.unity.netcode/Editor/Templates/";
static string OverridesPath = "Assets/Scripts/NetCodeGen/Templates";

static partial void RegisterTemplates(List<TypeRegistryEntry> templates, string defaultRootPath)
{
templates.Add(new TypeRegistryEntry
{
Type = "Unity.Collections.FixedList512Bytes",
Quantized = false,
Smoothing = SmoothingAction.Clamp,
SupportCommand = true,
Composite = false,
Template = $"{TemplatesPath}DefaultTypes/GhostSnapshotValueFixedString32Bytes.cs",
TemplateOverride = $"{OverridesPath}/IntFixedList512.cs"
//TemplateOverride = ""
});
}
}
}

It has the Assembly Reference too, just as in the samples.

My Template

namespace Generated
{
public struct GhostSnapshotData
{
struct Snapshot
{
#region __GHOST_FIELD__
public FixedList512 __GHOST_FIELD_NAME__;
#endregion
}
public void SerializeCommand(ref DataStreamWriter writer, in IComponentData data, in IComponentData baseline, StreamCompressionModel compressionModel)
{
#region __COMMAND_WRITE__
//The writer does not support WriteFixedList etc etc..
writer.WriteInt(data.__COMMAND_FIELD_NAME__.Length);
//brute force the serialiation would looks like or use the unsafe version and serialize all the bytes you need
for (int i = 0; i < data.__COMMAND_FIELD_NAME__.Length; ++i)
writer.WriteInt(data.__COMMAND_FIELD_NAME__*);*
*#endregion*
*}*
*public void DeserializeCommand(ref DataStreamReader reader, ref IComponentData data, in IComponentData baseline, StreamCompressionModel compressionModel)*
*{*
*#region __COMMAND_READ__*
*data.__COMMAND_FIELD_NAME__.Length = reader.ReadInt();*
*for (int i = 0; i < data.__COMMAND_FIELD_NAME__.Length; ++i)*
<em>data.__COMMAND_FIELD_NAME__ *= reader.ReadInt();*</em>
_*#endregion*_
_*}*_
_*}*_
_*}*_
_*```*_
5

Hey,

mmm… the template registration looks incorrect.
You should register the concrete type:

            templates.Add(new TypeRegistryEntry
            {
                Type = "Unity.Collections.FixedList512Bytes<int>",
                ...

If you don’t do that, the type cannot find a match. This is probably why is was not working (you may have changed this?)

It is great that you get the template working. This is always good to know how that thing work, in case you need to add any other custom type. +1000 Kodus!

However, while this work in your specific use case, I would suggest to override also the following regions (eventually just empty)
GHOST_WRITE
GHOST_READ
COMMAND_WRITE_PACKED
COMMAND_READ_PACKED

The reason being, the template you wrote only work for RPC: if this list is added to a command or a replicated component, then thing will break (because it wants to serialize a FixedString32).

I will use this occasion to talk a bit more about this use case (override template or adding template for new types).
This may be useful to anyone that would read this tread (just in case).

The template you written, CAN ONLY WORK FOR RPC and it is not overriding all the required fragments.
So, while it work for you, it is by no mean a “best practice” to write a partial template for a type in this way.

When you make a template override that uses a base template (in this case the FixedString32) and you are changing the snapshot type you are using (in this case is a FixedList512 instead of FixedString32) you need to check how the base template implements the various fragments and overrides all the ones that are not compatible with the new type. In particular, for this one:
GHOST_WRITE
GHOST_READ
COMMAND_WRITE
COMMAND_READ
COMMAND_WRITE_PACKED
COMMAND_READ_PACKED

Also, in general, as soon as you register a type to code-gen, some assumptions are made:

1- it is supposed to work for type that are present in replicated component. The following regions must be present (even if empty):
GHOST_WRITE
GHOST_READ
GHOST_PREDICT
GHOST_COPY_TO_SNAPSHOT
GHOST_COPY_FROM_SNAPSHOT
GHOST_RESTORE_FROM_BACKUP
GHOST_CALCULATE_CHANGE_MASK
GHOST_CALCULATE_CHANGE_MASK_ZERO
GHOST_REPORT_PREDICTION_ERROR
GHOST_GET_PREDICTION_ERROR_NAME

2- if the template SupportCommand is true, the following regions must also be present:
COMMAND_WRITE
COMMAND_READ
COMMAND_WRITE_PACKED
COMMAND_READ_PACKED

6

I changed the user defined template to your suggestion:

Type = "Unity.Collections.FixedList512Bytes<int>",

I think there may be a bug in the code generation… It is extremely flaky! Sometimes it works and picks up my user changes but at other times it will not generate no matter how many times I force generation!

7

If you add a template you must force code-generation. If you change the template, you must force a code-generation.
Template are not detected by Unity compilation pipeline, so it is required to invoke the force-generation every time you make a change.

In any event, can you please report a bug for that? So we can track it.
Please add as much details as you can about your workflow or what you were doing.
If you can add you template and/or mini-project would be also great.

8

I don’t really want to start a new thread just for this but on a related note, I found it a bit weird ‘double’ isn’t a natively supported serialized type. I’ve added my own TypeRegistryEntry with a quantized supported version, it just seemed a bit odd not to be supported by default.

9

I agree, that was a missing basic type we should support.

I think we can also try to add support for generic types like FixedList<> or in general other fixed size container of us. But it is not on the road map yet.

For double and other types you guys tend to use and you are seeing we are missing some support please continue to report.

10

I would appreciate native support for FixedList<> by Netcode for Entities for serializing RPC commands and synchronizing ghosts.

I tried making a FixedList code generation template like described in this thread, but failed, because the template was never picked up by the code generation itself. Don’t know why. Tried various things mentioned here but had no luck. Also it seems in version 1.0.0 of Netcode a few things changed regarding the templating which may be the reason.

As a workaround I now use the FixedStringXyzBytes types for transferring list data as part of RPC commands.

1 Like
11

I would also like to add my +1 to have FixedLists supported in RPCs.

Until that is the case, it would be great if the manual would indicate which types are and aren’t supported for rpc’s.

2 Likes
12

Hey! yes, we know this is super useful! And we will add something for that!
The manual actually tell what are the supported types ,but this info is inside a specific documentation for the template and code-generation, check:
https://docs.unity3d.com/Packages/com.unity.netcode@1.0/manual/ghost-types-templates.html

13

using System.Collections.Generic;

namespace Unity.NetCode.Generators
{
public static partial class UserDefinedTemplates
{
public const string TemplatesPath = “Library/PackageCache/com.unity.netcode@1.0.11/Editor/Templates/DefaultTypes/”;
static partial void RegisterTemplates(List templates, string defaultRootPath)
{
templates.AddRange(new[ ]{
new TypeRegistryEntry
{
Type = “Unity.Collections.FixedList512Bytes”,
Quantized = false,
Smoothing = SmoothingAction.Clamp,
SupportCommand = true,
Composite = false,
Template = $“{TemplatesPath}GhostSnapshotValueFixedString32Bytes.cs”,
TemplateOverride = “Packages/com.meta.framework/NetCodeGen/Templates/FixedList512.NetCodeSourceGenerator.additionalfile”,
},
}) ;
}
}
}
#templateid: Custom.FixedList512
namespace Generated
{
public struct GhostSnapshotData
{
struct Snapshot
{
#region GHOST_FIELD
public FixedList512Bytes GHOST_FIELD_NAME;
#endregion
}

public void Serialize(ref Snapshot snapshot, ref Snapshot baseline, ref DataStreamWriter writer, ref StreamCompressionModel compressionModel, uint changeMask)
{
#region GHOST_WRITE

#endregion
}

public void Deserialize(ref Snapshot snapshot, ref Snapshot baseline, ref DataStreamReader reader, ref StreamCompressionModel compressionModel, uint changeMask)
{
#region GHOST_READ

#endregion
}

public void SerializeCommand(ref DataStreamWriter writer, in IComponentData data, in IComponentData baseline, StreamCompressionModel compressionModel)
{
#region COMMAND_WRITE
writer.WriteShort(data.COMMAND_FIELD_NAME.Length);
for (int i = 0; i < data.COMMAND_FIELD_NAME.Length; ++i)
writer.WriteInt(data.COMMAND_FIELD_NAME);
#endregion
#region COMMAND_WRITE_PACKED
#endregion
}
public void DeserializeCommand(ref DataStreamReader reader, ref IComponentData data, in IComponentData baseline, StreamCompressionModel compressionModel)
{
#region COMMAND_READ
data.COMMAND_FIELD_NAME.Lengh = reader.ReadShort();
for (int i = 0; i < data.COMMAND_FIELD_NAME.Length; ++i)
data.COMMAND_FIELD_NAME = reader.ReadInt();
#endregion
#region COMMAND_READ_PACKED
#endregion
}
}
}
this is my template and register， but it didn’t work， and has an error " C:\projects\dots\Packages\com.unity.netcode\Runtime\SourceGenerators\Source~\NetCodeSourceGenerator\Generators\TemplateFileProvider.cs(81,1): error NetCode: NetCode AdditionalFile ‘NetCodeGen/Templates/FixedList512.NetCodeSourceGenerator.additionalfile’ (named ‘Custom.FixedList512’) is a valid Template, but it cannot be matched with any UserDefinedTemplate (probably a typo). Known user templates:[Unity.Collections.FixedList512Bytes[Library/PackageCache/com.unity.netcode@1.0.11/Editor/Templates/DefaultTypes/GhostSnapshotValueFixedString32Bytes.cs]]." I don’t know how to fix it

14

Is that FixedList<> RPC will be supported at next 1.1 exp release?

Contents of FixedList is not properly serialized/deserialized
15

No it is not.

16

My netcode version is 1.2.0-pre.6， and i write my .additionalfile and UserDefinedTemplates correctly according to project NetcodeSamples, but unity Editor has error : error NetCode: NetCode AdditionalFile ‘E:/GOODS/Unity/YIEntity/Assets/NetCodeGen/Templates/Rotation2d.NetCodeSourceGenerator.additionalfile’ (named ‘Custom.Translation2d’) is a valid Template, but it cannot be matched with any UserDefinedTemplate (probably a typo). Known user templates:[ ]. I don’t know why, and I would appreciate it if you can give some advice.

17

This is saying it cannot find any UserDefinedTemplates, implying this step was not performed correctly. Can you double check your .asmref?

18

I’ve implemented this version (a custom serializer for that RPC) and couldn’t figure out why the rpc is empty. It took me some time to notice that there is an “in” and “ref” RpcLevelLoaded data that supposedly should be used to write and read from. So the fix was data.DesiredLocation[i] instead of direct DesiredLocation[i]

1 Like
19

Ah, yeah, I ran into that myself last month, with IRpcCommandSerializer<T>. We can change it for 2.0 likely, but not 1.x.

1 Like
Compressing FixedString and IRpcCommandSerializer<T> List Serialization with Netcode for ECS
20

In my situation, i was implementing the serialization functions in the command itself rather than the serializer.

The exception is so random that you will ask yourself “Why?” around 2 hours until you find this post.

Don’t read from (or write to) the struct field values themselves (do not read or write in-place), instead read from (and write to) the by-ref argument data
(Fancy way of saying: ONLY do serialization in the serializer!)

// Wrong
public struct Command: IComponentData, IRpcCommandSerializable // Custom interface
{
    public readonly void SerializeTo(ref DataStreamWriter writer, in RpcSerializerState state)
    { ... }

    public void DeserializeFrom(ref DataStreamReader reader, in RpcDeserializerState state)
    { ... }
}

internal struct CommandSerializer : IComponentData, IRpcCommandSerializer<CommandSerializer>
{
    public void Serialize(ref DataStreamWriter writer, in RpcSerializerState state, in Command command)
    {
        command.SerializeTo(...);
    }

    public void Deserialize(ref DataStreamReader reader, in RpcDeserializerState state, ref Command command)
    {
        command.DeserializeFrom(...);
    }
}


// Correct
internal struct CommandSerializer : IComponentData, IRpcCommandSerializer<CommandSerializer>
{
    public void Serialize(ref DataStreamWriter writer, in RpcSerializerState state, in Command command)
    { ... }

    public void Deserialize(ref DataStreamReader reader, in RpcDeserializerState state, ref Command command)
    { ... }
}