Serialization. How to iterate over all components of a given entity?

How to serialize/desalinize all data components associated with entity ?

this obviously should be super easy as long as all data components are structs…

public class SaveSystem:ComponentSystem{

    public struct Saveable : IComponentData{}
 
    public struct Data{
      public ComponentDataArray<Saveable> Saveable;
      public EntityArray Entities;
      public int Length;
    }
 
    [Inject] private Data data;

    protected override void OnUpdate(){

      if( Input.GetKeyDown( KeyCode.S ) ){
        Debug.Log( data.Length );
     
        for (int i = 0; i != data.Length; i++){
         Entity e = data.Entities[i];
         var types =  this.EntityManager.GetComponentTypes( e );
          for( int j = 0; j != types.Length; j++ ){
            ComponentType t = types[j];
            ... ??? ...
          }
          types.Dispose();
        }
     
      }
   
    }

the problem is that the only public method to get component data i’ve found is

EntityManager.GetComponentData<T>(Entity)

and it require a compile time Type
but i have a Type in a variable

i’ve found a vary ugly solution using reflection:

MethodInfo methodInfo = typeof(EntityManager).GetMethod("GetComponentData");
MethodInfo genericMethodInfo = methodInfo.MakeGenericMethod(t);
var parameters = new object[]{e};
object componentData = genericMethodInfo.Invoke( EntityManager, parameters );

There must be a better way to get all entity data…

3 Likes

i see that EntityManager has few interesting internal methods

internal unsafe void SetComponentDataRaw(Entity entity, int typeIndex, void* data, int size)
internal unsafe void* GetComponentDataRaw(Entity entity, int typeIndex)
internal unsafe object GetComponentBoxed(Entity entity, ComponentType componentType)
internal unsafe void SetComponentBoxed(Entity entity, ComponentType componentType, object boxedObject)

is it right direction?

I wonder why nobody commenting here, is it bother only me?

Easiness of implementing saving system is one of key benefits of data-oriented paradigm,
but for me it looks like there is no easy way for that in unity ecs… (right now at least)

Am I wrong? Or I missing something?
Please, please don’t ignore me )

1 Like

structs need to be their own class in order to be serialized, for example this will serialize in the inspector:

[Serializable]
public class SOTest : ScriptableObject//or monobehaviour
{
    public Test test;
}
[Serializable]
public struct Test : IComponentData
{
    public int testInt;
}

but not this:

[Serializable]
public class SOTest : ScriptableObject
{
    [Serializable]//SerializeField doesn't work either.
    public struct Test : IComponentData
    {
        public int testInt;
    }
}

So as of right now Entity can not be serialized but IComponentData structs can be if they are not declared within another class, so you might just have to segregate the code and be careful not to include any non-serializable variables in your component data structs.

there is no problem to serialize struct

the problematic part is to iterate over all components on a given entity

I’m still trying to wrap my head around parts of ECS but how I understand it is Entities that act as a collection of components are not accessible “objects” until the game starts whereas components are just containers of data thus accessible in edit mode. You might not be able to iterate through an entities components until it has been activated in play mode; at least in pure ECS. I guess I’ll just have to try and do it and see what I come up with, maybe try [ExecuteInEditMode] or system reflection if possible.

You are trying to see all an entities components in the inspector correct?

no, i’m not new to ECS concept
I understand how it works

i have a pure ecs setup with few entities.

those entities have different data components attached including:

public struct Saveable : IComponentData{}

those entities belong to different archetypes (have different set of components)

game is running

i can see my entities in the entity debugger window

also i have a SaveSystem from first comment
that system iterates over all entities containing Saveable comonent

for each entity i’m getting a list of component types and iterate trough types

var types =  this.EntityManager.GetComponentTypes( e );
for( int j = 0; j != types.Length; j++ ){
  ComponentType ct = types[j];
  Type t = ct.GetManagedType();

}
types.Dispose();

next i need to get a component data by its type
unity ECS has a generic method

GetComponentData<T>(Entity)

that is useless in this situation
because i have a type as a variable

the only workaround that i found is to use a reflection api to hack generics

MethodInfo methodInfo = typeof(EntityManager).GetMethod("GetComponentData");
MethodInfo genericMethodInfo = methodInfo.MakeGenericMethod(t);
var parmeters = new object[]{e};
object componentData = genericMethodInfo.Invoke( EntityManager, parmeters );

but this is a super ugly solution
and i asking a question if i missing something?
i refuse to believe that there is no better way to do that

2 Likes

question is not answered, i’m ready to hear: “it is not implemented yet” but please say something

4 Likes

I am currently in the same boat as you. After scouring the EntityManager API, it seems that at the moment this isn’t possible with the public API. Inside GetComponentData is

public unsafe T GetComponentData<T>(Entity entity) where T : struct, IComponentData
    {
      int typeIndex = TypeManager.GetTypeIndex<T>();
      ...
    }

You can get the TypeIndex of the desired component from the ComponentTypes array. So if Unity simply let us pass in a TypeIndex instead of a generic parameter, it would easily work. Not sure why it’s not an option at the moment.

1 Like

@sahildhanju : we want as much as possible of the API surface to be GC free. If we expose GetComponentData(Entity e, int typeIndex), how would you return the IComponentData without boxing? We could expose some lower level APIs that deal with pointers and preallocated Temp memory I guess, but I’m not a big fan of it.

That said, we are working on a way to serialize/deserialize Entities very efficiently, which will likely lend to new APIs to operate with IComponentData without knowing the concrete type.

10 Likes

Hi @GabrieleUnity , is that prefab will be reworked based on this and Game Object will also be reworked to Entity too in Scene?

Yes, once we are done we will support all (or similar) concepts to what we have today, and also allow easy transition between Unity’s OO and ECS concepts (where possible).

4 Likes

Do you have an idea about when (de)serialization will be available for testing ? Is it a matter of weeks, monthes, years ?
Thanks :slight_smile:

We are working on it right now. I’d say “weeks” unless something goes badly wrong - but don’t quote me on this :wink:

5 Likes

Quoted :wink:
I am really looking toward it !

1 Like

I can’t resist to quote it as well )

Might as well join the hype train.
PS: Now I see why you avoid giving ETAs. :slight_smile:

Any news on this?

now you can serialize the whole world to a binary file, and deserialize that file back to an empty world

but you still can’t

  • serialize single entity
  • serialize single component in a generic way
  • iterate over all entity components without reflection
  • save world partially
  • merge saved data with the existing world (without using second world)
  • use inspector to tweak serialized data
2 Likes

Seems that all public interface shields you from accessing any raw memory.

I’ve noticed EntityContainer at Unity.Entities.Properties that seems to allow to serialize entities (at least to JSON in the example), internally it uses EntityManager.GetComponentDataRawRW.

I wonder if they are planning to use it for flexible serialization or just for editing mode inspection.