Variadic Generic Systems?

[Piefayth]

I’m trying to write some generic systems for reuse. For example, I might create an Entity with “Foo” and “Request” components to cause another system to initialize me a new Entity with a Foo component. This allows me to keep the logic for initialization of new archetype instances confined to a single system, rather than at the various call sites of CreateEntity. I have been trying to create an inheritable system that allows me to quickly write a system supporting this pattern.

The interface I had in mind:

public class FooBarFactory : RequestSystem<Foo, Bar> { // literally all of the boilerplate
    protected override JobHandle Preprocess(NativeArray<Foo> foos, NativeArray<Bar> bars) {
        // return some job initializing each foo and bar
        // parent system will execute this, create the entity, destroy the request entity
    }
}

And, well, for a correctly-written RequestSystem<T0, T1>, the above works. However, as we all know, you need to have a separate (but nearly identical) RequestSystem definition for every number of type arguments + every combination of component/shared data you want to support. If C# supported metaprogramming, this would be fine, but it’s an ugly workaround to write all that out. RequestSystem<T0, T1, T2, T3Shared, T4Shared> is gross.

I considered, instead, having a single system that processes all the Request entities via chunk iteration. You then could, based on the archetype of each chunk, allow some other injected system to schedule the job to modify the components attached to the original request. Then, this RequestSystem could simply take all the now-updated-components from the Request entity and set them on a new Entity built from an Archetype defined by the injected system. This also works, in the simplest case, but becomes problematic when the injected systems’ jobs try to get write access to the same component at the same time (even, sadly, when they are not in the same chunk, which is always). Suppose you want a (Request, Foo, Bar) and a (Request, Bar) made at the same time. Both injected systems will attempt to mutate Bar components in the provided chunk so the RequestSystem can, at the end, move the data from those Bar components to the final entity being created. Despite the fact that they are operating on different chunks, this mutation is disallowed, even with [NativeDisableParallelForRestriction].

On top of that, it is confusing to me how I would even create the resultant Entities from the RequestSystem. Were I to schedule a job that blindly executes on all Request components to create the entities for each, I would need to pass some sort of collection of types to the job that can be iterated upon for each different archetype. Otherwise the job would not know how many SetComponent calls to do after creating the Entity.

tl;dr - I would like to reduce amount of code in certain systems that reuse logic across a variable number of component types. How?

[5argon]

What I have been waiting for is C# 8.0’s tentative interface default implementation, then I could have a common job with frequently used method, then specialized job struct implementing that interface could add its own thing without need for a template system class.

Also if ECB could enqueue “commands” instead of having to use Add Set Remove etc. I would be able to bring my own commands to the job and then repeat the command based on some criteria in the job. (Create() → Add/Set pattern, I could bring my own Add/Sets into the job to be inserted between each Create)

ECB could also use something more flexible than “previous entity” that it currently used. For example Begin-End pattern where it groups all created entities, then I could batch Add/Set or something.

[Piefayth]

@5argon

What I have been waiting for is C# 8.0’s tentative interface default implementation, then I could have a common job with frequently used method, then specialized job struct implementing that interface could add its own thing without need for a template system class.

Yeah, this is interesting for a lot of reusability situations. Even just being able to extend a job and provide a default implementation for execute that calls through to other “abstract” functions that do the specialized thing(s) for the job would be nice. Comes kinda close to solving my problem, because I could have different implementations of the “same job” take variable amounts of type (component) arguments, but the scheduling system still needs to know which types to pass in, and thus the scheduling systems all need to be specialized per-job still.

@5argon

Also if ECB could enqueue “commands” instead of having to use Add Set Remove etc. I would be able to bring my own commands to the job and then repeat the command based on some criteria in the job. (Create() → Add/Set pattern, I could bring my own Add/Sets into the job to be inserted between each Create)

ECB could also use something more flexible than “previous entity” that it currently used. For example Begin-End pattern where it groups all created entities, then I could batch Add/Set or something.

+1 - ECB seems like it could have the ability to run more arbitrary code, since it ultimately executes its contents on the main thread. It’d be really nice to be able to pass closures to the ECB that can make use of the Entity reference created by CreateEntity; I sometimes split up systems into two systems JUST so I can get a reference to that new Entity and do stuff with it. I wonder if there’s just restrictions WRT what you can pass in just because you’re in a job.

[Piefayth]

Excuse the double post, but I made some progress. I knew there had to be a way to mutate the same component type in two different chunks at the same time. Going back to the chunk iteration line of thinking that I had at the end of my first post, I’ve ended up with this “mega system” that handles Requests via chunk iteration. This full standalone example fulfills requests for “Foo, Baz” entities and “Baz” entities; modifying the Baz component in both cases.

using Unity.Entities;
using UnityEngine;
using Unity.Collections;
using Unity.Jobs;

public class Bootstrap : ComponentSystem {
    protected override void OnUpdate() { }

    protected override void OnCreateManager() {
        EntityManager entityManager = World.Active.GetOrCreateManager<EntityManager>();
        Entity e = entityManager.CreateEntity(typeof(Foo), typeof(Baz), typeof(Request));
        Entity e2 = entityManager.CreateEntity(typeof(Baz), typeof(Request));
        entityManager.SetComponentData(e, new Foo { foo = 43 });
        entityManager.SetComponentData(e2, new Baz { baz = 6 });
    }
}

public struct Foo : IComponentData {
    public int foo;
}

public struct Baz : IComponentData {
    public int baz;
}

public struct Request : IComponentData { }

public class RequestSystemBarrier : BarrierSystem { }
public class RequestSystem : JobComponentSystem {
    [Inject] RequestSystemBarrier barrier;

    EntityManager entityManager;
    ComponentGroup requestGroup;

    public EntityArchetype fooBazRequestArchetype;
    public EntityArchetype bazRequestArchetype;
    public EntityArchetype fooBazArchetype;
    public EntityArchetype bazArchetype;

    ArchetypeChunkComponentType<Foo> fooType;
    ArchetypeChunkComponentType<Baz> bazType;
    ArchetypeChunkEntityType entityType;
  

    protected override JobHandle OnUpdate(JobHandle inputDeps) {
        fooType = entityManager.GetArchetypeChunkComponentType<Foo>(false);
        bazType = entityManager.GetArchetypeChunkComponentType<Baz>(false);
        entityType = entityManager.GetArchetypeChunkEntityType();

        return new RequestsJob {
            commandBuffer = barrier.CreateCommandBuffer().ToConcurrent(),
            fooType = fooType,
            bazType = bazType,
            entityType = entityType,
            fooBazRequestArchetype = fooBazRequestArchetype,
            bazRequestArchetype = bazRequestArchetype,
            fooBazArchetype = fooBazArchetype,
            bazArchetype = bazArchetype,
        }.Schedule(requestGroup, inputDeps);
    }

    struct RequestsJob : IJobChunk {
        public EntityCommandBuffer.Concurrent commandBuffer;

        public ArchetypeChunkComponentType<Foo> fooType;
        public ArchetypeChunkComponentType<Baz> bazType;
        [ReadOnly] public ArchetypeChunkEntityType entityType;

        public EntityArchetype fooBazArchetype;
        public EntityArchetype bazArchetype;
        public EntityArchetype fooBazRequestArchetype;
        public EntityArchetype bazRequestArchetype;

        public void Execute(ArchetypeChunk chunk, int chunkIndex) {
            NativeArray<Entity> entities = chunk.GetNativeArray(entityType);

            if (chunk.Archetype == fooBazRequestArchetype) {
                NativeArray<Foo> foos = chunk.GetNativeArray(fooType);
                NativeArray<Baz> bazzes = chunk.GetNativeArray(bazType);

                Baz baz = bazzes[0];
                baz.baz = 591;
                bazzes[0] = baz;

                for (int i = 0; i < foos.Length; i++) {
                    commandBuffer.CreateEntity(chunkIndex, fooBazArchetype);
                    commandBuffer.SetComponent(chunkIndex, foos[i]);
                    commandBuffer.SetComponent(chunkIndex, bazzes[i]);
                    commandBuffer.DestroyEntity(chunkIndex, entities[i]);
                }
            } else if (chunk.Archetype == bazRequestArchetype) {
                NativeArray<Baz> bazzes = chunk.GetNativeArray(bazType);

                Baz baz = bazzes[0];
                baz.baz = 333;
                bazzes[0] = baz;

                for (int i = 0; i < bazzes.Length; i++) {
                    commandBuffer.CreateEntity(chunkIndex, fooBazArchetype);
                    commandBuffer.SetComponent(chunkIndex, bazzes[i]);
                    commandBuffer.DestroyEntity(chunkIndex, entities[i]);
                }
            }
        }
    }

    protected override void OnCreateManager() {
        entityManager = World.GetOrCreateManager<EntityManager>();
        requestGroup = entityManager.CreateComponentGroup(typeof(Request));
        fooBazRequestArchetype = entityManager.CreateArchetype(typeof(Foo), typeof(Baz), typeof(Request));
        bazRequestArchetype = entityManager.CreateArchetype(typeof(Baz), typeof(Request));
        fooBazArchetype = entityManager.CreateArchetype(typeof(Foo), typeof(Baz));
        bazArchetype = entityManager.CreateArchetype(typeof(Baz));
    }
}

There obviously would be a lot of noise there with Archetypes and SetComponents for many kinds of requests, but there is a great amount of code deduplication simply by not having to create a new system for every kind of Request.

Problem is, I don’t see a clear path to modularizing such a system. It would be great to define the specifics of each Request archetype in another file as needed to reduce some of the noise here. I thought that may be possible by scheduling a different job type per chunk based on the chunk’s archetype, but as soon as you pass the same writable ArchetypeChunkComponentType to two different jobs, the safety system tells you “no.” (If this is unclear, I can provide code). IJobChunk in the above example seems to circumvent this, since you are only scheduling the single job.

For this reason, I am also curious why there is IJobChunk but no IJobChunkParallelFor. If the safety system gets unhappy when I schedule different jobs on different chunks that share a writable component type, but any individual job scheduled on an individual chunk can write to its own contents in parallel, I don’t see why there shouldn’t be a built-in job that allows you to process all chunks and their contents in parallel.

[5argon]

IJobChunk is parallel by default. It is 1 worker thread per chunk. (Like IJobProcessComponentData, after the patch that removes integer argument on its Schedule, unless using ScheduleSingle)

[Spy-Shifty_1]

@5argon

What I have been waiting for is C# 8.0’s tentative interface default implementation, then I could have a common job with frequently used method, then specialized job struct implementing that interface could add its own thing without need for a template system class.

Also if ECB could enqueue “commands” instead of having to use Add Set Remove etc. I would be able to bring my own commands to the job and then repeat the command based on some criteria in the job. (Create() → Add/Set pattern, I could bring my own Add/Sets into the job to be inserted between each Create)

ECB could also use something more flexible than “previous entity” that it currently used. For example Begin-End pattern where it groups all created entities, then I could batch Add/Set or something.

Well what you can do at the moment is to define a extension method for an interface, that implements the generic default case!

interface IMyInterface {
     void Meth1();
     void Meth2();
}

static class IMyInterfaceExtension {

    public void DefaultMeth<TMyInterface>(this TMyInterface instance) where TMyInterface : IMyInterface {
        instance.Meth1();
        instance.Meth2();
    }
}

Usage:

//external
myInterfaceObject.DefaultMeth();

//internal
this.DefaultMeth();