Rewriting a system using chunks/queries instead of to-be-deprecated injected group structs

Since injected struct fields with component groups would be deprecated (according developer comments from “API usability” thread), I wanted to learn how to write systems without relying on them. To that end, I have three very simple systems handling removal, change or addition of instanced mesh renderer component based on tag components:

https://www.hastebin.com/fiwadulayi.cs

Rewriting these into a single query/chunk based system seemed like a good idea, especially since that was exactly what Unity developers did with the MeshInstanceRendererSystem they provided as a sample. So, following that example, I tried to rewrite the three systems above into one system accomplishing exact same thing with three queries.

https://www.hastebin.com/qakujicido.cs

I quickly encountered a few issues:

  • I’m not sure how to get entities from a chunk, but I assumed that chunk.GetNativeArray (entityType) is the correct way to do that.
  • I’m not sure how to get components from a chunk, but I assumed that chunk.GetNativeArray (componentType) is the correct way to do that.
  • I’m not sure if I understand the concept of chunks correctly - am I understanding right that to work with individual entities and components, I have to use two loop levels, one for chunk collection, and one for content of a chunk?
  • One of these assumptions here is wrong, or I’m making some other mistake in the code, because the system linked above fails to do anything (BlockRendererAddition components remain attached to entities forever with no apparent consequences). I’m not sure where the issue is, because there are no errors or warnings

Getting past all these details, the biggest question I have is simple: is chunk/query use supposed to be so verbose, or am I doing something wrong? Comparing this system to 3 old injection based systems, the difference in verbosity is huge. I really dislike magical attributes and I’m all for a shift away from injected fields, but the three systems linked at the top of this post are vastly easier to maintain and less error prone than the query/chunk based system I link. I’m inclined to believe I’m just doing something wrong, incorrectly complicating the implementation, so I’d love to hear any advice on the right way of using chunks/queries.

1 Like

Correct

Correct

Correct. ComponentDataArray and co basically do the same in the background.

Some issues I see:

  • GetArchetypeChunkEntityType/GetArchetypeChunkComponentType has to be executed in OnUpdate as this call is responsible for dependency tracking.
  • There currently is nothing like Injection for this, you actually have to execute your queries:
var chunks = EntityManager.CreateArchetypeChunkArray(query, Allocator.TempJob);

Currently it is but I think they are working on it.

3 Likes

You not getting chunks in your code, you must use something like this:
NativeArray<ArchetypeChunk> chunks = EntityManager.CreateArchetypeChunkArray(someEntityArchetypeQuery, someAllocatorType);
as @julian-moschuering says

@julian-moschuering @eizenhorn
Thanks for quick replies! I have changed a few things based on your advice:

  • Archetype chunk entity/component types are no longer fetched at manager creation and are instead filled on every update
  • Chunks are retrieved in every update using CreateArchetypeChunkArray

Additionally:

  • I’m not sure about this since I have little experience with native types, but I think the result of CreateArchetypeChunkArray has to be disposed, so I added a Dispose call for it at the end of each update
  • Looking at MeshInstanceRendererSystem, PostUpdateCommands is not used at all there - instead, a temporary EntityCommandBuffer is allocated, modified in the nested loop for each entity in a chunk, and disposed. I replicated the same

Here is the updated code:
https://www.hastebin.com/datenopeve.cs

This updated version seems to work, but it triggers an avalanche of warnings which I’m unable to pinpoint. Each warning goes like this:

A Native Collection has not been disposed, resulting in a memory leak. 
It was allocated at C:\Users\username\AppData\Local\Unity\cache\packages\
packages.unity.com\com.unity.entities@0.0.12-preview.15\
Unity.Entities\Iterators\ArchetypeChunkArray.cs:280.

This is not a very useful error message since it points at an internal ECS method as opposed to part of my code, but I suspect it has something to do with either a native array I get from CreateArchetypeChunkArray call or an EntityCommandBuffer. I dispose both, though, so I don’t know what’s causing this error.

3750667--311884--upload_2018-10-4_15-4-2.png

Yeah, that’s what I do, unless I’m misunderstanding something. Check each Process* method:

entityCommandBuffer.Dispose ();
chunks.Dispose ();

Empty arrays have to be disposed too. You only dispose if length > 0.

1 Like

Ah you fastest than me :stuck_out_tongue:

@bac9-flcl yes you allocate NativeArray but not dispose if length == 0.

1 Like

Oh dang, that was such a rookie mistake! :smile:

Thanks for pointing this out, disposing the chunk array in a case where I bail out at length == 0 solved the error. Final version (I hope?): https://www.hastebin.com/opajurohil.cs

Also I not see where you dispose “entities” and “additionComponents” (and other in other
Process*)

Those actually don’t have and don’t should be disposed. Only the Top chunk array must be disposed. GetNativeArray returns a wrapper around the actual chunk data.

1 Like

So, the correct steps for a basic system meant to do component additions/removals/changes are these, right?:

  • Get native array with chunks from a query

  • Allocate a new entity command buffer

  • For each chunk

  • Get a native array with entities from the chunk array based on chunk index

  • Get native arrays with relevant components from the current chunk

  • For each entity

  • Select entity and components from native arrays based on entity index

  • Call removal/addition/setting methods on entity command buffer using entity and components

  • Execute entity command buffer

  • Dispose entity command buffer

  • Dispose the chunks array

Edit: Removed disposal of entity and component arrays as @julian-moschuering suggested.

Yes you are right, I confused this piece with a manually allocated values array from the documentation which is then passed to the job :slight_smile: My bad :slight_smile:

Could you elaborate a bit more on how this “wrapper” works and can be identified? Both ArchetypeChunk.GetNativeArray and EntityManager.CreateArchetypeChunkArray return exact same type (native array) - what makes one require disposal and another not require it? I mean, now I know what to dispose in this particular case, but I don’t know how to identify the difference in the future across the rest of the ECS API.

One guess I have is that EntityManager.CreateArchetypeChunkArray creates an entirely new native array for you, which makes disposal your responsibility, while ArchetypeChunk.GetNativeArray gives you an existing array that was created without your input and is disposed by the chunk holding it when the time comes. So, no type difference or some internal state difference on the native array itself, just the context matters.

Because they not allocated and reuse allocated memory in chunk. (ArchetypeChunk.GetNativeArray)
3750925--311926--upload_2018-10-4_16-13-26.png

and EntityManager.CreateArchetypeChunkArray allocates memory (wich reused in ArchetypeChunk.GetNativeArray above)

3750925--311929--upload_2018-10-4_16-14-59.png

1 Like

There currently is no way to see it without looking at the source but you can’t really do it incorrectly.
Not disposing the ChunkArray will throw an exception and disposing the array returned by GetNativeArray will throw one also (The NativeArray can not be Disposed because it was not allocated with a valid allocator.)

Good to know, thanks!

I’m sad that injected structs are going to be deprecated. I have build a system around it.

Just move to Chunk Iteration, it’s not hard and clean, and you can write your own [Inject] in future :slight_smile:

I’m confident that the engineers at Unity are striving for a system that will as simple to use as Injection, but maybe with less magic involved. One example was explored by @SamOld in the API Usability topic, where the required components are specified to a generic class, and then just the inner part of the iteration is exposed to the user. (Similar to IJobProcessComponentData)

I think you could just as well stick with [Inject] until the new API design comes along.

I don’t think the plan is to have users deal with Chunk iteration in the ‘final’ API, it feels like one too many thing to iterate over that the user does not necessarily need to know/care about.

1 Like