Edit: I said NativeList in my original post, but really I mean NativeArray.
I’ve been reading about making structural changes to Entities, and it sounds like it’s a lot faster to batch the changes than to do them one at a time. For example, if you need to destroy a bunch of Entities, it’s faster to collect all of the entities together into a NativeList and call EntityManager.DestroyEntity (listOfEntitiesToDestroy) rather than to loop through the Entities sequentially calling EntityManager.DestroyEntity (entityToDestroy) for each individual Entity you want to destroy.
This makes a lot of sense. However, the drawback is that this approach uses EntityManager, which means you’re executing it on the main thread and this creates a sync point at that moment. This forces all of your scheduled Jobs to complete and the main thread is blocked until the Jobs complete. Effectively, you move from multi-threaded to single-threaded as everything syncs to the main thread.
The way you avoid creating the sync point (at that moment in time) is to defer the structural change(s) by using an Entity Command Buffer (ECB). However, looking at the documentation, ECBs don’t have any methods to support batched structural changes to Entities using a NativeList. I do see that there is a method using an ECB that supports batched structural changes using an EntityQuery, but this doesn’t provide the same sort of flexibility as passing an explicit list.
Therefore, it would seem that there exists a tradeoff between being able to batch structural changes but having to create a sync point (using the EntityManager) and having to execute structural changes sequentially but being able to defer the creation of the sync point (using the ECB).
I have a couple of questions about this:
First, is there something I’ve missed and I have this completely wrong?
Assuming I haven’t missed anything, is one of these two approaches generally more efficient than the other overall (i.e. batching w/ sync point vs sequential but deferred execution)? What constraints might tip the balance in favor of one approach or the other?
Are there any plans to implement methods to support batched structural changes using NativeLists on ECBs? Or is this a fundamental limitation of using ECBs?
ECB is “batched”, as it is only executed when its Playback is called. Never looked into the ECB internal code, but I think that a NativeList override would possibly just be the same - performance wise - as doing a foreach on the NativeList and calling the ECB method for each element.
Destroying entities - I haven’t observed much of a performance difference. But that might be because the entities that need to be destroyed are quite scattered throughout the different chunks.
Enable/Disable - There is no equivalent of this in EntityCommandBuffer that defers collecting the LinkedEntityGroup until playback. Over EntityManager without Burst, this is a huge win.
Instantiate - This took me a while to get right (as I also initialize component values), but it is significantly faster when the source prefab is shared by multiple instantiated entities.
I plan to make a much more extensive write-up of these optimizations and why they are optimizations in the coming weeks. But if you have any questions in the meantime, I will be more than happy to answer!
I took a quick look at the code you linked to and, if I understand correctly, it looks like you’re rolling your own command buffer functionality which is designed to both defer structural changes (like a standard EntityCommandBuffer) but also take advantage of the batching provided by EntityManager methods which use NativeList to execute a batch of structural changes.
Based on your write-up and the fact that you went through the trouble of creating this, I take it there is a meaningful performance difference between performing these structural changes sequentially via a standard ECB and executing them in a batch using the EntityManager. Furthermore, since there was that difference in performance, you decided to create these custom command buffers so that you could take advantage of the performance gains to be had by using the EntityManager but also be able to defer execution (like with a standard ECB) so that you didn’t introduce sync points all across your application (like you would if you just used the EntityManager batching methods). Is that about right?
An ECB is “batched” in the sense that it groups a series of commands together to be executed together at a later time. However, I think this batching is really more about avoiding the problems created by introducing a sync point into your application (thereby flushing your Job queue and blocking the main thread). This is different from the batching of structural changes being done by the EntityManager when you use one of the batched structural change methods.
Digging into the source (and using Entity destruction as an example), the EntityManager.DestroyEntity (NativeArray entities) method ultimately calls EntityComponentStore.DestroyEntities (Entity* entities, int count) which is optimized for destroying multiple entities at the same time (see line 47 in EntityComponentStoreCreateDestroyEntities.cs). The EntityCommandBuffer calls EntityComponentStore.DestroyEntityWithValidation (Entity entity), which ultimately wraps the call EntityComponentStore.DestroyEntities (&entity, 1). So it calls the same method, but it passes in a pointer to a single entity rather than a pointer to an array of entities.
So, while using an ECB “batches” a bunch of structural changes, in that it defers execution of structural change commands you add to it until a later point in the application, the actual execution of those structural changes is handled via a sequential series of method calls to the EntityComponentStore. The good news is that, because you’ve deferred the structural changes, you’re not introducing new multithreading bottlenecks to your application. The bad news is that the actual execution of the (now deferred) structural changes will be slower than if you had introduced a new multithreading bottleneck and used EntityManager.
Awesome. That clarifies a lot then. Thanks! I look forward to reading your more extensive write-up when you post it.
I guess the open question that I have is whether Unity has any plans to add deferred batched structural change functionality (that is, batching structural change operations with an ECB)? Or is this either too much of a corner case (hard to believe) or are there other technical considerations/constraints for this to be added into the base Unity ECS feature set?
Well, they already use batched structural change operations for EntityQuery overloads in ECB same as in EntityManager, they process structural change tracking on all matched chunks once for this call. Currently, ECB does not support same for NativeArrays like an EntityManager, but at least EQ batched operations gives you big win in performance already (if you can use them as they cover not a big amount of cases )
Yup, I knew that the EQ batched option was available in ECBs, and that’s definitely not nothing. It’s just that as I said earlier, using an EQ provides somewhat less flexibility/control than passing in an explicit list of Entities. However, as you said, the EQ functionality probably covers a lot of cases where you’d use the NativeArray.
In the specific case I was looking at in my code, the EntityQuery would probably suffice. However one caveat with the EQ is that in the documentation for EntityCommandBuffer, the “Remarks” section for the EQ batch operations methods states “The query is performed at playback time, not when the method is called.”. This meant that I needed to take a closer look at whether there would be a difference in the EQ results at the time of the Job execution versus the ECB playback.
This isn’t a problem per say, but it does mean that if you’re using the EQ-based ECB batch methods, you need to keep in mind that the Entities operated on won’t be those that match your EQ at the time the Job runs, but rather at the time that the ECB plays back. This would be different from how the NativeArray-based methods would operate, because those would capture the entities at the time the Job runs. It’s a subtly that a developer would need to keep in mind as it could lead to some undesirable (and unexpected if one didn’t read the Remarks section) results.
I would not be surprised if Unity added more batched APIs for ECBs. However, merging commands to take advantage of batch processing during playback is another matter. I know Unity has tried, but I don’t think the flexibility of ECB lends itself to an efficient implementation. You really want a much more rigid data structure so that you don’t have to manually check if commands can be merged. But such a rigid data structure either sacrifices sequential equivalency or flexibility.
Hi,
You seem to be knowledgable about commandbuffers.
Would it be possible/pertinent to have a command buffer to add custom operation on components like have a Health component with a Add method that take a Health component and do the addition of both ?
I’m asking in regard to my ability system, I think it would help decouple the logic between the effect and the actual update of the component value. There may be some benefit to just queue the add command from several efffects and run all of them per component type at a single point in the system.
)