EntityCommandBuffer and Dynamic Buffers

Entity Component System (ECS)
1

Is it possible to interact with dynamic buffers through an EntityCommandBuffer?

I’m hoping to do things like add/remove IBufferElementData’s at specific positions in the buffer.

Thanks for any advice.

2

You don’t need to use ECB in order to interact with the buffer as you want. There are methods to do what you want like myDynamicBuffer.RemoveAt(index) It’s advised to get the buffer on the entity with IJobProcessComponentDataWithEntity and add the field for your buffer to the Job struct.

[NativeDisableParallelForRestriction]
public BufferFromEntity<MyBuffer> buffers;

inputDeps = new MyJob{
            buffers = GetBufferFromEntity<TargetKnowledge>(),
        }.Schedule(this, inputDeps);
4 Likes
3

You probably have seen it, but just in case
** [Example] IJobProcessComponentDataWithEntity and IBufferElementData **

2 Likes
4

@riskparitygawd & @Antypodish , thank you very much for the leads. :slight_smile:

From what I’ve read, GetBufferFromEntity() doesn’t play nice with IJobParallelFor, since you can’t guarantee that parallel jobs wouldn’t be appending (or otherwise writing to) a dynamic buffer at the same time.

I’m sure you know this, but EntityCommandBuffer avoids this issue by queuing entity operations, and deferring them until later.

It would be nice to have a similar way to queue and defer dynamic buffer operations, so that they happen deterministically, back on the main thread.

Am I correct in thinking this isn’t currently possible? Most of the components in my project are IBufferElementData’s, fwiw. So I’m potentially looking at never/rarely being able to use IJobPerallelFor. Please correct me if I’m wrong.

It feels a bit incongruent to work one way for IComponentDatas and another way with IBufferElementDatas. It would be nice to use the same logical approach for both types of components.

Thanks!

5

If you can guarantee you won’t be writing to different buffers from different threads you just use

[NativeDisableParallelForRestriction]

This should work fine 99% of cases because you usually only access a buffer of the entity that it belongs.

2 Likes
6

Thanks! I must admit I am a bit hesitant to go that route. @Joachim_Ante_1 wrote a good 4 sentences of caution before discussing it back in August. :stuck_out_tongue:

I understand what he’s saying here, and if you can absolutely guarantee nothing is writing to the Dynamic Buffer, it should be fine.

I may just need to live with the idea for a bit, and rule out other options, and then eventually become comfortable and use it. :stuck_out_tongue:

7

What I’d really like to do is defer the work until later. For example, I’d like to run 40 jobs in parallel, which each give a deferred command to append an IBufferElementData to the Dynamic Buffer. Then later, in a BarrierSystem, those commands would be carried out in a deterministic way on the main thread. Same model as with IComponentData work.

…This is a complete shot in the dark, but maybe with the source available for EntityCommandBuffer, one could write their own version that works with DynamicBuffers. Something like a “DynamicBufferCommandBuffer”, or just extension methods for EntityCommandBuffer. I’m not at a computer where I can take a look, so maybe this is crazy talk. :smile:

8

shrug

I have 23 uses of NativeDisableParallelForRestriction in my project. I don’t consider it risky at all as it’s very obvious when you won’t have race conditions (though I am quite confident writing parallel code so that may play into it.)

If you’re using IJobChunk or IJobProcessComponentData it’d be pretty rare for it not to be safe.

I’m not sure what type of algorithm you’d be using with IJobParallelFor because it’s so rarely used now except for specialized cases that aren’t usually 1 to 1 entity related in which case it probably isn’t safe.

9

I’ll look into this. Thanks.

Really, IJobParallelFor is rarely used now? I didn’t know this- so much of what I’ve read on the forums seemed to imply it was the future, whenever possible.

10

There’s no real reason to use IJobParallelFor anymore because it’s replaced for most use cases by IJobChunk or IJobProcessComponentData

-edit-

was curious about my own project uses of IJobParallelFor

IJobParallelFor: 5%
IJobChunk: 8%
IJobProcessComponentData: 87%

1 Like
11

My project I am working on, I use both IJobParallelFor (if I got just large arrays) and IJobProcessComponentDataWithEntity.
Also I use plenty of buffers with NativeDisableParallelForRestriction.

1 Like
12

Thank you. It is hard to keep track sometimes, not having been tuned in for those changes when they happened. :slight_smile:

Something I left out before, fwiw: I want to run many jobs in parallel which would each write to the same dynamic buffer. I specifically want these jobs to add things to it, so I imagine [NativeDisableParallelForRestriction] can’t apply.

That might lead someone to say “Ok, well then your jobs shouldn’t be running in parallel.”. Point taken, but with something like EntityCommandBuffer, you can do this type of thing with IComponentDatas. I imagine that using the same deferred command approach would solve this problem for writing to DynamicBuffers as well. Please correct me if I’m wrong.

13

What’s your use case for this? I’m struggling to think of a good one.

14

Here’s a quick, dirty version. I hope it’s detailed enough. Thank you for considering it:

  • Entity A represents an Ai Agent.
  • Entities B-D represent that agent’s brains.
  • System E reads state information about the brains, and add components to the agent, representing its active behaviors.
  • Systems F-J read those behavior components, and carry out the behaviors.

It’s valid for an agent to be performing more than one instance of the same behavior. That means multiple instances of the same behavior component need to coexist on the same entity. So those behaviors are defined as IBufferElementDatas, rather than IComponentDatas.

System E should ideally process all of the brains, for every agent, in parallel. So, there could be multiple jobs scheduled by System E which would be adding new behavior components to the same dynamic buffer of a single agent entity.

If that’s just not possible, then I could have System E not run its jobs in parallel. That would work, but it would be slower. Or, I could reorganize the System so that it runs a separate parallel job for each agent, etc.

Or I could just restrict the design, and say agents can only perform one instance of the behavior at a time. That’s also an option, but I’d like to avoid it if possible.

15

Ah the infamous AI system.
I’ve stayed out of this discussion because I don’t like the solution you’ve gone with; it’s far from what I’ve done and would advocate and others have been extremely helpful.
To me, it seems to run into way way too many issues like this and feels like your constantly battling ECS rather than working with it.

1 Like
16

Let’s talk more in PM. I respect your opinion and would love to know your thoughts, if you’re up for it.

17

I assume it was
** Porting a Generic Behavior System to ECS **
Somehow was for me looking complicated, so I also didn’t intervened, or I maybe was tired / busy.

18

I recommend instead of trying to buffer writes directly to component changes, create intermediate Entities that represent changes. This is what I plan to do and has the following benefits:

  • Multiple systems can read from the state and use EntityCommandBuffers to write out the desired changes with basically no overlap.
  • Separate pass can resolve conflicting changes, although if system processing queries are built correctly this should never happen and can be reasonably eliminated.
  • Can run various effects based on the pending change entities before they’re applied and deleted, such as spawning / despawning stuff, playing a particle system, cutting to certain animations, etc.

You can have a setup like so:

…other game logic… → AI Process → PendingAIChangeBarrier → Scan changes and generate effect events → ApplyAIChangeBarrier → …other game logic…

1 Like
19

Yes, that was a start. And it looks too complicated to me too. :slight_smile: You can tell by the post dates in that thread that the design was changing quickly day to day. That didn’t stop - I just stopped posting. :stuck_out_tongue:

What I’m implementing now is a different animal. It has changed in large part due to the excellent advice on these forums. Nearly every time someone has told me I was “fighting ECS”, that resulted in a refactor based on their advice. :slight_smile:

But of course it could still be bad. I have sincerely benefited from the critiques. You guys have been awesome.

1 Like
20

That’s fine.

1 Like