Create an array, list or NativeArray WITHIN a job?

Unity Engine
,
1

I have a collection of entities, within that group I will have a variable range of them that match a criteria. I then need to choose a random one from the resulting subset.

eg:
A, 1
A, 2
B, 1
B, 2
B, 3

So I loop through looking for all instances where the first column equals “A”. Then I need to choose one of those.

Thing is, I don’t know how many matches I have until I loop through the entire parent NativeArray. So I would have to loop through multple times, the first time to get a count, and then a second time to place the entities within the new subset NativeArray.

Any thoughts?

Rant: Can we PLEASE get more precise query abilities like SQL, as this could solve SO many issues… anyway, I know, this is early… I am only asking to let staff know someone thinks this is important)

2

Is the EntityQueryDesc not what your looking for ?

https://docs.unity3d.com/Packages/com.unity.entities@0.0/manual/component_group.html

3

Thanks kindly, but no.

I am using an EntityQuery to get the full set of objects, I need a way to store a subset of those objects INSIDE the Execute.

4

that won’t be sufficient if the conditions for validity aren’t purely archetype-related

1 Like
5

Well said.

I hope that future versions of EntityQuery will have a Filter ability, that would be helpful.

6

I don’t think that doing the filtering with an SQL-style or Linq-style query in an EntityQuery would be efficient, though. Unless maybe codegen is on the table, which it already is for Entities.ForEach. So you never know!

But one solution to your problem would be using NativeQueue, and 2 jobs:

  • Job 1 iterates through all entities and stores the valid ones in a NativeQueue (supports parallel writing)
  • Job 2 knows the total count, gets a random nb from that count, and dequeues until that random one is reached (this one can’t be parallelized though)

Maybe we can avoid requiring 2 jobs, but that depends on the capabilities of NativeStream. I asked the question here:
https://forum.unity.com/threads/is-it-okay-to-write-and-then-read-a-nativestream-inside-the-same-job.778292/

7

Thanks, but that isn’t really a good answer. I will give it some real thought on how I would handle it, but it’s not really gonna be a good way to get there.

8

how come?

Using multiple jobs shouldn’t be regarded as “bad”. On the contrary, it can sometimes be the most efficient solution

9

The Reader/Writer might, the problem there is each entity archetype has two int values, and then an array (IBufferElementData) of float3, so I don’t know if the Reader/Writer will resolve this requirement.

Detail:
I have a traffic system, using Easy Roads 3D (nice asset, great support) Each Road has Lanes, each Lane has a series of points. Intersections can lead from one lane to another, so when the Auto gets to the end of a road’s points I need to get a list of Intersections that have the same starting Lane and Road as the Auto’s current Road/Lane. Because there can be multiple Exit Roads, and multiple Exit Lane on each road, I need the ability to get a subset of all connections.

Each Auto has it’s own set of current points (this avoids having to go to the Road/Connection points for each frame. (a post I recently made asking about duplicate data vs speed and a rep from Unity mentioned that duplicate data would be more performant) In either case, it’s not each frame that I need to get the next collection, so the multiple jobs FEELS like a nightmare of spaghetti code. I will think about it, see if I can find a way to get it working

10

Here’s the thing:
Multithreading comes with an unavoidable set of challenges that you’ll have to face regardless of if you’re using DOTS or any other language/engine

There’s nothing stopping you from doing non-jobified stuff in DOTS if you don’t want to deal with the challenges/limitations of multithreading. Non-jobified logic will just have the same performance as something that you’d implement using monobehaviour (maybe even a bit better)

In DOTS, the worst-case scenario is the standard every-day scenario that we had in monobehaviour

1 Like
11

You can create any array or NativeContainer within job actually. What they said is you can pass in NativeContainer & cannot pass in array, not you can’t create them in job.

1 Like
12

Care to show me how to create a NativeArray and add values within a job? (I’m not above begging)

13

Sure, there is nothing special, just do it like normally.

var hits = new NativeList<int>(1, Allocator.Temp);
hits.Add(2);

btw, the allocator must be temp.

2 Likes
14

Ok, the problem I have is that I don’t know how many I have until I traverse the entire set of entities. Running the loop the first time to get a count, and then again to fill the array SEEMS expensive. Is it?

15

That’s what NativeList is for. Unlike NativeArray, it can grow as needed.

1 Like
16

Really? AWESOME. Thanks, I’ll look into it.

Have a great weekend

17

What about using a NativeMultiHashMap?

You have to manage it manually. Thereby each car is a key and has multiple Linepoints or each line end is a key and has multiple connected lines…
Something like that. This way you can access a subset really fast. You just have to care about keeping it up to date.

18

I do not know about NativeMultiHashMap, never even heard about it til right now so thank you very much. I’ll look into it.

Have a wonderful weekend

1 Like
19

I’m facing the same problem here.
I’m looking for a way to generate multiple data vectors within parallel jobs and return them to the main thread which is quite tricky.
I tried the NativeMultiHashMap but it doesn’t offer any easy way to get some kind of (key: values) and using the
GetKeyValueArrays is very inefficient it eats ~0.3ms for 400 Elements.
So I’m trying to give some tries with NativeStream container hoping I can found what I’m looking for.
Unity is saying that we can store multiple arrays inside a single stream.( I hope I’m not wrongly interpreting what thay are saying)

A deterministic data streaming supporting parallel reading and parallel writing. Allows you to write different types or arrays into a single stream.
please correct me if I’m wrong.
Thanke you

20

I am not returning data from within the Job to the main thread. I am creating data/entities on the main thread and have used those entities within the job.

Here is the source code for what I did: GitHub - Blissgig/Easy-Road-3D-ECS-Traffic: Unity DOTS/ECS traffic using Easy Roads 3D for the data

The data is created in ER3D_Traffic.cs and used inside ER3D_TrafficSystem.cs I understand this is the opposite of what you are looking for, but maybe seeing how it works in this manner could be a clue on how to get that data back.

1 Like