I am porting to Unity 2020.1a, Entities 0.1.1.
I used until now successfully
[Unity.Collections.LowLevel.Unsafe.NativeSetThreadIndex]
private int threadIndex ; // Starts from 1 to n threads.
Which returned value 1 to 16, for my 8 core processor.
But after moving to Unity 2020.1, now my threadIndex is 29.
There is however an info, when hovering over NativeSetThreadIndex, stating,
“This index is guaranteed to be unique to any other job that may be running in parallel.”
My understanding is that applies only to the job that NativeSetThreadIndex is within.
Not the other jobs, or other systems’ jobs, which my run in parallel. But on other hand, that only explanation, why my threadIndex is greater than 16. Yet it shouldn’t
That explains the reason for issue.
Any other way, to get actual processor logical core index of currently running job?
One thing I don’t understand is (not that important), why I got that error now and never had it before.
I explained other day one of my use case, but is useful for example, when summing up some values of processed entities in a job.
For example, to calculate sum of taken damage in a frame, when 10k entities are processed in IJobChunk. Then having an array of size, of number of logical cores and sum all damage per logical core index.
When job is done, is just quit for loop summation of elements in an array, to get final result.
Alternatively, I can run in such case an array, to store sums, per chunk index. So array is as large, as count of chunks.
I also used it in case of IJobParallelFor, when iterated through whatever number of elements, but increasing value of relevant index in array. Then just quick sum up.
As I said, it worked somehow in my previous use cases So I would like have good alternative, when possible.
While generally a short lived thread will probably stick to the same core, threads can change cores when needed and it’s the OS responsibility to schedule threads to cores.
I believe NativeSetThreadIndex is fine to use for the example use case you gave, as long as you use JobsUtility.MaxJobThreadCount for the array size. If that doesn’t work then there may be a bug somewhere, because Unity’s own container code uses the same idea (for example in NativeQueue’s implementation).
Thx, I get the concept. But I am not sure, if thread can change core, once is running particular job. So lets say having 10k elements in IJobParallelFor. Then having set Schedule batch count to 100, for simplicity.
I would assume, that each thread running batch of 100, once is running, does not switch logic processor.
But could switch, after batch of 100 is done, when picking another batch.
Yep, for now, I indeed decided to set array size equal to MaxJobThreadCount, which by default in my case is 128.
I think this is the safest route so far. Ideally I would keep that 128 threads max count lower, but is not that big deal breaker. I don’t want to break something by accident
That’s not how a CPU / OS works. Simply output the logical core in a job every now and then and it’s easy to show. Even just an IJob, which is only a single thread, will change cores hundreds of times throughout it’s execution.
[DllImport("Kernel32.dll"), SuppressUnmanagedCodeSecurity]
public static extern int GetCurrentProcessorNumber();
void Start()
{
this.handle = default(Job).Schedule();
}
private struct Job : IJob
{
public void Execute()
{
for (var i = 0; i < 500; i++)
{
var prime = this.FindPrimeNumber(i);
Debug.Log($"Thread Id = {Thread.CurrentThread.ManagedThreadId}, CoreId = {GetCurrentProcessorNumber()}");
}
}
private long FindPrimeNumber(int n)
{
int count=0;
long a = 2;
while(count<n)
{
long b = 2;
int prime = 1;// to check if found a prime
while(b * b <= a)
{
if(a % b == 0)
{
prime = 0;
break;
}
b++;
}
if(prime > 0)
{
count++;
}
a++;
}
return (--a);
}
}
-edit- anyway this was a bit off topic but i think it’s always good to know what’s going on.
it seems you’ve got a solution for your original problem which is great!
Uuuu. That is very interesting. I would never suspect for that.
This maybe beyond the scope of discussion, but why thread is jumping between cores? Wouldn’t be more optimal, to stay on single core, until job is done?
A very short, poorly explained version (I’m a terrible teacher)
Unity does not have any (minimal) control over or cares about what CPU core you’re executing on.
One of if not the primary purpose of an operating system is to schedule tasks (threads to the CPU.)
Thread aren’t executed till completion otherwise if you had 4 core cpu you could only run 4 applications at a time!
So different threads are sliced by the OS scheduler.
So a thread is not being worked on and another core becomes idle, why would you not send it there rather than wait for the original core to finish processing!
Thx @tertle , this definitely enlightened me a bit more.
This thread surely allowed me for better understanding and to prevent running into potential issues, with unexpected data overriding. Could be hard to track down. At least until we maybe will get feature, to toggle multi/single threading of the job/system, for debugging purposes.
Regarding cores and threads, more like starts at 25. Before is more about graphics. But yes nice talk about performance. Is actually mentioned earlier discussed points, regarding jumping threads between cores.
Let me resume this thread, as I have a question about MaxJobThreadCount.
Is that the maximum number of threads that the JobSystem spins up in total (and not per job) ? I hope so. I actually believe that 128 is high , could be lower, although surely number of CPU cores are actually growing so it’s future proof.
What is the relationship between JobIndex and the NativeThreadIndex? What I need is an unique identify per thread, so I can be sure that datastructures are safe in their own thread, but with 128 possible threads, seems I have to create 128 datastructures.
If you use IJobParallelForBatch than you can calculate batchcount through length / batchSize. Real batch count of course will be lessequal then your value, but if your batch count lower then 128 then you can use batch count dependent job version instead of threadIndex version. This is the most of i can invent
yeah batch count is a different problem than what I wanted to solve, also I never understood if I really need to do that or DOTS can work out a good batch size on its own. I have my own strategy anyway.
So I would like have good alternative, when possible.
