I hope it’s OK to touch two topics in this case - main reason for that is that they’re IMHO tightly coupled.
I converted some heavy calculations of a project to jobs. Problem is, these jobs take really long time (cca 100 - 1000ms) which causes following problems:
Log warning “Internal: JobTempAlloc has allocations that are more than 4 frames old - this is not allowed and likely a leak” - this is printed for any job (including ones allocating nothing) if it lasts longer than 4 frames. This is really common in the project I’m working on.
Higher-priority, shorter jobs are blocked, because all worker threads are working on long-time jobs (the project generates lots of them).
And finally, some of these jobs actually are not that important as others. Yes, I can prioritize them by sorting them before scheduling; but what about jobs which need to be scheduled later? (Example: prepare 50 jobs, sort by priority, schedule. A few frames later: 10 of them are done. I have 20 new jobs, half of them have higher priority than half of the already scheduled ones.)
So, is there a way how to prioritize jobs? Is there a way how to reserve some worker threads for shorter, higher-priority jobs?
Note: the calculations which needs to be done are already separated into lots of jobs; what I’m trying to say, this is not case of “one single huge blob job doing everything”.
For now, it’s IJob. It could be distributed using IJobParallelFor, but I don’t see how it would help, because of dependencies. The current system works like this: j0 - j1 - j2 - j3 where j(i) is one job. j(i) depends on j(i-1) (except j0, obviously). The only one which can be distributed using IJobParallelFor is j2. The j3 depends on all sub-jobs of j2. Another thing is that this whole job string (j0 - j1 - j2 - j3) represents one “bigger task” and I have lots “job-strings” like that.
Edit: So even if j0 to j2 were separated into lots of smaller jobs, j3, depending on the previous one, would wait most of the time (up to 1000ms) until previous ones are finished, spawning the warning log.
So using IJobParallelFor would make one job shorter, yes, but it would not help in any way in prioritizing and with problem that it’s blocking other, higher-priority jobs.
I have a similar problem which can’t be solved with IJobParallelFor. I’m researching on IJobParallelForBatch but it’s not officially documented, and the page linked here points to an non-existent page, I haven’t found anything about it on the web yet… could you please paste a link to some documentation/example.
Disagree. If your jobs has large loops then IJobParallelFor (With Burst of course) give you HUGE execution speed and your job chain executes very fast. Anyway without some code snippets and core idea for why you use this chain, I can’t tell more.
If I understand the job system correctly, the main difference between IJobParallelFor and IJobParallelForBatch is only that in case of IJobParallelFor, the execute method means “calculate i-th item” which in case of IJobParallelForBatch it means “execute from i-th to j-th item”. That can be useful when one item is processed very quickly (so overhead for calling Execute is significant) or when one iteration of your algorithm needs to work on multiple items at a time.
In other words, I don’t see how either of them helps with the problem. Or, if there is a way how one of them helps, then I believe the other would help, too, in same way.
I can try that. Actually, I’m going to anyway. But it’s going to take a lots of time and I simple don’t believe that it would give me that big execution speed (like from 1000ms to 50ms). That’s why I created this thread - to find out, whether it’s worth trying or whether to leave job system.
Everything is about processing data. At some point, I receive data A. I want to transform them to data B. It’s very computationally expensive. Jobs j0 and j1 pre-process the A. j2 does the main work. j3 post-process the data preparing them for integration on main thread. Time to do j0 to j2 takes lots of time.
Each job process the data in little bit different domain (doesn’t make sense to implement them all in one job).
This is just pseudo-code:
class App
{
DataProcessorChain m_dataProcessorChain;
// Called at the initialization time
void InitializeSystem()
{
// prepare instances of DataProcess a, b, c and d
m_dataProcessorChain = new DataProcessorChain ();
m_dataProcessorChain.AttachDataProcessor(a);
m_dataProcessorChain.AttachDataProcessor(b);
m_dataProcessorChain.AttachDataProcessor(c);
m_dataProcessorChain.AttachDataProcessor(d);
}
// Updates running system
void UpdateSystem()
{
var newlyReceivedData = FetchData();
if(newlyReceivedData != null)
{
foreach(DataBatch data in newlyReceivedData)
{
m_dataProcessorChain.ScheduleProcessingData(data);
}
}
m_dataProcessorChain.IntegrateOnMainThread();
}
List<DataBatch> FetchData()
{
// Receives data to process or return null if there are none
}
void IntegrateData(DataBatch processedData)
{
// After the data were processed, here they are integrated in main thread
}
}
class DataProcessorChain
{
// List of data which are being processed
List<ProcessedData> m_waitingForDataProcessing = new List<ProcessedData>();
void ScheduleProcessingData(DataBatch data)
{
JobHandle jobHandle = default(JobHandle);
foreach(DataProcessor dataProcessor in m_dataProcessors) // m_dataProcessors contains a, b, c and d
{
jobHandle = dataProcessor.ScheduleProcessingData(data, jobHandle);
}
m_waitingForDataProcessing.Add(new ProcessedData(data, jobHandle));
}
void IntegrateOnMainThread()
{
for(int i = 0; i < m_waitingForDataProcessing.Count; )
{
ProcessedData processedData = m_waitingForDataProcessing[i];
if(processedData.jobHandle.IsCompleted)
{
m_waitingForDataProcessing.RemoveAt(i);
// In order to make native containers available on main thread
processedData.jobHandle.Complete();
// Takes the processed data and integrates with the rest of the game
app.IntegrateData(processedData.data);
}
else
{
++i;
}
}
}
}
// This is actually abstract class and a, b, c and d are instances of different classes deriving from the main
// base class implementing different steps of processing the data.
// This pseudo-code just shows what each of them does in general.
class DataProcessor
{
JobHandle ScheduleProcessingData(DataBatch data, JobHandle dependency)
{
Job job = new Job()
{
m_fieldA = data.m_fieldA, // input
m_fieldB = data.m_fieldB, // output
};
return job.Schedule(dependency);
}
struct Job : IJob
{
[ReadOnly]
NativeContainer m_fieldA;
[WriteOnly]
NativeContainer m_fieldB;
void Execute()
{
// Transform data from m_fieldA to m_fieldB
}
}
}
You could make sure that your Jobs are only using a specific number of worker thread by manually adding dependencies between the different long jobs and thus making one or more long-job-chains. The downside is, that even when there are no short Jobs the other CPU cores will not work on the long Jobs.
As the worker thread count is fixed to the number of cores and IJob can not be interrupted to prevent context switches a good solution using the Job framework is probably not possible. For these kind of tasks I would suggest spawning actual Threads although I’m pretty sure Unity will present a solution for this in the future.
A terrible waste of a jobs, why for each data unit create 4 jobs? Why don’t get all feched data to one native array, put this array in to m_dataProcessorChain.ScheduleProcessingData(data); without foreach, and after all just create 4 parallel jobs (for each processor)
That would help with my problem, but to make it optimal, the number of jobs has be based on the number of cores your processor has, not fixed in 4, do you know if there’s a way to know the number of cores/processors available?
I understand and I guess 4 is a safe assumption, but if you had 8 cores and you only scheduled 4 jobs you will be missing the chance to get some extra performance.
Absolutley not, as i see you not understand. I say 4 because topic starter have 4 different data processors with different logic. If one parallel job (IJobParallelFor) can do all job he must be one, and more parallel jobs absolutley don’t give performance gains, on the contrary, there will be a loss on the scheduling of the jobs. Repeat again - IJobParallelFor splits on ALL avaliable worker threads (if resources on this WT free)
Thanks for the suggestions. But it seems to me that burst optimizations won’t compensate penalties caused by that.
I see, so they’re going to fix that. That’s a good news.
But my problem is that the warning spawn even when I do allocate nothing. Just empty job with loop doing nothing and long enough to take more than 4 frames triggers the warning. That’s what I was trying to say with “this is printed for any job (including ones allocating nothing) if it lasts longer than 4 frames” in original post.
But as I wrote above; if they’re going to fix that, it’s OK.
Jobs j0 to j3 on one DataBatch represent one task which must be done. These tasks have different priorities and I need some of them to be finished sooner then the others. If I should put all the data together, I would have to wait not ~1s for first batch to be done, but ~20s when all of them are finished. Another problem is that integration to main thread is expensive. The most expensive part is creating colliders so it’s separated into several frames. So the current flow is:
Schedule batch #1, batch #2, #3, etc. to ~#20
After ~1s when batch #1 is finished, fetch data (jobHandle.Complete())
Now, each frame the data are partially integrated (one collider created + lot’s of different things done)
So at this moment, batch #1 is being integrated in main Unity thread while batch #2 is being processed by job system
When the batch is integrated, user can see the results and interact with it. It took ~2s.
Soon, batch #2 is completed and it’s integration begins while batch #3 is being processed by job system.
But with flow you suggest, it would be:
Schedule batches #1 - #20 in one single job string (j0 - j3)
Wait ~20s until last job (j3) is completed
Now, begin integrate the data.
So at this moment, batch #1 is being integrated in main Unity thread while job system does nothing.
When first batch is integrated, user can see the results and interact with it. It took ~21s.
And I would like to remind that result of j(i) depends on j(i-1) (except j0). So, for example, j1 a j2 cannot run parallel for one data batch.
You don’t need to think like that. Loot at the Execute method of IJobParallelFor; it takes an index. Now look at the Schedule method; it takes number of iterations (array length) and batch size (innerloop batch count). Now, what does it mean:
You have, let’s say, 100 items you want to process.
Create one (!) job (deriving/implementing IJobParallelFor) and when scheduling, set array length parameter to your item count (100).
Now Unity job system will call Execute method 100 times with index (parameter of Execute method) different each time: 0, 1, 2, …, 99.
Each call can be done from different working thread! Unity will split the range into different threads, so you don’t need to care about core count, you don’t need to care about job worker thread count, you don’t need to schedule multiple jobs.
Now, what’s batch size (innerloop batch count)? It means: when working thread starts processing your job, process N (= batch size) items in one run.
(Warning: big simplification follows.) So, for example, if the batch size (innerloop batch count) is set to 10, each worker thread will process 10 consecutive items in one loop. So if unity job system have 4 working threads, first thread will call Execute with index 0, 1, 2, …, 9, second thread will call Execute with 10-19, third 20-29 and last 30-39. When first thread is done (depends on many factors), it starts calling Execute with indices 40-49 and so on and so until all items are processed. If unity job system have 8 working thread, all of them will start processing you job (first with indices 0-9, second 10-19, etc. until last 70-79).
How to choose the batch size (innerloop batch count)? It’s up to you. Main factor is - how long does it take to process one item (how long one Execute(index) will take)? If it’s fast, it’s good to use big batch size in order to decrease internal overhead (when a worker thread starts processing some range, it must notify other threads so they can work on different range). If processing one item takes longer, it’s better to use smaller batch size so all items can be distributed across all threads.