So I have several jobs which are currently in different system, which much execute after each other, in order. For example these two, in order:
Serializer.Job
PacketWriter.Job
This means that Serializer.Job must be finished before I start PacketWriter.Job, currently I’m getting this warning when I’m trying to start the PacketWriter.Job in it’s system.
InvalidOperationException: The previously scheduled job Serializer`1:Job writes to the NativeArray Job.data.NetworkedComponentType. You must call JobHandle.Complete() on the job Serializer`1:Job, before you can read from the NativeArray safely.
So if i put inputDeps.Complete() in the Update() of the Serializer system, this of course works. What is the recommended approach here? Should I put all the jobs in the same system, and first queue up the Serializer.Job and then hand the JobHandle from those to the PacketWriter.Job ?
Edit: The reason I’m unsure of how to do about this is because the PacketWriter job doesn’t have access to the JobHandle from the Serializer system.
You can use the UpdateBefore/After attributes on your component systems to specify the order in which components updated (relative to other components). As to the dependencies, if you use the JobComponentSystem types, it should automatically handle the dependencies between systems.
You will have to do an equivalent of JobComponentSystem manually, that is you have to pass dependency between systems like @M_R said. JCS also pass JobHandle around different systems in some way, the same as your requirement right now :
Just that JCS has a kind of JobHandle “bus” operating under them. Now you will have to create one for your manual update loop, maybe have a bus over them instead. (Like, after calling Update manually a public getter property enforced by a new interface should be set, then the loop runner get that out for use, or make a new class inherit from ComponentSystem and change OnUpdate to new name OnUpdate which requires returning JobHandle )
JCS keep one JobHandle as an internal field too so it can complete itself in the next frame. You could do the same but expose that for the loop runner.
The JobComponentSystem.Update() method handles all job dependency setup for you. So that should be totally fine.
If you use JobComponentSystem correctly then it is expected that any dependency chains will be setup correctly independent of the order in which you call JobComponentSystem.Update();
Thats the fundamental design. Now if you have shared containers that are not IComponentData etc.
Eg. some NativeArray<> being passed around then you have to also pass around the job handles manually.
Yeah… I have a lot of these, a few NativeArray but manually allocated memory with UnsafeUtility.Malloc. Basically I have a case where I need to start a simulation tick, all jobs that do the simulation can have their dependancy chains setup using the JobComponentSystem.Update() mechanism, but once the simulation is done I need to make sure this case holds:
Make sure all systems doing the simulation are done
Run a set of jobs in exact sequence, which basically are: SerializeJob => DeltaCompressJob => WritePacketJob, and the rest of the simulation can’t run at this point.
My current setup works like this:
// perform simulation
foreach (var manager in _simulationManagers) {
manager.Update();
}
// make sure simulation completes
_entityManager.CompleteAllJobs();
// run Serializer, DeltaCompressor, PacketWriterJobs and then PacketSend jobs, in order.
_networkManager.Update();
// make sure all neworking stuff from _networkManager is done, then we are done with this tick
_entityManager.CompleteAllJobs();
It seems to work, is this correct? My native allocated memory is ONLY touched inside of the _networkManager.Update() so I can gurantuee it’s a ordering and parallelized split which I know is safe.
If you have NativeArrays that are writable from multiple jobs, and you are yourself somehow certain that two jobs writing to the same native array in parallel is totally safe then you want to use [NativeDisableContainerSafetyRestrictionAttribute] on all the container fields in jobs.
This disables the checks for those specific container / job and thus you can schedule your job in parallel. This does mean that you need to cover all safety aspects. For example NativeArray.Dispose() called on the main thread in parallel to your job will not be detected and result in memory corruption instead of a useful error message when you use [NativeDisableContainerSafetyRestrictionAttribute].
Thanks so much for this confirmation, sorry for all of these questions but when digging deep currently there’s very little documentation (not complaining, just trying to justify asking so much on the forums!).
Also I do not use NativeArray because I need to be able to pass nested arrays, so have my own array type called ‘UnsafeArray’ which is untyped and does not have a dispose sentintel so I can put several of it inside of a regular NativeArray.
So I built a ref-“counted” (not really, it uses reference indices that need to be returned to it) UnsafeArray, like this:
public unsafe struct UnsafeArray : IDisposable {
[NativeDisableUnsafePtrRestriction]
void* _ptr;
Int64 _refs;
Int32 _length;
IntPtr _typeHandle;
public Int32 Length {
get { return _length; }
}
public UnsafeArray(Type type, Int32 length) {
Assert.Check(UnsafeUtility.IsBlittable(type));
// byte size of the memory we need
var byteSize = length * UnsafeUtility.SizeOf(type);
_refs = 0;
_length = length;
_typeHandle = type.TypeHandle.Value;
// allocate memory
_ptr = UnsafeUtility.Malloc(byteSize, 4, Unity.Collections.Allocator.Persistent);
// clear memory
UnsafeUtility.MemClear(_ptr, byteSize);
}
public void Dispose() {
Assert.Check(_refs == 0);
if (_refs != 0) {
throw new InvalidOperationException("All refs have not been released");
}
if (_ptr != null) {
// free memory
UnsafeUtility.Free(_ptr, Unity.Collections.Allocator.Persistent);
// clear pointer
_ptr = null;
_length = 0;
_typeHandle = default(IntPtr);
}
}
public T Get<T>(Int32 index) where T : struct {
Assert.Check(typeof(T).TypeHandle.Value == _typeHandle);
if (index >= 0 && index < _length) {
return UnsafeUtility.ReadArrayElement<T>(_ptr, index);
}
else {
throw new IndexOutOfRangeException();
}
}
public void Set<T>(Int32 index, T value) where T : struct {
Assert.Check(typeof(T).TypeHandle.Value == _typeHandle);
if (index >= 0 && index < _length) {
UnsafeUtility.WriteArrayElement<T>(_ptr, index, value);
}
else {
throw new IndexOutOfRangeException();
}
}
public UnsafeArrayRef AllocRef(Int32 refIndex) {
Assert.Check(refIndex != 0);
if ((_refs & (1L << refIndex)) == 0) {
// claim reference index
_refs |= (1L << refIndex);
// return reference
return new UnsafeArrayRef(_ptr, _length, _typeHandle, refIndex);
}
throw new InvalidOperationException(String.Format("Reference {0} has already been allocated", refIndex));
}
public UnsafeArrayRef AllocRef() {
for (Int32 i = 1; i < 64; ++i) {
if ((_refs & (1L << i)) == 0) {
return AllocRef(i);
}
}
throw new InvalidOperationException("All 63 refs have been allocated");
}
public void ReleaseRef(ref UnsafeArrayRef reference) {
if ((_refs & (1L << reference.Index)) != 0) {
// release reference
_refs &= ~(1L << reference.Index);
// clear reference also
reference = default(UnsafeArrayRef);
}
}
}
public unsafe struct UnsafeArrayRef {
[NativeDisableUnsafePtrRestriction]
void* _ptr;
Int32 _length;
Int32 _refIndex;
IntPtr _typeHandle;
public Int32 Length {
get { return _length; }
}
public Int32 Index {
get { return _refIndex; }
}
public Boolean IsValid {
get { return _ptr != null && _refIndex != 0; }
}
public UnsafeArrayRef(void* ptr, Int32 length, IntPtr typeHandle, Int32 refIndex) {
_ptr = ptr;
_length = length;
_typeHandle = typeHandle;
_refIndex = refIndex;
}
public T Get<T>(Int32 index) where T : struct {
Assert.Check(typeof(T).TypeHandle.Value == _typeHandle);
if (index >= 0 && index < _length) {
return UnsafeUtility.ReadArrayElement<T>(_ptr, index);
}
else {
throw new IndexOutOfRangeException();
}
}
public void Set<T>(Int32 index, T value) where T : struct {
Assert.Check(typeof(T).TypeHandle.Value == _typeHandle);
if (index >= 0 && index < _length) {
UnsafeUtility.WriteArrayElement<T>(_ptr, index, value);
}
else {
throw new IndexOutOfRangeException();
}
}
}
This is obviously not a general solution, but it works for my specific use case. I’m 100% aware of how easy it is to trample memory using this.
The initial problem in my first post in the thread is solved, that was just a mistake on my end. The current problem I’m having is basically this:
I have X types of jobs that need to execute after one another, but the memory that needs to be protected so first one set of jobs writes to it and then another set of jobs reads from it is allocated manually inside of my UnsafeArray, and I could not figure out how to make the JobHandle system register a dependancy on this.
I solved it by putting all job types inside of the same system, and schedule it in X batches with a JobHandle.CompleteAll after each batch, basically:
Schedule serializer jobs, call JobHandle.CompleteAll on the handles for these jobs
Schedule delta compressor jobs, call JobHandle.CompleteAll on the handles for these jobs
Schedule packet writer jobs, call JobHandle.CompleteAll on the handles for these jobs again
This was my only solution, since I could not figure out how to attach a dependancy to JobHandle for a piece of manually allocated memory.
I dont understand, what it has to do with the UnsafeArray or the containers used in the jobs in general…
You should just remove all JobHandle.Complete / JobHandle.CompleteAll with passing the same JobHandle to the second job as a dependency and then the third job with a dependency on the second job.
Usually for modularity reasons you have some kind of class / struct that is referenced by all 3 systems that stores the active JobHandle so you essentially use it as dependency then change the dependency to the just scheduled job. Thus creating a chain with every step that you introduce.
I’ll try to explain, I’ll do a simple two-job example and not straight from my code here try to make it clear what I’m trying to do, it’s possible that I’m confused on how JobHandle exactly detectes job dependencies.
The first job, writesdata into UnsafeArray instances
The second job, readsdata fromthe same UnsafeArray instances
They have [ReadOnly] only dependancy on a bunch of actual ECS components, but there’s no writing to any ECS components at all.
If these jobs are scheduled with dependancies on each other, will they not execute at the same time? Because there is no conflict from the job systems point of view (i.e. they are only [ReadOnly] on all ECS related data)?
This is where I’m confused: How to get the job system to pick up on that these two have a conflicting dependency on manually allocated memory, so it can make them run after one another.
But will this automatically make the jobs wait for each other, even if they don’t have any conflicting dependencies on any ECS data from the job systems perspective?
Edit: It’s possible i’m assuming that the JobHandles are ‘too smart’ and simply work like regular wait handles? So if you but a dependancy on one then that has to be completed before the job that depends on it will start, no matter what’s getting read/written from the ecs components.
The job system itself does what is declared in dependencies.
The Job debugging system (Which is emitting errors) checks that dependencies of all scheduled jobs follow the rule that containers may only be written / read from if there is no other job writing to it in parallel. So if a container is not declared (Unsafe Ptr or [NativeDisableContainerSafetyRestrictionAttribute]) then the safety system does not know anything about it thus it will allow the two jobs to run in parallel without raising an error.
Job dependencies are enforcing ordering on the job queue. Its a gurantee that the job you depend will only begin execution once the dependent job has completed.
The advantage of job dependencies vs the Schedule->Complete->Schedule pattern is that it creates no stall on the main thread. The general goal is that the main thread simply schedules all jobs and at the end of the frame calls .CompleteAll(); Thats not always possible but thats optimally where you want to end up.
The dependancies between the jobs is what enforces ordering (essentially they are wait handles).
The job debugger system additionally on top of this makes sure that two jobs without dependancy on each other which writes/reads from the same data can’t run at the same time (logging the errors I’ve seen when I eff this up).
Okey this is much more what I’m used to, I assumed (wrongly so) that there was more magic with JobHandle/Dependencies and they could somehow figure out their internal execution order depending what they needed to read/write to, thanks for your patience and the time you took to answer me!