NativeMultiHashMap<TKey, TValue>.ParallelWriter don't support concurents writes accross jobs

Unity Engine
,
1

Hello,

I’m making a system where I have a job that loop through elements in a native queue and dispatch the element in a NativeMultiHashMap based on some attribute of the queue element.

I expected using the NativeMultiHashMap.ParallelWriter to be able to schedule each job in parallel but it seems taht container only support concurent writes within the same job, not accross job.

            NativeArray<JobHandle> MapperJobHanldes = new NativeArray<JobHandle>(CommandsQueues.Count, Allocator.Temp);
            for (int i = 0; i < CommandsQueues.Count; i++)
            {
                MapperJobHanldes[i] = new MapCommands()
                {
                    CommandsMap = CommandsMap.AsParallelWriter(),
                    CommandsQueue = CommandsQueues[i]
                }.Schedule(AfterResize);
            }
            MapperJobHanldes.Dispose();

When a schedules the jobs squencially, it works fine.

           JobHandle AfterMap = AfterResize;
            for (int i = 0; i < CommandsQueues.Count; i++)
            {
                AfterMap = new MapCommands()
                {
                    CommandsMap = CommandsMap.AsParallelWriter(),
                    CommandsQueue = CommandsQueues[i]
                }.Schedule(AfterMap);
            }

Is there a way to work around that ? Or the fact that each job could potentially write to the same key of the map at the same time is not handled by the ParallelWriter ?

2

Try use the same ParallelWriter, instead of making new ones each time you Schedule a new job.

            NativeArray<JobHandle> MapperJobHanldes = new NativeArray<JobHandle>(CommandsQueues.Count, Allocator.Temp);
            var ParallelWriter = CommandsMap.AsParallelWriter();
            for (int i = 0; i < CommandsQueues.Count; i++)
            {
                MapperJobHanldes[i] = new MapCommands()
                {
                    CommandsMap = ParallelWriter,
                    CommandsQueue = CommandsQueues[i]
                }.Schedule(AfterResize);
            }
            MapperJobHanldes.Dispose();

ParallelWrite from different job is Okay. It’s just By calling AsParallelWriter from NativeMultiHashMap you are creating multiple safetyHandle to the same container.
If it is still not working consider use UnsafeHashMap.

3

Thanks, for your answer.
Did not work with NativeMultiHashMap , had to use UnsafeMultiHashMap.

4

For you job add this [Unity.Collections.LowLevel.Unsafe.NativeDisableContainerSafetyRestriction]
to your ParallelWriter

1 Like
5

Works too :smile:

The job

 [BurstCompile]
        struct MapCommands : IJob
        {
        
            public NativeQueue<COMMAND> CommandsQueue;

            [WriteOnly]
            [NativeDisableContainerSafetyRestriction]
            public NativeMultiHashMap<ulong, COMMAND>.ParallelWriter CommandsMap;

            public void Execute()
            {
                COMMAND command;
                while (CommandsQueue.TryDequeue(out command))
                {
                    CommandsMap.Add(command.RegistryReference.TypeId, command);
                }

            }
        }

The scheduling

  NativeArray<JobHandle> MapperJobHanldes = new NativeArray<JobHandle>(CommandsQueues.Count, Allocator.TempJob);
            var CommandsMapParallelWriter = CommandsMap.AsParallelWriter();
          
            for (int i = 0; i < CommandsQueues.Count; i++)
            {
                MapperJobHanldes[i] = new MapCommands()
                {
                    CommandsMap = CommandsMapParallelWriter,
                    CommandsQueue = CommandsQueues[i]
                }.Schedule(JobHandle);
            }
6

I had this same problem and I believe the solution is that the ParallelWriter versions of Native Containers are supposed to be used with IJobFor, not IJob. That’s what solved it for me.