Reading from NativeMultiHashMap that is being written to by another system

Unity Engine
, ,
1

Hi all,

I’m struggling my way through ECS and slowly making progress, but am currently getting stuck with reading from a nativemultihashmap that is being written to by another system.
I’ve tried altering my setup so that I have two maps that are written to on alternate frames, and then reading from the one that isn’t being written that frame, but still receiving the same error.

This system writes each of my item entities into a grid:

protected override void OnUpdate()
        {
            var parallelHashMap = PrepareMapWrite().AsParallelWriter(); // Gets the map to be written to this frame as a parallel writer
           
            Entities.WithAll<ItemTag>().ForEach((Entity entity, in Translation translation) => {
                int key = GetHashKey(translation.Value.x, translation.Value.z);
                parallelHashMap.Add(key, new ItemHashData
                {
                    item = entity,
                    position = translation.Value
                });
               
            }).ScheduleParallel();

            mapOne = !mapOne; // Boolean used to determine whether to write to map 1 or 2
        }

And then this code attempts to read from the map so it can add the closest item to a dynamic buffer:

protected override void OnUpdate()
    {

        if(!Keyboard.current.fKey.wasPressedThisFrame)
        {
            return;
        }
        NativeMultiHashMap<int,ItemHashData> map = ItemHashSystem.GetMapRead(); // Gets the map that isn't being written to this frame
        Entities.WithoutBurst().WithAll<InventoryComponent>().ForEach((DynamicBuffer<PrefabListComponent> inventory, Entity entity, int entityInQueryIndex, in Translation invLocation) => {
            int hash = ItemHashSystem.GetHashKey(invLocation.Value.x, invLocation.Value.z);
            float distance = ItemHashSystem.cellScale * 3; // Just get a number bigger than the allowed cell size
            ItemHashData closest = new ItemHashData();
            if (map.TryGetFirstValue(hash, out ItemHashData i, out NativeMultiHashMapIterator<int> iterator))
            {
                closest = i;
                distance = math.distance(invLocation.Value, i.position);
           
                while (map.TryGetNextValue(out i, ref iterator))
                {
                    if(math.distance(invLocation.Value, i.position) < distance)
                    {
                        distance = math.distance(invLocation.Value, i.position);
                        closest = i;
                    }
                }

                 // Add the item to inventory bufffer and disable it, disabled until read is working correctly.
                /*if(!closest.Equals(default(ItemHashData)) && math.distance(closest.position, invLocation.Value) <= 2)
                {
                    // we do the pick up
                    inventory.Add(new PrefabListComponent { Prefab = closest.item } );
                    EntityManager.SetEnabled(closest.item, false);
                }*/
                Debug.Log($"Closest entity is {i.ToString()}");
            }
           
        }).Run();            
    }

When the second system runs, it immediately throws the below error:
InvalidOperationException: The previously scheduled job ItemHashSystem:<>c__DisplayClass_OnUpdate_LambdaJob0 writes to the Unity.Collections.NativeMultiHashMap`2[System.Int32,Inventory.ItemHashData] <>c__DisplayClass_OnUpdate_LambdaJob0.JobData.parallelHashMap. You must call JobHandle.Complete() on the job ItemHashSystem:<>c__DisplayClass_OnUpdate_LambdaJob0, before you can read from the Unity.Collections.NativeMultiHashMap`2[System.Int32,Inventory.ItemHashData] safely.

I had hoped moving to two NMHMs would stop it, but no luck. Every example I can find that is reading from NMHM involves older non SystemBase systems, and seem to work fine.

Is there a way to do this using SystemBase/Entities.ForEach, or do I need to mix and match the way my systems work (and/or wait for new features)

2

Your first system is schedule so it don’t run on the main thread, your second system use run, so it runs on hte main thread. BY the time you rech the second system the first one is not done so it’s still writing to the map.

You should try to use schedule for your second system and add the fist job handle to the second job so taht they wait on each other.

If they are in separate classes, you’ll need to had a private jobhandle in your second system class and a method to set it fro the first system.

3

Ah, I thought using UpdateAfter would be enough to ensure that the job would have completed.
Schedule/ScheduleParallel don’t return job handles, so I guess I’d need to re-write using job syntax rather than entities.fopreach, right?

Rethinking it though, I can set the second system to us UpdateBefore rather than UpdateAfter and it works with the data from the last frame.
I’m sure it’s not optimal, but it’s working well enough that I can move on to the next problem

4

It returns, not explicitly but through Dependency property on the system. Whole Entites.ForEach will be converted to IJobChunk by codegen in this case (look at DOTS Compiler inspector) which will use Dependency property as input dependency and output dependency.

5

Ah thankyou, I got it working based on dependencies without needing the second map, so thanks for the 50% memory savings!

posting the full classes for anyone having similar troubles, feel free to critique:

public struct ItemHashData
    {
        public Entity item;
        public float3 position;

    }
   
    public class ItemHashSystem : SystemBase
    {
        EntityQuery findItems;
        public readonly static float cellScale=10;
        public static NativeMultiHashMap<int, ItemHashData> ItemMap;        public static JobHandle hashDeps;
       
        protected override void OnCreate()
        {
            base.OnCreate();
            findItems = GetEntityQuery(new EntityQueryDesc()
            {
                All = new ComponentType[] {ComponentType.ReadOnly<ItemTag>(), ComponentType.ReadOnly<Translation>()}
            });
            ItemMap = new NativeMultiHashMap<int, ItemHashData>(0, Allocator.Persistent);
        }

        protected override void OnDestroy()
        {
            ItemMap.Dispose();
            base.OnDestroy();
        }


        public static int GetHashKey(float posX, float posZ)
        {
            return (int)(math.floor(posX / cellScale) + (math.floor(posZ / cellScale) * 1000));
        }

        protected override void OnUpdate()
        {
            var parallelHashMap = PrepareMapWrite().AsParallelWriter();
            hashDeps = Entities.WithAll<ItemTag>().ForEach((Entity entity, in Translation translation) => {
                int key = GetHashKey(translation.Value.x, translation.Value.z);
                parallelHashMap.Add(key, new ItemHashData
                {
                    item = entity,
                    position = translation.Value
                });
               
            }).ScheduleParallel(Dependency);
            Dependency = hashDeps;
        }

        private NativeMultiHashMap<int, ItemHashData> PrepareMapWrite()
        {
            int length = findItems.CalculateEntityCount();
            ItemMap.Clear();
            if(length > ItemMap.Capacity)
            {
                ItemMap.Capacity = length;               
            }
            return ItemMap;
        }

        public static NativeMultiHashMap<int, ItemHashData> GetMapRead()
        {
            return ItemMap;
        }               
    }

And this is my very basic “pickup” system that adds the entities to a dynamic buffer:

public class ItemTakeSystem : SystemBase
{
    BeginSimulationEntityCommandBufferSystem bufferSystem;
    World defaultWorld;

    EntityQuery findItems;
    protected override void OnCreate()
    {
        defaultWorld = World.DefaultGameObjectInjectionWorld;
        bufferSystem = defaultWorld.GetOrCreateSystem<BeginSimulationEntityCommandBufferSystem>();
        findItems = GetEntityQuery(new EntityQueryDesc()
        {
            All = new ComponentType[] {ComponentType.ReadOnly<ItemTag>(), ComponentType.ReadOnly<LocalToWorld>()}
        });

    }
   
    protected override void OnUpdate()
    {
        var commandBuffer = bufferSystem.CreateCommandBuffer().AsParallelWriter();
        if(!Keyboard.current.fKey.isPressed)
        {
            return;
        }
        Dependency = JobHandle.CombineDependencies(Dependency, ItemHashSystem.hashDeps);

        NativeMultiHashMap<int,ItemHashData> map = ItemHashSystem.GetMapRead();
        Entities.WithoutBurst().WithAll<InventoryComponent>().ForEach((DynamicBuffer<PrefabListComponent> inventory, Entity entity, int entityInQueryIndex, in Translation invLocation) => {
            int hash = ItemHashSystem.GetHashKey(invLocation.Value.x, invLocation.Value.z);
            float distance = ItemHashSystem.cellScale * 3; // Just get a number bigger than the allowed cell size
            ItemHashData closest = new ItemHashData();
            if (map.TryGetFirstValue(hash, out ItemHashData i, out NativeMultiHashMapIterator<int> iterator))
            {
                closest = i;
                distance = math.distance(invLocation.Value, i.position);
           
                while (map.TryGetNextValue(out i, ref iterator))
                {
                    if(math.distance(invLocation.Value, i.position) < distance)
                    {
                        distance = math.distance(invLocation.Value, i.position);
                        closest = i;
                    }
                }
                if(!closest.Equals(default(ItemHashData)) && math.distance(closest.position, invLocation.Value) <= 2)
                {
                    // we do the pick up
                    inventory.Add(new PrefabListComponent { Prefab = closest.item } );
                    commandBuffer.AddComponent(entityInQueryIndex, closest.item, new Disabled());
                }
            }
           
        }).Schedule();
    }
}
6

I’m now finding this technique does not seem to work “generically”, as attempting to turn my hashing system into a generic base where I override the entity query starts throwing errors whenever I try to work with the hashmap from a different system (I’ll have to post code later).

Weirdly the error I get indicates that I can’t write from a system because that system is already writing, but the system it indicates is doing the writing is only running a GetFirstValue. Looking at the stack trace the error actually originates from when the hash System clears the map, so I’m very confused.

I’d very much appreciate if someone could point out any issues in the above code that would stop the dependencies from working as I expect, and I’ll post the current code that’s erroring when I can.

7

This is my attempt at a generic hash base system to inherit from, unfortunately since Entities.ForEach doesn’t allow dynamic code the “RunForEach” function needs to be more verbose than I’d like.

Generic Hash Base:

using Unity.Collections;
using Unity.Entities;
using Unity.Jobs;
using Unity.Mathematics;
using UnityEngine;

namespace Inventory
{
   
    public struct GenericHashDetails
    {
        public readonly static float cellScale=10;
        public static int GetHashKey(float posX, float posZ)
            {
                return (int)(math.floor(posX / cellScale) + (math.floor(posZ / cellScale) * 1000));
            }
    }

     public struct EntityHashData
    {
        public Entity item;
        public float3 position;

    }
   
    [DisableAutoCreation]
    public class GenericHashBase : SystemBase
    {

        protected EntityQuery entityQuery;
        protected static NativeMultiHashMap<int, EntityHashData> EntityMap;
        protected static JobHandle HashJobDependency;
        protected override void OnCreate()
        {
            Debug.Log($"Creating {this.ToString()}");
        }

        protected override void OnUpdate()
        {
            // Only run if we have an entity query, otherwise disable system           
            if(entityQuery == default(EntityQuery))
            {
                Debug.LogWarning($"DISABLING SYSTEM: No query for {this.ToString()}");               
                this.Enabled = false;
                return;
            }
            else
            {           
                int length = entityQuery.CalculateEntityCount();
                EntityMap.Clear();
                if(length > EntityMap.Capacity)
                {
                    EntityMap.Capacity = length;
                }   
                HashJobDependency = RunForEach();
                Dependency = HashJobDependency;
            }
        }

        public static JobHandle GetOutputDependency()
        {
            return HashJobDependency;
        }
        protected virtual JobHandle RunForEach()
        {
            return Dependency;
            // Example hash loop commented below

            // var parallelHashMap = EntityMap.AsParallelWriter();
            // return Entities.WithName("Base_Hash").WithAll<UnusedTag>().ForEach((Entity entity, in Translation translation) => {
            //     Debug.LogWarning("Running Base ForEach");
            //     int key = GenericHashDetails.GetHashKey(translation.Value.x, translation.Value.z);
            //     parallelHashMap.Add(key, new EntityHashData
            //     {
            //         item = entity,
            //         position = translation.Value
            //     });
            // }).ScheduleParallel(Dependency);
        }

        public static NativeMultiHashMap<int, EntityHashData> GetMap()
        {
            return EntityMap;
        }

        // Dispose of the EntityMap if it's been created
        protected override void OnDestroy()
        {
            if(EntityMap.IsCreated)
                EntityMap.Dispose();
            base.OnDestroy();
        }
    }
}

This is the Interactible Hash System that inherits from the base:

using Unity.Burst;
using Unity.Collections;
using Unity.Entities;
using Unity.Jobs;
using Unity.Mathematics;
using Unity.Transforms;
using UnityEngine;

namespace Inventory
{
   
    public class InteractibleHashSystem : GenericHashBase
    {
        protected override void OnCreate()
        {
            entityQuery = GetEntityQuery(new EntityQueryDesc()
            {
                All = new ComponentType[] {ComponentType.ReadOnly<InteractibleComponent>(), ComponentType.ReadOnly<Translation>()}
            });
            EntityMap = new NativeMultiHashMap<int, EntityHashData>(0, Allocator.Persistent);
            base.OnCreate();
        }

        protected override JobHandle RunForEach()
        {          
            var parallelHashMap = EntityMap.AsParallelWriter();
            return Entities.WithName("Interactible_Hash").WithAll<InteractibleComponent>().ForEach((Entity entity, in Translation translation) => {
                int key = GenericHashDetails.GetHashKey(translation.Value.x, translation.Value.z);
                parallelHashMap.Add(key, new EntityHashData
                {
                    item = entity,
                    position = translation.Value
                });         
            }).ScheduleParallel(Dependency);
        }    
    }
}

And finally here is the Highlight system that is supposed to simply print a debug as to whether or not there are interactibles in the same hash location as the player (as a pre-cursor to highlighting nearby interactible objects):

using Unity.Collections;
using Unity.Entities;
using Unity.Jobs;
using Unity.Transforms;
using UnityEngine;

namespace Inventory
{
    public class HighlightInteractiblesSystem : SystemBase
    {
        protected override void OnUpdate()
        {         
            // Combine Dependencies
            JobHandle combined = JobHandle.CombineDependencies(Dependency, InteractibleHashSystem.GetOutputDependency());
           
            // Get the map and display whether we have items in the current hash location.
            var interMap = InteractibleHashSystem.GetMap();
            Dependency = Entities.WithAll<PlayerInputComponent>().ForEach((Entity entity, ref ItemInteractionComponent interactionComponent, in Translation translation) => {
                int hash = GenericHashDetails.GetHashKey(translation.Value.x, translation.Value.z);
                Debug.Log($"At Grid: {hash}. Items: {interMap.TryGetFirstValue(hash, out EntityHashData i, out NativeMultiHashMapIterator<int> iterator)}");

            }).Schedule(combined);
           

        }
    }
}

When running, the systems do actually work, and I get correct debugs printing out.
However, I also get incessant exceptions:

InvalidOperationException: The previously scheduled job HighlightInteractiblesSystem:<>c__DisplayClass_OnUpdate_LambdaJob0 writes to the Unity.Collections.NativeMultiHashMap`2[System.Int32,Inventory.EntityHashData] <>c__DisplayClass_OnUpdate_LambdaJob0.JobData.interMap. You must call JobHandle.Complete() on the job HighlightInteractiblesSystem:<>c__DisplayClass_OnUpdate_LambdaJob0, before you can write to the Unity.Collections.NativeMultiHashMap`2[System.Int32,Inventory.EntityHashData] safely.
Unity.Collections.LowLevel.Unsafe.AtomicSafetyHandle.CheckWriteAndBumpSecondaryVersion (Unity.Collections.LowLevel.Unsafe.AtomicSafetyHandle handle) (at <a9810827dce3444a8e5c4e9f3f5e0828>:0)
Unity.Collections.NativeMultiHashMap`2[TKey,TValue].CheckWrite () (at Library/PackageCache/com.unity.collections@0.12.0-preview.13/Unity.Collections/NativeMultiHashMap.cs:568)
Unity.Collections.NativeMultiHashMap`2[TKey,TValue].Clear () (at Library/PackageCache/com.unity.collections@0.12.0-preview.13/Unity.Collections/NativeMultiHashMap.cs:138)
Inventory.GenericHashBase.PrepareMapWrite () (at Assets/ECS/Generic/Systems/GenericHashBase.cs:68)
Inventory.InteractibleHashSystem.RunForEach () (at Assets/ECS/Items/Systems/InteractibleHashSystem.cs:26)
Inventory.GenericHashBase.OnUpdate () (at Assets/ECS/Generic/Systems/GenericHashBase.cs:55)
Unity.Entities.SystemBase.Update () (at Library/PackageCache/com.unity.entities@0.14.0-preview.19/Unity.Entities/SystemBase.cs:411)
Unity.Entities.ComponentSystemGroup.UpdateAllSystems () (at Library/PackageCache/com.unity.entities@0.14.0-preview.19/Unity.Entities/ComponentSystemGroup.cs:513)
UnityEngine.Debug:LogException(Exception)
Unity.Debug:LogException(Exception) (at Library/PackageCache/com.unity.entities@0.14.0-preview.19/Unity.Entities/Stubs/Unity/Debug.cs:19)
Unity.Entities.ComponentSystemGroup:UpdateAllSystems() (at Library/PackageCache/com.unity.entities@0.14.0-preview.19/Unity.Entities/ComponentSystemGroup.cs:518)
Unity.Entities.ComponentSystemGroup:OnUpdate() (at Library/PackageCache/com.unity.entities@0.14.0-preview.19/Unity.Entities/ComponentSystemGroup.cs:461)
Unity.Entities.ComponentSystem:Update() (at Library/PackageCache/com.unity.entities@0.14.0-preview.19/Unity.Entities/ComponentSystem.cs:107)
Unity.Entities.DummyDelegateWrapper:TriggerUpdate() (at Library/PackageCache/com.unity.entities@0.14.0-preview.19/Unity.Entities/ScriptBehaviourUpdateOrder.cs:333)

You’ll note the exception is claiming that HighlightInteractiblesSystem cannot write because it is already writing, despite that system only having a TryGetFirstValue, and also that further down the stacktrace it seems the error actually occurs when I attempt to clear the EntityMap from the InteractiblehashSystem.

I’ve tried so many different things at this point, and I have no idea why it doesn’t work when the first system I attempted did.
I suspect I am doing something wrong with dependencies, but am not certain what, or how to fix it.

Any help is much appreciated!

8

Entities.Foreach is code generated to an IJobChunk by the engine. using the map inside the ForEach labda expressions tells the engine to capture the variable to be used in the IJobChunk.
The code generation don’t acctually check for assignement in the landba expression to tell if the code write to the capture variable (if it was written to within a sub method it would most likely miss it or be too expensive to genertate the code).

So what you probably need to do is to add a WithReadOnly(variable) to your ForEach statement.

Look at this doc for more details :
https://docs.unity3d.com/Packages/com.unity.entities@0.14/manual/ecs_entities_foreach.html#capturing-variables

1 Like
9

Thanks, I think that is probably part of the puzzle, if not the full solution.
Error now says I can’t write until a read has finished:

InvalidOperationException: The previously scheduled job HighlightInteractiblesSystem:<>c__DisplayClass_OnUpdate_LambdaJob0 reads from the Unity.Collections.NativeMultiHashMap`2[System.Int32,Inventory.EntityHashData] <>c__DisplayClass_OnUpdate_LambdaJob0.JobData.interMap. You must call JobHandle.Complete() on the job HighlightInteractiblesSystem:<>c__DisplayClass_OnUpdate_LambdaJob0, before you can write to the Unity.Collections.NativeMultiHashMap`2[System.Int32,Inventory.EntityHashData] safely.
Unity.Collections.LowLevel.Unsafe.AtomicSafetyHandle.CheckWriteAndBumpSecondaryVersion (Unity.Collections.LowLevel.Unsafe.AtomicSafetyHandle handle) (at <a9810827dce3444a8e5c4e9f3f5e0828>:0)
Unity.Collections.NativeMultiHashMap`2[TKey,TValue].CheckWrite () (at Library/PackageCache/com.unity.collections@0.12.0-preview.13/Unity.Collections/NativeMultiHashMap.cs:568)
Unity.Collections.NativeMultiHashMap`2[TKey,TValue].Clear () (at Library/PackageCache/com.unity.collections@0.12.0-preview.13/Unity.Collections/NativeMultiHashMap.cs:138)
Inventory.GenericHashBase.OnUpdate () (at Assets/ECS/Generic/Systems/GenericHashBase.cs:50)
Unity.Entities.SystemBase.Update () (at Library/PackageCache/com.unity.entities@0.14.0-preview.19/Unity.Entities/SystemBase.cs:411)
Unity.Entities.ComponentSystemGroup.UpdateAllSystems () (at Library/PackageCache/com.unity.entities@0.14.0-preview.19/Unity.Entities/ComponentSystemGroup.cs:513)
UnityEngine.Debug:LogException(Exception)
Unity.Debug:LogException(Exception) (at Library/PackageCache/com.unity.entities@0.14.0-preview.19/Unity.Entities/Stubs/Unity/Debug.cs:19)
Unity.Entities.ComponentSystemGroup:UpdateAllSystems() (at Library/PackageCache/com.unity.entities@0.14.0-preview.19/Unity.Entities/ComponentSystemGroup.cs:518)
Unity.Entities.ComponentSystemGroup:OnUpdate() (at Library/PackageCache/com.unity.entities@0.14.0-preview.19/Unity.Entities/ComponentSystemGroup.cs:461)
Unity.Entities.ComponentSystem:Update() (at Library/PackageCache/com.unity.entities@0.14.0-preview.19/Unity.Entities/ComponentSystem.cs:107)
Unity.Entities.DummyDelegateWrapper:TriggerUpdate() (at Library/PackageCache/com.unity.entities@0.14.0-preview.19/Unity.Entities/ScriptBehaviourUpdateOrder.cs:333)

I’m thinking I’m running into a circular dependency, where the read job can’t happen while the write job is happening, and vice versa.
Is there any way to resolve that without making one of them Run()?

10

Yes you need to add the job handle of the 1st job to the second one.

Because your jobs are in 2 different systelm, yous have 2 options :

  • put both jobs in the same system and give the handle of the 1st job to the seconde one
  • create a jobhandle member to your second system set it to the job hanlde of hte 1st job from the 1st sustem and then use the member jobhandle as a combined dependancy in your 2nd job/system.

If it does not make a monster class I’d advise to use the 1st option which is much mode simple.

11

And the second option would get even more complex if I introduced a third system that read from the same hashmap.

The first way does indeed work immediately, so I’ll stick with that for now I think, and possibly refactor it later if dependency management becomes a bit more straightforward than it currently is.

Thanks for the help!

12

If you have one system to write and several that read, you should be able to read in parrallel from several job/system I think. then managing the dependancy is not so much an issue, reading system just need to get the handle form the writting system.

Anyway, glad it work and I could help.