Burst compile error and IJobEntity dependency troubleshooting

Unity Engine
, , ,
1

Hey everyone,

I am working with ECS for quite a while now, so I am definitely not a beginner. Just yesterday I ran into a baffling burst compile error, that I just can’t seem to track down, so I was hoping I can get some help here.

The error message I am getting:

(0,0): Burst error BC1091: External and internal calls are not allowed inside static constructors: Interop.BCrypt.BCryptGenRandom(System.IntPtr hAlgorithm, byte* pbBuffer, int cbBuffer, int dwFlags)

While compiling job:
Unity.Entities.JobEntityBatchIndexExtensions+JobEntityBatchIndexProducer`1[[Tellas.Storylets.ECS.Systems.FindAvailablePersonsJob, Tellas.Storylets, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null]], Unity.Entities, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null::Execute(Unity.Entities.JobEntityBatchIndexExtensions+JobEntityBatchIndexWrapper`1[[Tellas.Storylets.ECS.Systems.FindAvailablePersonsJob, Tellas.Storylets, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null]]&, Unity.Entities, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null|System.IntPtr, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089|System.IntPtr, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089|Unity.Jobs.LowLevel.Unsafe.JobRanges&, UnityEngine.CoreModule, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null|System.Int32, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089)
Unity.Entities.JobEntityBatchIndexExtensions+JobEntityBatchIndexProducer`1[[Tellas.Storylets.ECS.Systems.RemoveDuplicatesJob, Tellas.Storylets, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null]], Unity.Entities, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null::Execute(Unity.Entities.JobEntityBatchIndexExtensions+JobEntityBatchIndexWrapper`1[[Tellas.Storylets.ECS.Systems.RemoveDuplicatesJob, Tellas.Storylets, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null]]&, Unity.Entities, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null|System.IntPtr, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089|System.IntPtr, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089|Unity.Jobs.LowLevel.Unsafe.JobRanges&, UnityEngine.CoreModule, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null|System.Int32, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089)

The system in which the error is supposedly happening:

using Tellas.Character.ECS.Components;
using Tellas.Core.ECS;
using Tellas.PhysicsEngine.ECS.Components;
using Tellas.Storylets.ECS.Components;
using Unity.Burst;
using Unity.Collections;
using Unity.Entities;
using Unity.Transforms;

namespace Tellas.Storylets.ECS.Systems
{

    [UpdateInGroup(typeof(TellasLateSimulationSystemGroup))]
    public partial class FindAvailablePersonsSystem : SystemBase
    {
        EndSimulationEntityCommandBufferSystem commandBufferSystem;

        protected override void OnCreate()
        {
            commandBufferSystem = World.GetExistingSystem<EndSimulationEntityCommandBufferSystem>();
        }

        protected override void OnUpdate()
        {
            EntityCommandBuffer ecb = commandBufferSystem.CreateCommandBuffer();

//Can be parallelized
            Entities
               .WithName("RemoveAvailablePersonTag")
               .WithAll<Tag_AvailablePerson>()
               .WithStructuralChanges()
               .ForEach((Entity e) =>
                {
                    //
                    EntityManager.RemoveComponent<Tag_AvailablePerson>(e);
                }).Run();

//not sure how to parallelize - remove and recreate efficient enough?
            Entities
               .WithName("ClearAvailableToTalk")
               .WithAll<AvailableToTalk>()
               .WithAll<AvailableToTalk>()
               .WithStructuralChanges()
               .ForEach((Entity e) =>
                {
                    //
                    EntityManager.GetBuffer<AvailableToTalk>(e).Clear();
                }).Run();

            Dependency = new FindAvailablePersonsJob
                {
                    canTalks = GetComponentDataFromEntity<CanTalk>()
                  , persons = GetComponentDataFromEntity<Person>()
                  , availablePersons = GetComponentDataFromEntity<Tag_AvailablePerson>()
                  , ecb = ecb.AsParallelWriter()
                }.ScheduleParallel(Dependency);

            Dependency = new RemoveDuplicatesJob
                {
                    ecb = ecb.AsParallelWriter()
                }.ScheduleParallel(Dependency);

            commandBufferSystem.AddJobHandleForProducer(Dependency);
        }
    }

    [BurstCompile]
    [WithAll(typeof(Tag_FindAvailablePersons))]
    [WithAll(typeof(Tag_PhysicsTrigger))]
    [WithAll(typeof(CollisionTriggerResult))]
    public partial struct FindAvailablePersonsJob : IJobEntity
    {
        [ReadOnly] public ComponentDataFromEntity<CanTalk> canTalks;
        [ReadOnly] public ComponentDataFromEntity<Person> persons;
        [ReadOnly] public ComponentDataFromEntity<Tag_AvailablePerson> availablePersons;

        public EntityCommandBuffer.ParallelWriter ecb;

        public void Execute(Entity e, [EntityInQueryIndex] int entityInQueryIndex, in Parent parent, in DynamicBuffer<CollisionTriggerResult> collisionResults)
        {
            Entity parentEntity = parent.Value;

            for(int i = 0; i < collisionResults.Length; ++i)
            {
                CollisionTriggerResult triggerResult = collisionResults[i];

                Entity other = triggerResult.a == e ? triggerResult.b : triggerResult.a;
                if(!canTalks.HasComponent(other) || !persons.HasComponent(other))
                    continue;

                if(!availablePersons.HasComponent(other))
                    ecb.AddComponent<Tag_AvailablePerson>(entityInQueryIndex, other);

                AvailableToTalk element = new(other, persons[other]);

                ecb.AppendToBuffer(entityInQueryIndex, parentEntity, element);
            }
        }
    }

    [BurstCompile]
    [WithAll(typeof(AvailableToTalk))]
    public partial struct RemoveDuplicatesJob : IJobEntity
    {
        public EntityCommandBuffer.ParallelWriter ecb;

        public void Execute(Entity e, [EntityInQueryIndex] int entityInQueryIndex, ref DynamicBuffer<AvailableToTalk> buffer)
        {
            ecb.RemoveComponent<AvailableToTalk>(entityInQueryIndex, e);
            DynamicBuffer<AvailableToTalk> newBuffer = ecb.AddBuffer<AvailableToTalk>(entityInQueryIndex, e);

            NativeParallelHashSet<AvailableToTalk> set = new(buffer.Length, Allocator.Temp);
            for(int i = 0; i < buffer.Length; ++i)
            {
                set.Add(buffer[i]);
            }

            int j = 0;
            newBuffer.Length = set.Count();
            foreach(AvailableToTalk availableToTalk in set)
            {
                newBuffer[j++] = availableToTalk;
            }

            set.Dispose();
        }
    }

}

The only constructors being called are on the AvailableToTalk dynamic buffer element and the hashset for duplicate removal (if there is a faster way, please let me know!)

AvailableToTalk looks like this:

[InternalBufferCapacity(8)]
    public struct AvailableToTalk : IBufferElementData, IEquatable<AvailableToTalk>
    {
        // Actual value each buffer element will store.
        public Entity value;
        public Person person;

        public AvailableToTalk(Entity value, Person person)
        {
            this.value = value;
            this.person = person;
        }

        public bool Equals(AvailableToTalk other)
        {
            return value.Equals(other.value) && person.Equals(other.person);
        }

        public override bool Equals(object obj)
        {
            return obj is AvailableToTalk other && Equals(other);
        }

        public override int GetHashCode()
        {
            return HashCode.Combine(value, person);
        }

        public static bool operator ==(AvailableToTalk left, AvailableToTalk right)
        {
            return left.Equals(right);
        }

        public static bool operator !=(AvailableToTalk left, AvailableToTalk right)
        {
            return !left.Equals(right);
        }
    }

And Person is simply an IComponentData with a single int for an ID, nothing more.

I really don’t understand where the error is coming from. I assume it is in the codegen from IJobEntity, but even there I don’t see anything specific.

Before I moved the code to an IJobEntity from a Entities.ForEach it was working fine without errors, but it was also running without burst originally, so no clue if the problem was already occuring before or not.

Any help is greatly appreciated!
Loofou

2

It’s probably the HashCode.Combine call in AvailableToTalk.GetHashCode (the hash set should rely on GetHashCode to function). The .NET HashCode struct as implemented by the BCL Unity uses probably does some magic stuff in its static constructor (considering that the current version on dotnet/runtime main does so), so you’d want to derive your own hashing for that struct, or just use an IDE function to make one.

1 Like
3

Hm right, let me try to combine manually via prime multiplication.

Edit:

Yeah that one should’ve been obvious. Damn you code-completion automation :smile:

Thanks for the hint on that, the error is gone. Unfortunately I get another error now, but I hope I can figure this one out myself.

The system Tellas.Storylets.ECS.Systems.FindAvailablePersonsSystem reads Tellas.PhysicsEngine.ECS.Components.CollisionTriggerResult via FindAvailablePersonsJob but that type was not assigned to the Dependency property. To ensure correct behavior of other systems, the job or a dependency must be assigned to the Dependency property before returning from the OnUpdate method.

It’s a bit weird because I am adding both jobs to the Dependency field…

4

Yeah I can’t figure this one out either. All jobs are added to the Dependency field, I even removed the main-thread entities.foreach temporarily and it still complains. The only way to get it to stop is to not use ECBs at all and run everything on the main thread, but I refuse to go down that route unless there is really no other option.

Usually when this error occured I did some weird stuff with jobhandle combines that it could not detect properly, but in this case everything is super straight forward so I really don’t get it :confused:

After writing 50 systems without any big issues, this one is really getting on my nerves :smile:

5

It’s the HashCode.Combine - it uses some managed static’s within the implementation, which Burst cannot handle. We need to improve that error though (at least give you a callstack that caused it perhaps).

1 Like
6

Right, thanks for confirming. And yes, a callstack would’ve been extremely helpful :slight_smile:
Replacing the HashCode.Combine with a manual hashing worked. I think I will replace the auto-creation template from Rider with a more burst-friendly approach.

Right now I ran into a follow-up error (see my last message), which I also never had like this before. I’m sure there is a similar simple solution, but from my knowledge I can’t figure it out.

7

That looks like a job system thing or something like that, way outside my wheelhouse (they only let me out to talk to users about Burst :smile: ). I’ll try ask around though!

1 Like
8

Haha, no problem at all :smile: much appreciated!

9

@Loofou Can you try making a local edit to Unity.Entities/SystemState.cs?

        internal void LogSafetyErrors()
        {
            if (!JobsUtility.JobDebuggerEnabled)
                return;

            var depMgr = m_DependencyManager;
            if (SystemDependencySafetyUtility.FindSystemSchedulingErrors(m_SystemID, ref m_JobDependencyForReadingSystems, ref m_JobDependencyForWritingSystems, depMgr, out var details))
            {
                bool logged = false;
                LogSafetyDetails(details, ref logged);

                if (!logged)
                {
                    Debug.LogError("A system dependency error was detected but could not be logged accurately from Burst. Disable Burst compilation to see full error message.");
                }

                m_DependencyManager->Safety.PanicSyncAll(); /// <--------Remove this line
            }
        }

I know we have a bug fix yet to be released where safety errors were being suppressed making the “be sure to assign to Dependency” message appear instead of the real safety issue appearing. That could be the case here (only taking a quick glance at your code)

10

Thanks for the answer! How can I make a local edit to this file without Unity redownloading the package every time I try to save?

Edit:
Nevermind, found it. (here, for anyone else looking: Reddit - Dive into anything)

This does indeed change the error to

[ERR] [UNITY] "InvalidOperationException: The previously scheduled job FindAvailablePersonsJob writes to the Unity.Entities.EntityCommandBuffer FindAvailablePersonsJob.JobData.ecb. You must call JobHandle.Complete() on the job FindAvailablePersonsJob, before you can write to the Unity.Entities.EntityCommandBuffer safely."

So does that means I need one command buffer per job? I never knew that, always just reused the same one, but it makes sense. I hope the order is still dependent on the Dependency handle if I use two ecbs. I will try this.

Edit2:

I think I got it. My second job is reading the buffer I’m writing in the job before, but I’m not actually waiting for the ecb to be finished. I should probably find a better way to remove duplicates from my buffer :smile:

Thanks everyone for the help! Much appreciated!

2 Likes
11

8409780--1111209--upload_2022-9-2_10-32-50.png
Your issue is ecb.AsParallelWriter In first job you call it and pass to job, and schedule, after that point any read\write access of ecb (safety checks inside AsParallelWriter check if someone write to ecb) - not allowed by safety system. What you should do in that case (if jobs depends on each other) is - declare ecbParallel right before first job and assign ecb.AsParallelWriter(), and just pass ecbParallel to both of your jobs.

1 Like