Unknown Size Intermediary NativeList Sizes for Jobs

Ziddon · April 3, 2022, 4:38am

Hi, I’ve got a question on a topic that’s totally stumped me.

I have an IJobParallelForDefer that I run in consecutive groups (~7 to 12 jobs).
Each Job in the group takes the previous one as a dependency.

Right now I’m prepping the NativeLists that get passed to each respective and guessing how large I think they might be.
Each list’s potential size depends on the result of the previous job.

 for (int i = 0; i < LODs; i++)
        {
            nodesToEvaluate.Add(new NativeList<ChunkNode>(currentEvalPotential, Allocator.Persistent));
            currentEvalPotential = math.min(30000, currentEvalPotential * 8);
        }

This is working excellently for speed, but atrociously for memory.

On the low end, we might have a size of a few hundred.
On the extreme end, it could be in the 2097152 (this number would be quite surprising, but possible lol. I don’t want to allocate that much memory each time I run this! XD). I guess I should note that I can have a lot of these job groups running concurrently as well, so the memory usage is not trivial, but could be very manageable if I’m not over-allocating.
My guestimate approach is terrible, because if I don’t guess high enough I crash, but even if I guess decently I’m wasting tons of memory.

I know I could just wait for each Job to complete in an OnUpdate and assign the next NativeList and schedule the next consecutive Job, but I don’t want to wait for the OnUpdate to schedule the next job if possible (If I’m being silly here just let me know).
I’d like to Allocate memory for each respective NativeList as a result of the previous job the same way I’m Scheduling the number of iterations of each job based on the result of the previous using a DeferredJobArray in the IJobParallelForDefer Job.

Is such a thing possible, or do you guys have any suggestions for how I might approach this issue?

Thanks!

DreamingImLatios · April 3, 2022, 5:58am

NativeList has an AsDeferredJobArray() function which lets it be treated as an array in a job with a length captured at the start of the job. You can combine this with an intermediate IJob which calls ResizeUninitialized() on it.

Ziddon · April 4, 2022, 4:56am

Hey DreamingImLatios, thanks for the response!
I’m trying to understand ResizeUninitialized, which says in the docs, “Changes the list length, resizing if necessary, without initializing memory.”

I tried using ResizeUninitialized inside a Job to set the NativeList size. It does work, but with 2 major problems.
The first is it seems to break Unity. Every time I try it it’ll work a couple times, then I’ll get a completely unique error (the error changes each time) and Unity crashes. The other problem is that it sets the actually size of the NativeList instead of allocating memory like I need it to. I was able to solve the size problem by calling Clear() after ResizeUninitialized.

Here’s my test System. Maybe there’s something obvious I’m doing incorrectly.

using Unity.Collections;
using Unity.Entities;
using Unity.Jobs;
using UnityEngine;

//allocates 8 times the last items length to new items
public struct AllocateMemory : IJob
{
    [ReadOnly]
    public NativeList<int> lastItems;
    public NativeList<int> newItems;
    public void Execute()
    {
        Debug.Log("previous had" + lastItems.Length + " resizing to: " + lastItems.Length * 8);
        newItems.ResizeUninitialized(lastItems.Length * 8);
        newItems.Clear();
    }
}

//randomly random number of items, up to 8, to new items for each item in last items
public struct FillList : IJobParallelForDefer
{
    [ReadOnly]
    public NativeArray<int> lastItems;
    public NativeList<int>.ParallelWriter newItems;
    public void Execute(int index)
    {
        for (int i = 0; i < 8; i++)
        {
            int random = Unity.Mathematics.Random.CreateFromIndex((uint)index).NextInt(0, 11);
            if (random < lastItems[index])
            {
                newItems.AddNoResize(random);
            }
        }
    }
}

public class TestingDeferred : SystemBase
{
    const int innerLoopBatchCount = 1;

    NativeList<int> initialItems;
    NativeList<int> items1;
    NativeList<int> items2;
    NativeList<int> items3;

    JobHandle finalHandle;

    bool complete = false;

    // Start is called before the first frame update
    protected override void OnCreate()
    {
        initialItems = new NativeList<int>(8, Allocator.TempJob);
        initialItems.Add(5);
        initialItems.Add(4);
        initialItems.Add(3);
        initialItems.Add(2);
        initialItems.Add(1);
        items1 = new NativeList<int>(Allocator.TempJob);
        items2 = new NativeList<int>(Allocator.TempJob);
        items3 = new NativeList<int>(Allocator.Persistent);
        //ROUND 1
        AllocateMemory allocateMemory1 = new AllocateMemory()
        {
            lastItems = initialItems,
            newItems = items1
        };
        JobHandle am1Handle = allocateMemory1.Schedule();
        FillList fillList1 = new FillList()
        {
            lastItems = initialItems.AsDeferredJobArray(),
            newItems = items1.AsParallelWriter()
        };
        JobHandle fl1Handle = fillList1.Schedule(initialItems, innerLoopBatchCount, am1Handle);

        //ROUND 2
        AllocateMemory allocateMemory2 = new AllocateMemory()
        {
            lastItems = items1,
            newItems = items2
        };
        JobHandle am2Handle = allocateMemory2.Schedule(fl1Handle);
        FillList fillList2 = new FillList()
        {
            lastItems = items1.AsDeferredJobArray(),
            newItems = items2.AsParallelWriter()
        };
        JobHandle fl2Handle = fillList2.Schedule(items1, innerLoopBatchCount, am2Handle);

        //ROUND 3
        AllocateMemory allocateMemory3 = new AllocateMemory()
        {
            lastItems = items2,
            newItems = items3
        };
        JobHandle am3Handle = allocateMemory3.Schedule(fl2Handle);
        FillList fillList3 = new FillList()
        {
            lastItems = items2.AsDeferredJobArray(),
            newItems = items3.AsParallelWriter()
        };
        JobHandle fl3Handle = fillList3.Schedule(items2, innerLoopBatchCount, am3Handle);

        initialItems.Dispose(fl1Handle);
        items1.Dispose(fl2Handle);
        items2.Dispose(fl3Handle);

        finalHandle = fl3Handle;
    }

    protected override void OnUpdate()
    {
        if (!complete && finalHandle.IsCompleted)
        {
            complete = true;
            finalHandle.Complete();
            Debug.Log("Final List Length of: " + items3.Length);
            items3.Dispose();
        }
    }
}

I’m afraid I may be stomping over memory @_@

The most common errors are

NullReferenceException: Object reference not set to an instance of an object
AssertionException: Assertion failure. Value was False
Invalid memory pointer was detected!
Unaligned memory is passed to ValidateIntegrity

Thanks again for the assistance!

DreamingImLatios · April 4, 2022, 5:28am

I don’t have enough energy to look through what might be wrong. But Capacity is what you actually want, not ResizeUninitialized.

Ziddon · April 4, 2022, 5:30am

Thanks. I’ve already tried changing capacity as well. It also causes Unity to implode.
Perhaps it’s not possible to safely allocate NativeList memory inside a Job (which is really my objective here, or allocate between scheduled jobs).

Certainly don’t feel obligated though, I know it’s a chunk of code. I’m not in any rush, perhaps somebody else will be able to lend a hand :). Or I’ll figure it out, lol XD.

DreamingImLatios · April 4, 2022, 6:11am

I’ve resized a NativeList inside of a job several times. It is legal, and I would share code here if my usages of it weren’t 1000 lines long. You are doing something else wrong.

eizenhorn · April 4, 2022, 7:06am

Depends on which Collections version you use, resizing NativeList in job can break it’s ParallelWriter. Which means your items1.AsParallelWriter will be broken immediately when you resize it in one of the previous jobs. If you use Collections before 1.0.0-pre.7 where Branimir have fixed that then you’ll see this bug.

The reason is how parallel writer was made:

As you can see it stores ListData->Ptr separately and just as pointer field and use it for all parallel list operations. As result when you resize original list in any way (For example by changing Capacity, it will call Realloc, on Add or Resize it will call Resize which is call Realloc under the hood), which is freeing old pointer and allocate new one, as result ParallelWriter version which stores pointer directly as a separate field will loose the reference to the actual data and will points to old memory which were freed and anyone already can use it, as result it will cause memory corrupt, and wouldn’t change original list content:

There is example of this bug

using Unity.Collections;
using Unity.Entities;
using Unity.Jobs;
using UnityEngine;

public class NativeListBug : SystemBase
{
    private NativeList<int> _list;

    private struct ParallelJobTest : IJobParallelFor
    {
        [WriteOnly] public NativeList<int>.ParallelWriter ListParallel;
      
        public void Execute(int index)
        {
            ListParallel.AddNoResize(5);
        }
    }
  
    protected override void OnCreate()
    {
        _list = new NativeList<int>(10, Allocator.Persistent);
    }

    protected override void OnDestroy()
    {
        if (_list.IsCreated)
            _list.Dispose();
    }
  
    protected override void OnUpdate()
    {
        var list = _list;
        var listParallel = _list.AsParallelWriter();

        Job.WithCode(() =>
        {
            list.Resize(20, NativeArrayOptions.ClearMemory);
        }).Schedule();
        Dependency = new ParallelJobTest()
        {
            ListParallel = listParallel
        }.Schedule(10, 1, Dependency);
      
        Dependency.Complete();

        Debug.Log($"Length:{_list.Length} Capacity:{_list.Capacity}");
        for (int i = 0; i < list.Length; i++)
        {
            Debug.Log(list[i]);
        }
      
        Debug.Break();
    }
}

Ziddon · April 4, 2022, 1:40pm

Thanks Eizenhorn for the in-depth explanation. That’s super helpful. I’ll take a deeper look at the code snippet and versions when I get home from work

Ziddon · April 5, 2022, 2:06am

I updated versions and it now works like a charm.

Thanks again for all the help!

Are there any performance implications I should be considering allocating memory from a job instead of from the main thread?

Topic		Replies	Views
Setting Length on a NativeList Unity Engine Entities , com_unity_entities	1	1902	October 23, 2018
NativeCollection references between scheduled jobs Unity Engine Entities , Question , com_unity_entities	7	1350	June 20, 2021
NativeList, Capacity and IJobParallelForDefer Unity Engine Entities , Question , com_unity_entities	3	1558	July 13, 2021
Using a NativeList as a NativeArray across jobs is generating an error. Unity Engine Entities , Question , com_unity_entities	5	2638	April 12, 2021
NativeList weird exception when clearing and resizing Unity Engine Entities , Question , com_unity_entities	5	1134	April 5, 2022

Unknown Size Intermediary NativeList Sizes for Jobs

Related topics