Example of AsyncReadManager.Read for multiple files in Burst Job?

Unity Engine
, ,
1

Hi!

I am looking for examples regarding AsyncReadManager.Read to read multiple files from within a burst job.

  • What is the idea/concept/benefit of using multiple ReadCommands vs a single large read command in AsyncreadManager.Read? Was designed with having one large database file in mind and maintaining all indices/offsets within this file our own and then working on subranges within this file?

  • How to effiicently load multiple files with AsyncReadManager?

  • Is FixedString{Size} the correct new string struct to be used to pass the file path into a bursted job?

2

When streaming entity binary files we want to stream directly into the chunks. Those might not have contiguous memory addresses. Thus we will read a file linearly but with multiple read commands.

Larger read commands are always better, wherever that is possible.

1 Like
3

Thank you for the explanation. That makes a lot of sense.

Is there any plan for a variant of AsyncManager.Read that does support any of the string types as filename that is supported via Burst? Currently I am getting (expected as strings are not fully support per documentation)

    [BurstCompile]
    public struct ReadAsyncFileJob : IJobParallelFor
    {
        public NativeArray<ReadCommand> ReadCommands;

        [ReadOnly]
        public NativeArray<FixedString512> Paths;

        [WriteOnly]
        public NativeArray<ReadHandle> ReadHandles;

        public unsafe void Execute(int index)
        {
            ReadHandles[index] = AsyncReadManager.Read(Paths[index].ToString(), (ReadCommand*) ReadCommands.GetUnsafePtr() + sizeof(ReadCommand) * index, 1);
        }
    }

[Error] E:\repositories\visionary\Visionary\Assets\Plugins\AsyncFileJobs\AsyncFileJobs.cs(22,4): Burst error BC1016: The managed function `Unity.Collections.FixedString512.ToString(Unity.Collections.FixedString512* this)` is not supported
at Plugins.AsyncFileJobs.ReadAsyncFileJob.Execute(Plugins.AsyncFileJobs.ReadAsyncFileJob* this, int index) (at E:\repositories\visionary\Visionary\Assets\Plugins\AsyncFileJobs\AsyncFileJobs.cs:22)
at Unity.Jobs.IJobParallelForExtensions.ParallelForJobStruct`1<Plugins.AsyncFileJobs.ReadAsyncFileJob>.Execute(ref Plugins.AsyncFileJobs.ReadAsyncFileJob jobData, System.IntPtr additionalPtr, System.IntPtr bufferRangePatchData, ref Unity.Jobs.LowLevel.Unsafe.JobRanges ranges, int jobIndex)
While compiling job: System.Void Unity.Jobs.IJobParallelForExtensions/ParallelForJobStruct`1<Plugins.AsyncFileJobs.ReadAsyncFileJob>::Execute(T&,System.IntPtr,System.IntPtr,Unity.Jobs.LowLevel.Unsafe.JobRanges&,System.Int32)
at E:\repositories\visionary\Visionary\Assets\Plugins\AsyncFileJobs\AsyncFileJobs.cs:line 22
4

@Joachim_Ante_1 Any update reading blob files from BurstedJobs?