ArgumentException on native collectioons of elements containing a NativeSlice

Unity Engine
1

Our project currently can’t be upgraded to Unity 2019.3 because of a change in CollectionHelper that impacts restrictions on native collections.

Previously our collection elements could contain NativeSlice fields and worked fine up until Unity 2019.2 and we use it extensively in our project.

In Unity 2019.3 however instead of checking if (!UnsafeUtility.IsUnmanaged<T>()) it now checks if (!UnsafeUtility.IsValidNativeContainerElementType<T>()). This invalidates NativeSlices from being used.

If this is intended behaviour, what is the recommended workaround or migration path away from using them? We could use pointers or a new dummy native slice replacement that holds pointers in the same way, but I don’t want to work around the safety systems if there is a valid reason for the change that we may not be aware of.

2

if they are immutable, you can use blobs.

if they are mutable, store the indices and pass the original NativeArray as a separate job field
(then you can create the slice inside the job when needed)

3

They are the mutable output from several jobs and systems. They are a collection of the slices of delta changes to the array for network transmission.

4

then store the indices. you can always create a NativeSlice from that

5

That’s what we’re currently planning to do. There are a lot of other scattered similar usages in the project though, so we were hoping it was a bug. Refactoring all that code will be expensive. :frowning:

6

Here is one of our jobs as an example that is impacted by this. We can’t see any way to refactor this besides creating a new clone of the NativeSlice struct that doesn’t have the NativeCollection attribute or taking a huge performance hit.

does anyone else see a better path?

[BurstCompile]
        private unsafe struct GenerateSubchunkSlices : IJob {
            [ReadOnly] public NativeArray<ushort> SubchunkCounts; // The number of subchunks in each chunk, indexed per chunk.
            [ReadOnly] public NativeMultiHashMap<int, SubchunkInfo> Subchunks;
            public NativeArray<byte> Bytes; // The underlying data (allocated from totalLength in CountChunks)
            public NativeArray<NativeSlice<byte>> SubChunkSlices; // The backing array for the per-subchunk slice slices.
            [WriteOnly] public NativeArray<NativeSlice<NativeSlice<byte>>> SubChunkSlicesPerChunk; // The chunk-indexed array of slices into the 'chunkSlices' array.
            // For this I need to know the number of subchunks in each chunk, as well as the total number of subchunks overall.

            [WriteOnly] public NativeMultiHashMap<ushort, NativeSlice<byte>> DataPerSector; // This is the thing that the node repacker copies out of...
            // I can use DataPerSector to calculate the total length for each sector...
          
            public int ChunkCount;
            public void Execute() {
                // I can count the total number of entities per sector here as well...
                uint totalLength = 0;
                uint totalSubchunks = 0;
                for (int i = 0; i < ChunkCount; ++i) {
                    //Debug.Log($"Subchunk Slices for chunk {i}: {totalSubchunks} - {totalSubchunks + SubchunkCounts[i]}");
                    ushort currentSubchunkIndex = SubchunkCounts[i];
                    int start = (int)totalSubchunks;
                    SubChunkSlicesPerChunk[i] = new NativeSlice<NativeSlice<byte>>(SubChunkSlices, start, currentSubchunkIndex);
                    // This implicitly sorts the data by chunk. Is this helpful?
                    // I think it might be because it means the order is stable!
                  
                    if (Subchunks.TryGetFirstValue(i, out SubchunkInfo info, out var it)) { // MultiHashMaps are First In Last Out!
                        do {
                            currentSubchunkIndex--; // The last index is one less than the length...
                            //Debug.Log($"Subchunk order when Counting... {i}: {info.Sector}");
                            ushort sector = info.Sector;
                            uint subLength = info.SubchunkLength;
                            var subchunkSliceWithHeader = new NativeSlice<byte>(Bytes, (int) totalLength, (int) subLength);
                            UnsafeUtility.CopyStructureToPtr(ref info.Header, subchunkSliceWithHeader.GetUnsafePtr());
                            DataPerSector.Add(sector, subchunkSliceWithHeader); // For some reason this isn't in a stable order?
                            var subchunkSlice = new NativeSlice<byte>(subchunkSliceWithHeader, sizeof(SubChunkHeader));
                            // SubChunkSlices need to be in reversed order!
                            SubChunkSlices[start + currentSubchunkIndex] = subchunkSlice; // This means we can write directly into the slice!
                            totalSubchunks++;
                            totalLength += subLength;
                        } while (Subchunks.TryGetNextValue(out info, ref it));
                    }
                }
            }
        }

[StructLayout(LayoutKind.Sequential)]
private struct SubChunkHeader {
   public ushort ArchetypeIndex, Stride, Count;
}

private struct SubchunkInfo {
   public ushort Sector;
   public uint SubchunkLength; // The byte length of the subchunk (the total of all of these will be the total byte length of everything)
   public SubChunkHeader Header;
}