Make class a struct and use nativearrays in it

Unity Engine
,
1

How can i turn something like this:

public class VoxelData
    {
        public Entity Entity;
        public NativeArray<float> DensityMap;
        public NativeArray<byte> BiomeMap;
        public Vector2Int Chunk;
        public int Lod;
    }

Into a struct that i can use with burst? DensityMap for example is a flattened 3d array that i will use for calculating collisions with voxels.

I am using this in a Dictionary which im trying to swap into a NativeHashMap
But then: ““ArgumentException: Client.Generic.VoxelData used in native collection is not blittable, not primitive, or contains a type tagged as NativeContainer””

Is for example storing a pointer in that struct that points to the NativeArray an idea?

Would like to know how you would approach this.

2 
public struct VoxelData
    {
        public Entity Entity;
        [NoAlias] public NativeArray<float> DensityMap;
        [NoAlias] public NativeArray<byte> BiomeMap;
        public Vector2Int Chunk;
        public int Lod;
    }

Structs containing native containers are allowed in Burst. Native containers containing native containers are not allowed with or without Burst. If you put a container inside a struct, you should specify [NoAlias] in order to help Burst out with optimizing.

5 Likes
3

@DreamingImLatios I’m sorry i haven’t been clear enough in my question.

The VoxelData class was used in a Dictionary <Vector2Int, VoxelData>
When i am making it a struct and trying to turn the Dictionary to a NativeHashMap I’m getting the following exception:
“ArgumentException: Client.Generic.VoxelData used in native collection is not blittable, not primitive, or contains a type tagged as NativeContainer”
This is the reason i am asking.

So i want to swap out Dictionary <Vector2Int, VoxelData> for a NativeHashMap <Vector2Int, VoxelData> So i can use it in jobs/burst.

4

In your case, you have a NativeHashMap (a NativeContainer) which contains VoxelData, which contains NativeArrays (a NativeContainer. So you have a NativeContainer inside a NativeContainer.

There are plenty of posts on these forums that discuss different ways to store voxel data. For a start, you may want to try converting your NativeArrays into fixed arrays. Then you may want to store the voxel data in a NativeArray that uses 2D indexing math rather than a NativeHashMap.

5

There are plenty of posts on these forums that discuss different ways to store voxel data.
I tried to search but didnt come up with anything good, could you perhaps share some links?

For a start, you may want to try converting your NativeArrays into fixed arrays
Not sure what you mean by that. Currently my arrays are already fixed size. The are the same for each VoxelData class, maybe the name is a bit unhandy. VoxelData is actually a chunk of 64x64x255, the Density array gets sized this way. The Biome array only is 64x64 and stores which biome is at that location.

Then you may want to store the voxel data in a NativeArray that uses 2D indexing math rather than a NativeHashMap.
I have all the indexing working in 3d. Chunks are 64x64x255 in size and the indexing works fine. I just need a way to use this with burst/jobs now i can’t.

In other words, i have everything working, but not with Jobs / Burst

6

Did you try searching for “voxel” in the “Data Oriented Technology Stack” subforum?

I mean using the “fixed” keyword instead of NativeArray.

NativeContainers don’t work the same way as typical C# collections. They have built-in safety mechanisms so that you can use them in multi-threaded contexts without accidental race conditions. However, that imposes some additional restrictions which may prevent them from being a 1-1 replacement for the C# collections. You may have to redesign your data structures a little, or take risks diving into unsafe territory.

7

So you might find there are other containers provided that could do what you want. Instead of collecting all information together into one struct and then trying to jam that into a dictionary. You could use a NativeHashMap and NativeMultiHashMap for what you’re trying to achieve. Basically the following:

public struct Data
{
    public NativeHashMap<Vector2Int, Entity> Entities;
    public NativeMultiHashMap<Vector2Int, float> DensityMaps;
    public NativeMultiHashMap<Vector2Int, byte> BiomeMaps;
    public NativeHashMap<Vector2Int, Vector2Int> Chunks;
    public NativeHashMap<Vector2Int, int> Lods;
}

This now allows you to pass that struct to a job and should help with being able to then have Burst compatible code. Do note you’ll still have to deal with using the correct concurrent structures if you want to concurrently write to these items.

8

Thanks i’ll take a look at that. Never worked with NativeMultiHashMap before and i wonder how i would use the 3d flattened coordinate access in that.

9

Solved it as follows

ChunkManager:
NativeArray<float> DensityMap = new NativeArray<float>( DensityMapArraySize, Allocator.Persistent);
NativeArray<byte> BiomeMap = new NativeArray<byte>(BiomeMapArraySize, Allocator.Persistent);
VoxelData v = new VoxelData
{
    Chunk = point,
    Lod = GetLOD(distance),
    Entity = Entity.Null,
    PtrBiomeMap = BiomeMap.GetUnsafePtr(),
    PtrDensitryMap = DensityMap.GetUnsafePtr()
};


public struct VoxelData
    {
        public Entity Entity;
        public unsafe void* PtrDensitryMap;
        public unsafe void* PtrBiomeMap;
        public Vector2Int Chunk;
        public int Lod;
    }

And using NativeArrayUnsafeUtility to convert pointers to NativeArray

I use this successfully in Jobs with Burst :slight_smile:

10

You aren’t getting errors about NativeArrays not being disposed? That code looks very prone to memory leaks.

11 
NativeArrayUnsafeUtility.ConvertExistingDataToNativeArray<byte>(
                                    ChunkManager.Chunks[point].PtrBiomeMap,
                                    ChunkManagerSystem.DensityMapArraySize,
                                    Allocator.Invalid).Dispose();

I just tested this. I do indeed get errors. I thought the above code would Dispose() of the array the correct way but I get exception here. What would be the correct way to free the memory here? Would I just use UnsafeUtilities here?

12

how about:

UnsafeUtility.MemClear(ChunkManager.Chunks[point].PtrBiomeMap, ChunkManagerSystem.BiomeMapArraySize * sizeof(byte));
 
UnsafeUtility.MemClear(ChunkManager.Chunks[point].PtrDensitryMap, ChunkManagerSystem.DensityMapArraySize * sizeof(float));
13

MemClear should have nothing to do with this. The issue is that when you convert the NativeArray to the pointer, you lose track of the AtomicSafetyHandle and DisposeSentinel. Those are important to ensure what you are doing is safe. Otherwise you could have race conditions and memory leaks all over your code and no one will notice until it is too late.

14

I would love some code example to implement my solution in the proper (memory safe) way.

15

I don’t know enough about your data’s lifecycle to give you an example. Unlike in managed, there is no GC to clean up after you. So knowing the lifecycle of data is important.

16

The lifecycle end when the chunk (that holds the DenstiyMap / BiomeMap ) gets to far away from the player. In that moment the arrays should be deallocated.

In don’t mind even to manage these 2 arrays completely manually with AllocHGlobal and use pointer aritmethic to do 3d coordinate access, if that is possible…

17

If is very possible to do everything in an unsafe context. Have a look at UnsafeUtitltiy. The problem is that you are completely bypassing the safety system. If you want safety, you are probably going to have to write a custom Native Container that operates similarly to NativeHashMap (you can use UnsafeHashMap under the hood) and handles a pool of voxel chunks.