NativeLongList which supports large amounts of data

Community Showcases
1

There is a limit to max size of native collections like NativeArray and NativeList. I made a simple NativeLongList implementation which lets you store huge amounts of data with very low performance loss.

1 Like
2

Out of interest did you ever update this for Unity after 2020 (specifically 2021.3)? A lot of the documentation e.g. on JacksonDunstan, and even on the Unity docs at Unity - Scripting API: NativeContainerAttribute doesn’t actually compile anymore.

(I haven’t actually checked that the example on Unity’s page still doesn’t compile, but filed a doc bug months ago and it’s still open.)

3

Yeah, I’m still using it in 2021.3 and it seems to work.

4

Oh, thanks. I didn’t look in detail at the time (had already rewritten a function to use native Mallocs and C++), but took another look now because it would be great to have a bounds-checked container like yours and the ability to pass to Burst-ed code without using pointers all over the place…

It looks like with 2021.3, the only change necessary, is that at NativeLongList.cs:47:

_data = AllocatorManager.Allocate<NativeListData>(new AllocatorManager.AllocatorHandle { Value = (int)allocator });

There’s a compile error because AllocatorHandle.Value is now read-only, but that’s fine because we only used that AllocatorHandle as a temp to call Allocate, and we can can just use AllocatorManager.Allocate() directly, so replaced with this:

_data = AllocatorManager.Allocate<NativeListData>(allocator);

And everything then seems to work. Does that seem valid to you?

Thanks for your contribution!

5

Yup, no idea why I didn’t use the existing allocator there, thanks :smile: