(Jobs,Burst) NativeQueue<T>.ParallelWriter issues

Scripting Burst
, , ,
1

Hello all. I’m implementing the marching cubes algorithm with jobs and burst. As the title says I’m having weird issues with the NativeQueue.Parallelwriter version. I essentially have a job that does 1 voxel in the grid at a time. The vertices it returns are enqueued into the ParallelWriter. I schedule the job with ScheduleParallel, and for convenience sake instantly complete it. All works great, until it doesnt. At random or when modifying the terrain often, my chunks appear glitched. the vertices and indices look like they’ve been scrambled completely:

9176435--1277759--upload_2023-7-27_22-7-52.png
9176435--1277759--upload_2023-7-27_22-7-52.png1225×673 787 KB

This was after i’ve modified the terrain a lot. Peculiarly, if you edit the terrain again and time it right it will stop misbehaving, as it will switch between glitched and normal state very fast:
9176435--1277762--upload_2023-7-27_22-10-55.png
9176435--1277762--upload_2023-7-27_22-10-55.png1221×660 702 KB

Thing is, I’ve done the complete same approach with a NativeArray, by allocating a huge array big enough to store all possible vertices, and using the attribute [NativeDisableParallelForRestriction] to allow parallelization. Only swapping out the queue approach with the nativearray approach fixes the problem, but is way slower since you’re clearing / allocating / disposing huge arrays.

After some more testing while writing this post I can say a couple of things: the problem still occurs with/without vertex welding. The chunk mesh indices and vertices counts are valid in glitched states (both multiples of 3), and nothing weird happens with the counts too.

If anyone knows whats up, please leave a reply!

Edit: I found out it’s not relative to how often you edit terrain, but how many vertices and indices that chunk ends up with, with more of them causing more often glitches.

Edit 2: After further testing I found out it’s something to do with the job running in parallel or not. With Schedule() I get no issues whatsoever (except worsened performance than ScheduleParallel) and with ScheduleParallel the chunks start misbehaving after seemingly a lot of enqueueing calls.

2

Bump

3

Most likely a thread race condition if running on a single thread works fine (via .Schedule). NativeDisableParallelForRestriction disables safety for those kind of cases.
After disabling checks its up to you ensure algorithm never reads / writes incorrect values / simultaneously.

Faster != correct in this case.
Review your code where there is a posibility of reading / writing to the same indicies exist.

In any case, without any code - its impossible to tell.

4

Pretty sure I’m not writing any wrong values. the disable restriction attribute also has no effect afaik on the queue.parallelwriter since its already set up for parallelization. Anyways I just went with a NativeStream container that writes from multiple threads.

5

[Initial] values may be correct, but the order of operation may vary depending which thread changes memory first.

NativeQueue works differently than writing directly to the NativeArray.
And NativeStream works differently than NativeQueue.

NativeQueue.ParallelWriter is non-deterministic, NativeStream is:
https://docs.unity3d.com/Packages/com.unity.collections@1.2/api/Unity.Collections.NativeStream.html
https://discussions.unity.com/t/833635

6

Interesting, thank you for your insights! That thread also basically mentions it’s better to use a concurrent NativeStream instead of a parallel NativeQueue, so thanks for the indirect confirmation.