Compute Shaders, populating buffers, reading computed results and general debugging advice

Unity Engine
1

Hi guys,

I can’t help but feel like i missed something extremely fundamental about how compute is supposed to work or something.
I’ve been asking all sorts of questions on stack exchange in the chat rooms, on the game dev site and on unity answers and don’t appear to be getting anything finite so I figured i might try here with a very specific example of my problem so here goes …

Lets assume we havea brand new project in unity with literally no changes and im setup for windows builds and unity is using DX11 mode …

As per the question heading im trying to use compute to do various large scale parallel computing tasks, as a basic test I thought I would try some simple code to get started so I wrote the following compute shader …

#pragma kernel CSMain1D
#pragma kernel CSMain2D
#pragma kernel CSMain3D

RWStructuredBuffer result1d;
RWTexture2D result2d;
RWTexture3D result3d;

[numthreads(2,1,1)]
void CSMain1D (int3 id : SV_DispatchThreadID)
{
result1d[id.x] = 1;
}

[numthreads(2,2,1)]
void CSMain2D (int3 id : SV_DispatchThreadID)
{
result2d[id.xy] = 1;
}

[numthreads(2,2,2)]
void CSMain3D (int3 id : SV_DispatchThreadID)
{
result3d[id.xyz] = 1;
}

I then add the follow CPU code to call my compute shader …

using System.Collections;
using System.IO;
using UnityEngine;

public class Test : MonoBehaviour {

public ComputeShader compute;

// Use this for initialization
void Start () {
Dispatch1d();
Dispatch2d();
Dispatch3d();
}

void Dispatch1d()
{
using (var testBuffer = new ComputeBuffer(2, sizeof(int)))
{
var kernel = compute.FindKernel(“CSMain1D”);
compute.SetBuffer(kernel, “result1d”, testBuffer);
compute.Dispatch(kernel, 1, 1, 1);

var result = new int[2];
testBuffer.GetData(result);
DumpToFile(“1D Results”, result);
testBuffer.Release();
}
}

void Dispatch2d()
{
using (var testBuffer = new ComputeBuffer(4, sizeof(int)))
{
var kernel = compute.FindKernel(“CSMain2D”);
compute.SetBuffer(kernel, “result2d”, testBuffer);
compute.Dispatch(kernel, 1, 1, 1);

var result = new int[2,2];
testBuffer.GetData(result);
DumpToFile(“2D Results”, result);
testBuffer.Release();
}
}

void Dispatch3d()
{
using (var testBuffer = new ComputeBuffer(8, sizeof(int)))
{
var kernel = compute.FindKernel(“CSMain3D”);
compute.SetBuffer(kernel, “result3d”, testBuffer);
compute.Dispatch(kernel, 1, 1, 1);

var result = new int[2,2,2];
testBuffer.GetData(result);
DumpToFile(“3D Results”, result);
testBuffer.Release();
}
}

void DumpToFile(string heading, IEnumerable data)
{
using (var writer = new StreamWriter(File.OpenWrite(@“D:\Logs\Debug.log”)))
{
writer.BaseStream.Position = writer.BaseStream.Length;
writer.WriteLine(heading);
foreach (var i in data)
writer.WriteLine(i.ToString());
writer.WriteLine(string.Empty);
}
}
}

…

Ok thats the code sorted, so i then add a new game object to the default scene unity has been kind enough to create for us and to that game object i add the behavior defined above and set the compute property by dropping the compute file on it.

When I run this I don’t expect it to render anything but instead simply spit out a file with the results of each of the compute calls made and this is the content of the file i get …

1D Results
1
1

2D Results
1
0
0
0

3D Results
1
0
0
0
0
0
0
0

…

The plot thickens …
Before doing this I set out to generate voxels on the gpu using compute, having already built a CPU based implementation i figured “how hard could it be” so as a first phase I wrote a simple test voxel chunk generator and took the results and passed them to my cpu based mesh generator.

It worked perfectly, but the odd thing is … if i log out the results in my voxel buffer to a file in the same way I am doing above it tells me the buffer is full of all 0’s like I haven’t put anything in it at all.

This can’t be true else how would I get mesh generated from it?

So I have several questions really …

  1. Does the above example make sense / work as people here would expect?
  2. Why does VS + the file + Debug.Log() in unity all report that all those arrays are empty?

General debugging of compute shaders:
I noticed that microsoft suggests hooking up the graphics debugger which doesn’t seem to be possible with unity projects for some reason.
If I do “DEbug > Graphics > Start Diagnostics” i get a popup that reads “The graphics Diagnostics feature is not available for this target” … have I missed something / not set something up or are unity projects not possible to debug with Microsofts tooling?

  1. How do I correctly see either the content of the buffer on the gpu or the downloaded result of populating an array on the cpu side from visual studio?

  2. Is this something I did wrong or a bug in the tool chain somewhere?

I’m inclined to say its my code at fault but don’t know where to start getting to the bottom of all this so any advice would be really appreciated.

EDIT: I have uploaded a zip with the scene, compute, and MonoBehavior files in it for others to try out in case it helps

2505007–173186–Test.zip (5.07 KB)

2 Likes
2

Excellent post and example @TehWardy , to me it seems like Unity only supports 1D buffers. Be nice if anyone at Unity could check this out. I am wanting to use Compute shaders more and more, yet keep finding elements like this holding me back.

Maybe we have the wrong end of the stick and we are setting them up wrong, again be nice if some one at Unity could point out what we are getting wrong. :slight_smile:

If I get chance ill have a look at your sample project and see if I can see anything obvious in there :slight_smile:

1 Like
3

Thanks for support on this one @NemoKrad …

It’s likely something simple about the way Texture2D and Texture3D buffers work in unity or in DX that i’m missing but the oddity in all this is that my voxel generator example works but doesn’t report the correct values to anything (VS, file, unity console).

My current issue however means I need to debug the content of a 3D buffer and my thought process was … if I pull it back in to system ram I can examine the data in C# rather than trying to jump through the many hoops to setup graphics debugging.

Coincidentally though, I haven’t been able to find a really clear guide on how to debug compute shaders in unity, all seem to suggest that I need a C++ project which just seems odd …

Perhaps someone from unity could lend some light on this by crawling through the code we can’t in the API?

1 Like
4

@Aras Any ideas? (on my phone hope the tag works)

5

Had an interesting conversation last night on facebook with someone from a unity user group (thx Sebastian, really interesting feedback) …

The resut of the conversation is that “we don’t know why this doesn’t work” but Sebastian had an interesting theory that Texture3D is somehow only mappable to a Texture object on the cpu side.

Which raises an interesting question …
Given the purpose of a GPU is to work with 3D data massively paralleled why can I not map a 3D array on the CPU side to a 3D buffer on the GPU?

The other point that came up with seemingly no explanation (from anyone i’ve asked so far actually) was …

I keep referring to this example but have not yet provided it for many to see but this is what really confuses me about the whole problem:
How come this works …

#pragma kernel GenerateChunk

struct voxel
{
float weight;
uint type;
};

voxel newVoxel(float weight, uint type)
{
voxel voxel;
voxel.weight = weight;
voxel.type = type;
return voxel;
};

// params for voxel generation
int3 from;
int3 to;

// the voxel generation result buffer
RWTexture3D voxels;

[numthreads(3,3,3)]
void GenerateChunk (uint3 threadId : SV_DispatchThreadID)
{
int3 pos = threadId + from;

bool xOk = (pos.x > 0 && pos.x < to.x);
bool yOk = (pos.y > 0 && pos.y < to.y);
bool zOk = (pos.z > 0 && pos.z < to.z);
bool ok = (xOk && yOk && zOk);

if (ok)
{
voxels[threadId] = newVoxel(1, 1);
}

if(!ok)
{
voxels[threadId] = newVoxel(-1, 0);
}
}

… this generates a buffer with 3x3x3 voxels where only the center voxel has a weight value of 1, all the rest have a weight value of -1.
When I examine this buffer / log it out to a file I get the same result as in the original question (first element 1 the rest 0) and yet when the content of that buffer is read from the GPU using GetData(target3DArray) and pass to my CPU implementation of marching cubes I get the single voxel I would expect rendered to the screen.

Clearly it can read the data ok, my problem is how that data is presented to me in the tooling and the fact that I cannot even drop that readable data to a file to debug the code issue in my GPU based mesh generation code.

1 Like
6

I think the Unity GetData function must be broken for Compute shaders, but it would be good if someone from Unity could let us know if this is the case and that it’s being looked at, as at the moment all we are doing is chasing our tails trying to get this resolved, and it may be unresolvable due to an issue in the framework, but if it isn’t an issue with Unity, then again, would be good if someone from Unity could give us a reason why we are seeing the behaviour, and how to code for it…

1 Like
7

Cracked it :slight_smile:

1 Like
8

Can you please give details of the solution?

9

@TehWardy Thank you so much for being so specific, cause it’s often times hard to find good info on shaders. I know it’s been a while, but what was the solution that you found to your problem with the computer shaders?

10

It might be possible to inspect the data using RenderDoc captures. If you select a format for the texture that has a layout similar to the struct, you should be able to see the values. Otherwise just use StructuredBuffers where you can at least change the format for the visualization in RenderDoc to match the struct itself.