So, as far as I understand when a compute shader is dispatched, GPU runs multiple thread groups (the amount of groups is specified in Dispatch() arguments) and each of them contains multiple threads (specified in the [numthreads] attribute over our kernel function in compute shader). My question is whether multiple thread groups compute in parallel (that is, if one thread group is computing its threads, does a different group have to wait for it to finish) and whether one thread group computes all of it’s threads in parallel or one-by-one.
My understanding is this:
There is a difference between the logical workgroup size that you specify via the numthreads attribute and the number of threads that the GPU can execute in lockstep on a single streaming multiprocessor/compute unit. The physical number of threads are called warps/wavefronts/hw-threads, depending on GPU vendor.
The workgroup size can be up to 1024 threads. The warp/wavefront size is often specified as 32 threads on NVidia and 64 threads on AMD (might even be more on modern hardware - not sure).
If your workgroup fits inside a single warp/wavefront, all the threads of the workgroup are executed in lock-step. The GPU may even run multiple workgroups on a single SM/CU if there are still resources available. Group syncs like GroupMemoryBarrierWithGroupSync become a no-op in that case because everything runs in lock step (at least the group sync part of it).
If your workgroup does not fit inside a single warp/wavefront, some of the threads will be executed later than others or on a different SM/CU. In this case, it is necessary to use group syncs to synchronize between threads.
Since there are multiple SM/CUs, it is very likely that multiple workgroups are executed in parallel.
Here’s a very good blog post regarding compute shaders:
Thanks, it makes more sense to me now
Thanks, I’ll read those