Feedback Wanted: Mesh Compute Shader access

We've heard both internally & externally for a while, that ability to directly access Mesh & SkinnedMeshRenderer geometry data from a compute shader would be useful (one example: " GraphicsBuffer, Mesh vertices and Compute shaders " thread)

So here's a google doc outlining our current proposal & thinking: Unity Mesh API Compute Shader Access

(Everything described in the doc is out in Unity 2021.2 a19)

Feedback or thoughts most welcome, in the doc or here on the forum!

22 Likes

Hi Aras, looks good, thanks! I would be interested in "Ability to create a Mesh with your own GraphicsBuffers". My use case is GPU physics for surgical simulation. This includes mesh volumetric deformation computation, topological modifications (i.e. mesh cutting and tearing) and collision detection and respons between meshes. To make things fast I need to be able to render straight from the GPU (for now I modified the Standard Shader to read data from simulation ComputeBuffers, doing so for more shaders is a bit of a pain).

Another application is to use DXR raytracing capabilities for very fast collision detection based on raycasts between deformable meshes. Specifically, I need to build and update (refit and/or reconstruct) RT acceleration structures (ideally, on the GPU without any readbacks) around deformable meshes. I have got it already working but I need to setup via Mesh each frame, which, I guess, mitigates the performance gains.

Thanks and keep up the good work!

PS: Do I need a custom Unity build to test it? On latest 2021.2.0.a14 I ma getting:
Assets\NoiseBall\NoiseBall.cs(92,20): error CS1061: 'Mesh' does not contain a definition for 'vertexBufferTarget' and no accessible extension method 'vertexBufferTarget' accepting a first argument of type 'Mesh' could be found (are you missing a using directive or an assembly reference?)

1 Like

[quote=“korzen303”, post:2, topic: 837706]
Do I need a custom Unity build to test it
[/quote]
It’s not in any unity build yet.

I've been waiting for this for so long, happy to know it's finally going to be added :)

1 Like

Can be use to build RayTracingAccelerationStructure for DXR?

1 Like

What I am interested in:

"Ability to create a Mesh with your own GraphicsBuffers -- i.e. you create vertex/index buffers, and then create a Mesh that does not manage/own them at all, just “wraps” them into an object that looks like a Mesh to everything else in Unity. Conceptually this is similar to Texture2D.CreateExternalTexture"

... is placed at the very bottom of the document under "Open Questions / Future Considerations". In most cases, this is what I will need. If this feature existed, should I need to modify an existing mesh, I could simply upload it into my own GrapihcsBuffers, mess with the data and assign them to a new (gpu-only) mesh.

I am super exited that anyone at Unity is working on this. I can't think of a more important feature for my work.

2 Likes

I think my use case is relatively basic. I would like to animate a mesh using a custom compute shader. The goal for me is to implement a more advanced skinning algorithm and real-time subdivision.

The API addition for the SkinnedMeshRenderer seems a perfect fit, also to keep out of the box support for motion vectors. Would that then require some option to disable the default animation updates, if the user wants to do the animation entirely on their own?

My feedback atm is only that I would like it asap <3 :smile: & TY !!!

Hopefully it makes it in by 2021.2 release

2 Likes

Would be great to have this for the 2020 LTS version so we could use it together with ECS. Would be a huge improvement for my game, which contains runtime-generated custom terrain.

I have one question though, would it be possible to generate collider data out of modified mesh?

Update: all of this should be in 2021.2 alpha 18 soon.

12 Likes

[quote=“Aras”, post:10, topic: 837706]
Update: all of this should be in 2021.2 alpha 18 soon.
[/quote]

Related but unrelated question, do you have any recommendations on experimenting with mesh shaders in Unity? I’m in the middle of spawning my first custom SRP but not seeing anything for the new graphics api.

edit for reference: https://developer.nvidia.com/blog/introduction-turing-mesh-shaders/

3 Likes

[quote=“oldhighscore_1”, post:11, topic: 837706]
experimenting with mesh shaders in Unity
[/quote]
I don’t think Unity supports mesh shaders right now. I don’t know what the plans are; I’m not part of any graphics related teams, sorry!

2 Likes

[quote=“oldhighscore_1”, post:11, topic: 837706]
Related but unrelated question, do you have any recommendations on experimenting with mesh shaders in Unity? I’m in the middle of spawning my first custom SRP but not seeing anything for the new graphics api.

edit for reference: https://developer.nvidia.com/blog/introduction-turing-mesh-shaders/
[/quote]
Mesh shaders are on the roadmap for when D3D12 is out of beta, when that is… who knows. Likely won’t be 2021 though since it’s in the “Planned” section and not even “In Progress”.

https://portal.productboard.com/unity/1-unity-platform-rendering-visual-effects/c/201-directx12-out-of-preview

As far as I’m aware they haven’t created code to handle the Mesh shader stuff even if you have D3D12 enabled so far. As Mesh shaders are a D3D12_2 feature which is relatively new.

2 Likes

[quote=“Invertex”, post:13, topic: 837706]
Mesh shaders are on the roadmap for when D3D12 is out of beta, when that is… who knows. Likely won’t be 2021 though since it’s in the “Planned” section and not even “In Progress”.

https://portal.productboard.com/unity/1-unity-platform-rendering-visual-effects/c/201-directx12-out-of-preview

As far as I’m aware they haven’t created code to handle the Mesh shader stuff even if you have D3D12 enabled so far. As Mesh shaders are a D3D12_2 feature which is relatively new.
[/quote]

I was unaware that mesh shaders weren’t standardly available across DX, Vulkan, etc. Makes sense that Unity hasn’t tackled it yet then. Meanwhile Unreal is spitting down marketing junk terminology down peoples throats. This just happened to be a feature I wanted when first reading up on the next gen graphic APIs over a year ago so was curious.

Thanks for the info! I’ve taken a glimpse at a product board before from Unity but always wondered where are ALL the product boards for the public? Do you have a link that lists all the product boards? Also seems odd they don’t have a board for strictly SRP but w/e.

2 Likes

[quote=“oldhighscore_1”, post:14, topic: 837706]
I was unaware that mesh shaders weren’t standardly available across DX, Vulkan, etc. Makes sense that Unity hasn’t tackled it yet then. Meanwhile Unreal is spitting down marketing junk terminology down peoples throats. This just happened to be a feature I wanted when first reading up on the next gen graphic APIs over a year ago so was curious.

Thanks for the info! I’ve taken a glimpse at a product board before from Unity but always wondered where are ALL the product boards for the public? Do you have a link that lists all the product boards? Also seems odd they don’t have a board for strictly SRP but w/e.
[/quote]
https://unity.com/roadmap/unity-platform

2 Likes

[quote=“oldhighscore_1”, post:14, topic: 837706]
I was unaware that mesh shaders weren’t standardly available across DX, Vulkan, etc. Makes sense that Unity hasn’t tackled it yet then. Meanwhile Unreal is spitting down marketing junk terminology down peoples throats. This just happened to be a feature I wanted when first reading up on the next gen graphic APIs over a year ago so was curious…
[/quote]
It is part of the latest Vulkan, DX and OpenGL, but it was only added to those APIs in the past 2 years. And it being part of the API doesn’t make it automatically available in a game engine. The engine generally has to develop new code for their shader/compute compilers to handle the functionality, do all kinds of testing to try and find conflicts, etc…, figure out how to make the engine interact with it nicely and with Mesh Shaders especially it’s kind of a radical change in how meshes would be handled in the pipeline, so it’s not a simple “we’ve updated the DirectX library version!”.
Not to mention the oldest hardware Mesh Shaders are supported on is Nvidia 2xxx/11xx (Turing) and the latest AMD GPU (RDNA2)… So it wouldn’t be a big priority in that case due to the fairly small userbase support currently.

3 Likes

DXR have small user base and supported.

[quote=“vx4”, post:17, topic: 837706]
DXR have small user base and supported.
[/quote]
Yes but it’s also a much more publicized feature that people have been craving to reach realtime for decades… Unity not getting it for a long time would have been terrible publicity. Meanwhile barely anyone knows what mesh shaders are. It would have been a good media opportunity if Unity had implemented them when they implemented DXR and beat Unreal to the “look at all these polys” hype, but that ship has passed now.

Hopefully it won’t be years before they support that part of new hardware.

(also, DXR is simpler to implement into the pipeline compared to creating a pathway for mesh shading)

2 Likes

Alpha 19 is now out with the direct mesh access feature :)

5 Likes

[quote=“Invertex”, post:19, topic: 837706]
Alpha 19 is now out with the direct mesh access feature :slight_smile:
[/quote]

Party time

3 Likes