How to make vertex shader for mesh instances, usable with mesh particles?

Unity Engine
,
1

Hello Community,

am working to convert a relatively simple object-based system into a particles based system to increase performance.
To be exact it’s this fish (and bird) swarm simulator here: https://assetstore.unity.com/packages/3d/characters/animals/nvjob-simple-boids-flocks-of-birds-fish-and-insects-164188
7792656--984126--upload_2022-1-9_8-13-41.jpg

The actual swarming\movement mechanics can be converted nicely to the job\burst system (and I’ll likely offer a pull request on their Github once done). It more than doubles the max frames.

What I’m struggling with is porting a neat extra feature included: A shader that undulates the unanimated fish mesh as if it were swimming.

This shader does work on separate meshes but breaks when applied to mesh particles.
Here on the top-left is an object with regular mesh renderer. The tripplets of fishes are particles:
https://www.dropbox.com/s/yzvo9ftlw1djtec/Fish%20Shader%202.avi?dl=0

The effect is obviously applied wrongly. Not only to the wrong axis it seems but also awkwardly scale with the distance from the particle systems origin.

Renderer Configuration for the particles:
7792656--984123--upload_2022-1-9_7-52-58.png

This is the original vertex transformation code that works with regular meshes:

    void vert(inout appdata v) {
         UNITY_SETUP_INSTANCE_ID(v);
        float3 wp = mul(unity_ObjectToWorld, half4(1, 1, 1, 1)).xyz;

        float timeY = _Time.y;
        half sinT = sin((timeY + sin(wp.y * 0.5) + _ScriptControl) * _SwimmingSpeed);
        half flap = sinT * _SwimmingPower;
        half zf = (v.vertex.z * v.vertex.z) - (v.vertex.z * _WaveBody);
        v.vertex.x += sin(zf / _SwimmingScale) * flap;
        v.vertex.y += sin(zf / 10) * flap * _Wobble;
        v.vertex.y += sin((timeY + sin((wp.x + wp.y) * _WaveY * 0.6)) * _WaveYSpeed) * _WaveY;
    }

Now I have tried different tips read across the internet for this problem but cannot quite bend my mind around it.
Especially from this thread, it says to apply the “_Camera2World” matrix retrieved through C#:
https://discussions.unity.com/t/531927

So I wrote this which attempts to convert to world space, perform the undulation and then convert back:

        UNITY_SETUP_INSTANCE_ID(v);
        float3 wp = mul(unity_ObjectToWorld, half4(1, 1, 1, 1)).xyz;

        float4 worldSpaceVertex = mul(_Camera2World, v.vertex);

        float timeY = _Time.y;
        half sinT = sin((timeY + sin(wp.y * 0.5) + _ScriptControl) * _SwimmingSpeed);
        half flap = sinT * _SwimmingPower;
        half zf = (worldSpaceVertex.z * worldSpaceVertex.z) - (worldSpaceVertex.z * _WaveBody);
        worldSpaceVertex.x += sin(zf / _SwimmingScale) * flap;
        worldSpaceVertex.y += sin(zf / 10) * flap * _Wobble;
        worldSpaceVertex.y += sin((timeY + sin((wp.x + wp.y) * _WaveY * 0.6)) * _WaveYSpeed) * _WaveY;
       
        v.vertex = mul(_World2Camera, worldSpaceVertex);

Unfortunately the result did not seem to point in the right direction at all. It only makes distortions more extreme.

Is something similar the solution to the recurring problem?

Something interesting from that thread:

What are “instanced mesh particles”? I can only find information about GPU instancing when I’m searching this, but I doubt that’s what they meant.
It would be nice if there were a way to make the mesh particles behave like an instance (just without the gameobject and mesh renderer overhead).

Anybody have an idea or questions?

Huge thanks!

2

That’s exactly what I meant.
https://docs.unity3d.com/Manual/PartSysInstancing.html

3

Ah, thank you for clarifying. This setting does not have an effect on this particular shader and issue then.

4

If your shader isn’t written to work with Unity’s particle system, it won’t work properly, regardless of if you check that box or not.

The problem with the shader from that project is it needs to be able to modify the mesh in the original mesh space. Particles always convert the particle meshes into one single “batched” (aka merged) mega mesh that’s in world space. This breaks two aspects of the shader, as it also means the wp value is always 0,0,0, as that line is getting the mesh’s pivot position. The only way to fix that is to use a GPU instanced particle system, which requires you modify the shader to support Unity’s specific setup for particle GPU instancing. If you flip that setting on the particle system and nothing changes, it’s because it knows the shader isn’t setup for particle instancing and falls back to the default batched mesh.

Also, when people use the term “instance” in the context of meshes and Unity, 99% of the time they’re explicitly referring to GPU instancing.

The other options you have for rendering would be to use Graphics.DrawMesh() which would get rid of the reliance on game objects. But honestly you probably really do want to use GPU instancing, either in the form of particles, or maybe via Graphics.DrawMeshInstanced().

5

Wow it literally was that easy.
Adding the following, directly fixed it:

#pragma multi_compile_instancing
#pragma instancing_options procedural:vertInstancingSetup
#include “UnityStandardParticleInstancing.cginc”

It didn’t require to adapt the vert method itself at all.

Thank you for teaching me something @bgolus :slight_smile:

6

Is instancing properly supported using shadergraph shaders too? Most importantly, do all the “transform” nodes work properly - for example for converting from world space to object space for each individual particle? I am not being very successful in that specific regard at least (particle system and material have the instancing turned on).

I am trying to create a custom decals shader that would support particle system (imagine custom “decals projectors” spawned by particle systems → mesh cubes generated by the particle system that place decals where the cubes intersect the terrain).

// approach taken from this video, just using a modified custom depth buffer to support decals on some transparent objects as well for our game:
https://www.youtube.com/watch?v=f7iO9ernEmM

The shader works perfectly fine with regular cubes as separate game objects. Now I’m trying to make it work for cubes from a particle system, but those don’t work at all, because it seems the transform from world (for a reconstructed point from the depth buffer) to object space for that given “particle cube” is not working like it does for regular game objects.

I was hoping that just instancing alone would deal with all this, but it probably does not? Not sure why / if it should or shouldn’t?

Edit: we are using HDRP, which now I’m starting to think does not support gpu instancing for mesh particles for some reason? Which I find incredibly weird? Not sure but it certainly does not give the same results for object space conversion as individual meshes :confused:

7

HDRP expects you to use the VFX Graph, not Shuriken (the name of the built in particle system). Shader Graph does not support Shuriken particle instancing, nor do any of the Shader Graph nodes for object position or object space transforms work.

8

Fair enough, switching to that is not an issue really. I can use VFX Graph instead, but when I’ve tried it the result was the same. The “VFX Shader” (from shader graph) also considers “object space” relative to the “VFXEffect object’s transform”, instead of per individual generated cubes, so not sure if there’s a way even with VFX Graph and felt like there might be a better chance using shuriken instead since it’s older.

8302893--1089048--upload_2022-7-22_18-8-39.png
8302893--1089048--upload_2022-7-22_18-8-39.png520×580 53.7 KB

9

I guess the other half of it is the VFX Graph and Shader Graph teams don’t seem to really talk to each other much.:roll_eyes:

I’ve not delved too deeply into the inner workings of the VFX graph, though my understanding is it “should” be able support something like this, though likely not using the default transform node.

You might try checking out Unity’s official HDRP VFX samples as it includes the VFX graph spawning decals.