MaterialPropertyBlock support in MeshInstanceRendererSystem

Unity Engine
1

MeshInstanceRendererSystem is a wonderfully optimized bonus you can find in the Entities package, allowing you to render objects with MeshInstanceRenderer shared data components without having to deal with culling, LOD and other concerns associated with rolling Graphics.DrawMeshInstanced calls directly.

Unfortunately, that system has a huge piece of the puzzle missing, and that is support for MaterialPropertyBlocks. It simply passes null to the MaterialPropertyBlock argument of Graphics.DrawMeshInstanced.

With traditional GameObjects using instanced shaders, MaterialPropertyBlocks provide an unparalleled flexibility: you can render whole level with tens of thousands of blocks using just a few drawcalls without sacrificing the ability to customize materials of every instance separately. The process is simple:

  • Reuse a single MaterialPropertyBlock
  • Assign new values to it for every instance
  • Apply the MaterialPropertyBlock to MeshRenderer of a given instance
  • Move to next instance

For a simple example, this be called on every block based on block data, making every part of the level able to take on different damage values and colors:

materialPropertyBlock.SetVector (shaderID_Color, colorVector);
materialPropertyBlock.SetVector (shaderID_Damage, damageVector);
blockRenderer.SetPropertyBlock (materialPropertyBlock)


For a start, lets create a fork of Unity.Rendering part of the package. I love how simple that is - just copy a few classes out of the package, create a new assembly definition, reference Unity.Rendering there and you’re all ready to create your custom rendering system.

Now, let’s look into our options. The first obvious idea is to simply extend the MeshInstanceRenderer component data class, allowing it to pack a MaterialPropertyBlock reference along with Mesh and Material references that are sitting there originally.

Then, going back to where Graphics.DrawMeshInstanced calls are done in the MeshInstanceRendererSystem, let’s just plug that new MaterialPropertyBlock reference into the RenderBatch method that was previously stuffing nulls into the corresponding arguments.

Problem is, we’re getting nowhere: these changes do not allow you to request per-entity material properties. Since we are reusing a MaterialPropertyBlock instance over all Entities holding a MeshInstanceRenderer, there is no per-instance data to speak of. That is, unless you use the array methods on MaterialPropertyBlock and write custom shaders that know the right array index and utilize array reads to get to their properties.

While it’s a pain to rewrite shaders and modify MaterialPropertyBlock handling code to set these arrays, it’s doable, and according to the previous threads on the subject, it’s the recommended way of getting per-instance values to instances rendered with Graphics.DrawMeshInstanced. Except this solution is not possible here for one simple reason.

Your code is not in control of the instancing batches, MeshInstanceRenderingSystem is. Your level manager code knows the material values for each entity comprising the level, but it can’t stuff the arrays of a MaterialPropertyBlock object with those values because the exact order and number of batches is never up to your level code, just like the ultimate calls to Graphics.DrawMeshInstanced - that’s totally up to the instanced rendering system and depends on interplay of culling and other factors. So, you can’t predict which entity would end at which array index of which batch, making MaterialPropertyBlock.SetVectorArray and other similar methods quite useless in this case.

How was the old MeshRenderer based approach allowing us to ignore this complication? I guess the magic sauce of the old MeshRenderer approach was calling MeshRenderer.SetPropertyBlock, which let each MeshRenderer grab a copy of material data. I have no idea how that particular method is implemented, because it is not managed at all - according to Unity C# code reference repository, it just invokes an internal non-managed method. But I guess what it does is fill a struct with a copy of MaterialPropertyBlock data passed into it.

Subsequently, that copy of material data was picked up in the native code and used with a lower level counterpart of Graphics.DrawMeshInstanced, which allowed rendering from an array while allowing shaders to keep traditional non-array based properties. We obviously have no access to such a rendering method, so we’ll have to make Graphics.DrawMeshInstanced work for us.

I guess that the best way to get parity with MeshRenderers on MeshInstanceRendererSystem would be to do the following:

  • Drop the MaterialPropertyBlock use on the level manager side, since its ultimately a disposable container which can’t be usefully passed to Graphics.DrawMeshInstanced call happening in the instanced rendering system

  • Introduce a new non-shared data component, MeshInstanceMaterial, which would hold a set of property values instead of a property block reference. This data can’t exist on MeshInstanceRenderer, since that component is shared and can’t have per-instance values. In a way, that material data component would be similar to position/rotation/scale components

  • Set that struct per entity, mirroring what happens when you call MeshRenderer.SetPropertyBlock

  • For each batch constructed in the customized MeshInstanceRendererSystem, construct a MaterialPropertyBlock with property arrays holding 1023 entries corresponding to properties stored in MeshInstanceMaterial data component. Again, this would mirror what happens with scale/position/rotation, where a 1023 entries long array of matrices is constructed at the same batch preparation stage

  • Submit the per-batch MaterialPropertyBlock in DrawMeshInstanced argument

  • Write custom shaders reading arrays instead of traditional properties (no way around that without magic sauce Unity has for per-MeshRenderer instanced properties, as far as I see)

Please correct my thinking if you see any issues in my conclusions. I’d also appreciate any info from Unity developers on whether an official support for MaterialPropertyBlock+MeshRenderer like workflow is planned for MeshInstanceRendererSystem in the future.

5 Likes
2

With MeshInstanceRenderer you must also use culling (WorldMeshRenderBounds) and LODs (MeshLODComponent , MeshLODGroupComponent) for gain performance

1 Like
3

Problem with meshrenderer and materialpropertyblock is really annoying. But it is hybrid, so i think it will be rewrited in a future, as it was a few weeks ago, when they added chunk iteration. And I think they even will rewrite Graphics.DrawMeshInstanced or will create analog so it can takes float4x4 instead Matrix4x4.

A time ago I do some hacks with MeshInstanceRenderingSystem with approach as you describe with MaterialPropertyBlock.SetVectorArray, and that somehow works. What I did.
0. Prepared special shader.

  1. Forked MeshInstanceRenderingSystem, MeshInstanceRenderer, RenderingSystemBootstrap.
  2. Created TeamColor : IComponentData. It is a container for color value for each entity.
  3. Then in modified MeshInstanceRenderingSystem I just did same stuff with TeamColor like developers did with VisibleLocalToWorld component (Collect data from chunk, and rearrange data array to array of values). Except I used for-loop (that causes some performance regress maybe) instead unsafe but quick memcpy to copy values from ComponentData to array.
  4. And Then I had consistent array that coresponds to entities pretty well that I can push to MaterialPropertyBlock.SetVectorArray and then to DrawMeshInstanced.

I tested it with different colors, and it works well**.**

But yeah, it is very hacky and unmodular thing, and when they will change MeshInstanceRenderingSystem, I must also update my RenderingSystem. So we definitely need more modular rendering API with per-instance properties.

4

Thanks, that’s very valuable information! Could you, by any chance, post the code for the step where you collect the data from a chunk and perform a memcpy? I’ve never used a memory copy functionality before, so having a reference would be much appreciated!

On another note - yeah, I think Unity would move away from Matrix4x4 use in this system in the future - the TODO comment in the unsafe CopyTo function even references that. :slight_smile:

5

So am I. That’s why i used dirty for-loop:). About chunk iteration you can read there https://github.com/Unity-Technologies/EntityComponentSystemSamples/blob/master/Documentation/content/chunk_iteration.md . MemCpy is used in function that you mention:

static unsafe void CopyTo(NativeSlice<VisibleLocalToWorld> transforms, int count, Matrix4x4[] outMatrices, int offset)
        {
            // @TODO: This is using unsafe code because the Unity DrawInstances API takes a Matrix4x4[] instead of NativeArray.
            Assert.AreEqual(sizeof(Matrix4x4), sizeof(VisibleLocalToWorld));
            fixed (Matrix4x4* resultMatrices = outMatrices)
            {
                VisibleLocalToWorld* sourceMatrices = (VisibleLocalToWorld*)transforms.GetUnsafeReadOnlyPtr();
                UnsafeUtility.MemCpy(resultMatrices + offset, sourceMatrices, UnsafeUtility.SizeOf<Matrix4x4>() * count);
            }
        }

You can take this sources, it is almost the same like vanila unity sources, but with some heavy tweaks in VMeshRendererSystem. I didn’t test it enough, but for now it seams ok for my stuff.

3731785–309211–VMeshRendererSystem.cs (36.1 KB)
3731785–309220–VTeamColor.cs (185 Bytes)
3731785–309223–VMeshRenderer.cs (403 Bytes)

1 Like
6

What vitautart is writing is definitely the right approach. Ultimately i would love to make it so that

struct TeamColor : IComponentData, IInstanceRenderProperties { public float4 Color; }

And it automatically gets bound to the material via MaterialPropertyBlock. Extra cherry on top would be to have shader graph generate the C# struct.

There is nothing preventing you from doing this today by forking the MeshInstanceRendererSystem code. And we will add something like this at a later point.

9 Likes
7

Thanks for confirming this is the right way forward!

The pattern you outlined would certainly be great to have.

Oh, and looks like I misunderstood the role of SetVectorArray and other array methods on MaterialPropertyBlock. Using them with Graphics.DrawMeshInstanced works like using SetVector with MeshRenderers. This simplifies the implementation, I won’t need custom shaders then. :slight_smile:

8

I got the basic implementation working and I’d appreciate any advice on improving it! So far, I got support for up to 8 arbitrary vectors per entity working. None of the other types MaterialPropertyBlock supports setting are really essential - any floats can just be packed into vectors, any colors are vectors etc. Whole setup works like this:

First, we introduce a new data component called MeshInstanceMaterial and attach it to our entities along with MeshInstanceRenderer:

public struct MeshInstanceMaterial : IComponentData
{
    public int propertyVectorCount;
    public MaterialPropertyVector propertyVector0;
    public MaterialPropertyVector propertyVector1;
    public MaterialPropertyVector propertyVector2;
    public MaterialPropertyVector propertyVector3;
    public MaterialPropertyVector propertyVector4;
    public MaterialPropertyVector propertyVector5;
    public MaterialPropertyVector propertyVector6;
    public MaterialPropertyVector propertyVector7;
}

At first I assumed you could just use an array there, similar to how Materials and MeshRenderers do it, but I forgot that data components can’t contain non-blittable types like arrays. Looks dirty, but I’m not yet familiar with any other way to hold this on a data component - please tell me if I missed some sort of an ECS component friendly collection type.

The MaterialPropertyVector is a simple struct like this, mirroring material properties on traditional MaterialPropertyBlock.

public struct MaterialPropertyVector
{
    public int id;
    public Vector4 data;

    public MaterialPropertyVector (int id, Vector4 data)
    {
        this.id = id;
        this.data = data;
    }
}

To assign material properties to an entity, you construct the MeshInstanceMaterial like this (this matches what you usually do with MaterialPropertyBlock.SetVector relatively closely):

MeshInstanceMaterial mim = new MeshInstanceMaterial
{
    propertyVectorCount = 3,
    propertyVector0 = new MaterialPropertyVector (propertyID_A, propertyValue_A),
    propertyVector1 = new MaterialPropertyVector (propertyID_B, propertyValue_B),
    propertyVector2 = new MaterialPropertyVector (propertyID_C, propertyValue_C),
};
entityManager.SetComponentData (entity, mim);

I should probably wrap this in a convenience method that stops you from assigning an incorrect integer to propertyVectorCount and stops you from writing to incorrect properties, as if you were using MaterialPropertyBlock. But this works for now.

Next, in custom instanced renderer system, we add the following fields in the beginning:

private MaterialPropertyBlock m_PropertyBlock = new MaterialPropertyBlock ();
private Dictionary<int, Vector4[]> m_PropertiesVector = new Dictionary<int, Vector4[]> ();

I’m not sure if using a traditional dictionary is a good idea in a system, but I’m not yet familiar with any other types that can be used for a similar setup - I’d appreciate any pointers here!

We also modify the query the system performs to include the new data component. Next, now that we have these fields and material data components are grabbed, we can modify the Update*InstanceRenderer methods like this, making a new call right after CopyTo:

unsafe void UpdateDynamicInstanceRenderer ()
{
    ...
    var meshInstanceMaterialType =
        GetArchetypeChunkComponentType<MeshInstanceMaterial> (false);
    ...
    for (int i = 0; i < packedChunkCount; i++)
    {
        ...
        var materials = chunk.GetNativeArray (meshInstanceMaterialType);
        ...
        CopyTo (visibleTransforms, activeCount, m_MatricesArray, batchCount);
        CollectMaterialProperties (activeCount, batchCount, materials);
        ...
    }
    ...
}

The CollectMaterialProperties method collects the vectors from MeshInstanceMaterial data components and puts them into 1023 entries long arrays which can later be used with MaterialPropertyBlock.SetVectorArray method. It’s tempting to add convenience methods to MeshInstanceMaterial allowing to iterate through its properties as if it was a real array, but that would require a ton of branching repeated by number of ultimately utilized properties, so that’s not a good idea. Instead, I just attempt to read all properties in a row and bail out if an index goes over the property count from the data component. Maybe this can be improved further - and again, maybe individual fields are not even necessary if there is a way to put collections into data components.

void CollectMaterialProperties (int activeCount, int batchCount, NativeArray<MeshInstanceMaterial> materials)
{
    MeshInstanceMaterial mim;
    int shiftedIndex;

    for (int i = 0; i < activeCount; i++)
    {
        mim = materials[i];
        shiftedIndex = batchCount + i;

        CheckProperty (mim.propertyVectorCount, 0, mim.propertyVector0, shiftedIndex);
        CheckProperty (mim.propertyVectorCount, 1, mim.propertyVector1, shiftedIndex);
        CheckProperty (mim.propertyVectorCount, 2, mim.propertyVector2, shiftedIndex);
        CheckProperty (mim.propertyVectorCount, 3, mim.propertyVector3, shiftedIndex);
        CheckProperty (mim.propertyVectorCount, 4, mim.propertyVector4, shiftedIndex);
        CheckProperty (mim.propertyVectorCount, 5, mim.propertyVector5, shiftedIndex);
        CheckProperty (mim.propertyVectorCount, 6, mim.propertyVector6, shiftedIndex);
        CheckProperty (mim.propertyVectorCount, 7, mim.propertyVector7, shiftedIndex);
    }
}

// This method is only there to reduce boilerplate, allowing 1 line per property
void CheckProperty (int propertyCount, int propertyIndex, MaterialPropertyVector property, int shiftedIndex)
{
    // If this check fails, that property was not used
    // (is beyond the length of our make-believe array)
    if (propertyCount <= propertyIndex)
        return;

    // This condition would only be satisfied whenever the system
    // encounters a new property ID, so it won't cause array allocations often
    if (!m_PropertiesVector.ContainsKey (property.id))
        m_PropertiesVector.Add (property.id, new Vector4[1023]);

    m_PropertiesVector[property.id][shiftedIndex] = property.data;
}

After all this, we end up with the dictionary full of shader property ID keys, with a 1023 vectors attached to each key (in an array). Each vector correctly corresponds to ever-shifting position of entities within the instanced batches. The dictionary is never cleared, and arrays are never discarded - new arrays are created whenever new shader property ID is encountered, but that’s it.

Now, we can finally modify the RenderBatch method to put this to use, by calling MaterialPropertyBlock.SetVectorArray for each array in our dictionary and plugging the resulting MaterialPropertyBlock into the argument of the Graphics.DrawMeshInstanced method.

void RenderBatch (int lastRendererIndex, int batchCount)
{
    ...
    if (renderer.mesh && renderer.material)
    {
        m_PropertyBlock.Clear ();
        foreach (KeyValuePair<int, Vector4[]> vectorProperty in m_PropertiesVector)
            m_PropertyBlock.SetVectorArray (vectorProperty.Key, vectorProperty.Value);

        if (renderer.material.enableInstancing)
        {
            Graphics.DrawMeshInstanced (renderer.mesh, renderer.subMesh, renderer.material,
                m_MatricesArray, batchCount, m_PropertyBlock, renderer.castShadows,
                renderer.receiveShadows, 0, ActiveCamera);
        }
        else
        {
            for (int i = 0; i != batchCount; i++)
            {
                Graphics.DrawMesh (renderer.mesh, m_MatricesArray[i], renderer.material, 0,
                    ActiveCamera, renderer.subMesh, m_PropertyBlock,
                    renderer.castShadows, renderer.receiveShadows);
            }
        }
    }
}

This gives me reasonable parity with MaterialPropertyBlocks I used on traditional MeshRenderers previously. Sure, I can’t assign more than 8 vector properties and can’t use anything other than vectors, but I wouldn’t call going over those limits reasonable.

The setup is perfectly functional, but again, I’d really appreciate any advice on improving it. :slight_smile:

2 Likes
9

Isn’t this a perfect usecase for IBufferElementData?

10

You are right, I rewrote it yesterday using buffers! I also made the system store properties based on shader identifier since batches would never contain more than one shader and filling a property block with properties originating from more than shader is a waste.

On another subject, I wonder if it would be possible to move the whole system to Graphics.DrawMeshInstancedIndirect, since what I use the system for is a relatively static level which would really benefit from creating the draw buffer just once instead of per frame.

11

Graphics.DrawMeshInstancedIndirect requires custom shaders. But yes, its definitely faster if all data is already uploaded to the GPU. You then have to build an index list based on the culling results. Or do culling on the GPU.

It is all possible of course.

12

Thanks for clarifying that!

By the way, is this ECS renderer intersecting what SRP (or HDRP in particular) are doing, in any way? I’m not very familiar with SRP code, but I remember many related posts mentioning that pipelines bring improved culling and faster rendering (which is what this system tries to do too). If HD rendering pipeline or any pipeline in general already have a similar system as new default (as in, do fast culling and submit meshes for rendering through Graphics.DrawMeshInstanced), then this whole system is obviously redundant if I were to move the project to HDRP.

13

SRP goes through normal culling codepath and support Graphics.DrawMeshInstancedIndirect & Graphics.DrawMeshInstanced so ECS render code works in both legacy and SRP’s

14

Attached you find an early prototype of per instance properties more like @Joachim_Ante_1 suggested in hopes that it will be easy to switch to what Unity will release in the future.

It currently doesn’t use Unity’s MeshInstanceRendererSystem as it made prototyping simpler but it should be possible to integrate it within an hour.

To add a new property just add a component derived from IRenderProperty. Only float, float4 and float4x4 are supported, what the MaterialPropertyBlock supports. Should probably get replaced by derived interfaces eg FloatRenderProperty…

public struct HightlightColorProperty : IRenderProperty<float4>
{
    public float4 color;
}

The property name is generated by cutting ‘Property’ from the component name and adding an ‘_’ before it. So the above example will set ‘_HighlightColor’ Vector4Array in the MaterialPropertyBlock. A property is only set in the block if at least one entity has it overwritten using the component.

Instances having this component will get batched together with the ones not having it. The ones not having it will get the material’s value for the property.

It should already be pretty fast. There is a very expensive, very unoptimized mesh spawner in there. Just ignore it :slight_smile:

Edit: New version below.

1 Like
15

New version, now supporting non-instanced properties. MaterialPropertyBlock is basically fully supported now.
2018.3b5 required (C# 7.3)

Example Properties:

    public struct ColorInstancedProperty : IFloat4InstancedRenderProperty
    {
        public float4 color;
    }

    public struct MainTexProperty : ITextureRenderProperty
    {
        public Texture texture;
        public Texture Value{ get => texture; }   // <-- currently required to prevent boxing for value types
    }

Usage:

buffer.AddComponent(entity, new ColorInstancedProperty { value = new float4(1,0,0,1)});
buffer.AddSharedComponent(entity, new MainTexProperty { texture = someTexture});

EDIT: Now uses fork of Unity’s MeshInstanceRendererSystem. Frozen are not tested but implemented.

3773455–315418–RenderPropertyPrototype.zip (553 KB)

9 Likes
16

Update: Now uses fork of MeshInstanceRendererSystem (0.0.12-preview.17).

3773773--315388--Untitled.png
3773773--315388--Untitled.png857×611 745 KB

17

Wow, that’s incredible work, thanks a lot for looking into this! Out of curiosity, what exactly did you mean by the following comment in the fork (I’m not sure what “cache supported” means in that context)?

// todo: cache supported properties per material
18

Currently for each chunk the material is checked for properties that have RenderProperty components defined and only these properties are assigned to the MaterialPropertyBlock. Currently this is just checked each time and this could be cached. Will probably only have a real effect when there are ALOT of components, but I type this stuff done when they come to my mind.

Actually there is a problem in this implementation. Graphics.DrawMeshInstanced seems to merge batches under certain circumstances but ignores that some MaterialPropertyBlocks contain an overwrite and some don’t resulting in the ones without getting the overridden property too. I added a ‘// todo: report to unity’ for that. :slight_smile: Currently it just writes all material defaults to the MaterialPropertyBlock which isn’t that great and throws an exception when the material has eg a _MainTex field set to null, as MaterialPropertyBlock may not set textures/buffers to null. In our actual project MeshInstanceRendererSystem draws to CommandBuffers, which do not have this issue and just let me only set the properties actually set in the chunk so I didn’t optimize further.

Probably check for null in _Apply of Texture and ComputeBuffer for now to prevent exception.

19

Reported: (Case 1090093) Graphics.DrawMeshInstanced merges incompatible batches when overriding material properties using MaterialPropertyBlock

Unfortunately I don’t have a way to escalate it as support just ran out :cry:

20

MaterialPropertyBlocks are the only thing holding me back from using a completely pure ECS solution at the moment.

The work here is amazing, but this bug makes it unusable, get changing colors everywhere =(

side note

// not optimized... who uses non instanced materials?

me =( much better performance on unique meshes.