New SharedComponentData filtering issues in Entities 0.2 (Case 1204153)

I detected that some of my systems that filtered on the RenderMesh component behaved in a very odd way in Entities 0.2. These filtered EntityQueries no longer matched all RenderMeshes. In fact, they matched only a fraction of them. I am not sure if this is caused by issues in the RenderMesh implementation or the EntityQuery filtering implementation, but I have verified that there exists some kind of issue that was introduced with Entities 0.2.

To isolate and demonstrate this, I wrote a test system that demonstrates a few working test cases and then also this issue. The system and it’s tests are written to simply sum up the total amount of entities using RenderMesh components in different ways. I have tested it in both Entities 0.1 and Entities 0.2 and could confirm that it worked properly in 0.1 and was broken in 0.2. I also had a friend try this out in his project, and he could reproduce it. The issue I’m focusing on here is demonstrated in “Test 3”, which looks like this:

/// [TEST 3]
// Fetching all Entities by using filtering.
// This worked in 0.1, but no longer works in 0.2. It prints a very odd result, only matching a few select chunks.
private CalculationResult Test_3_CalculateWithSharedComponentFilter()
{
    // Just a helper class that sums up the amount of entities per material
    CalculationResult result = new CalculationResult();

    // Fetch all unique RenderMeshes
    List<RenderMesh> renderMeshes = new List<RenderMesh>();
    EntityManager.GetAllUniqueSharedComponentData<RenderMesh>(renderMeshes);

    for (int i = 0; i < renderMeshes.Count; i++)
    {
        _RenderMeshQuery.ResetFilter();

        // Filter on a certain RenderMesh and calculate how many entities matches the filter.
        // In Entities 0.2, for almost all of the RenderMeshes I have, this matches zero entities.
        // The sum won't add up to the full number of Entities with RenderMeshes.
        // In comparison to the other tests that properly reported 1571 render meshes, this test only counts 84.
        _RenderMeshQuery.SetSharedComponentFilter(renderMeshes[i]);
        int filteredEntityCount = _RenderMeshQuery.CalculateEntityCount();

        var material = renderMeshes[i].material;
        result.AddEntityCount(material, filteredEntityCount);
    }

    return result;
}

It’s also worth mentioning that most of these entities with RenderMeshes are built and loaded through a SubScene. Not sure if that affects the end result, as I believe they should be treated as entities regardless. I’m using the following versions to trigger the issue:

  • Unity 2019.3.0f1
  • Entities 0.2.0 preview 18
  • Hybrid Renderer 0.2.0 preview 18

Here’s the complete test system file. You should be able to run it in any project using Entities and the Hybrid Renderer. Feel free to paste it into your project to see the oddities for yourself.

using System.Collections.Generic;
using Unity.Collections;
using Unity.Entities;
using Unity.Rendering;
using UnityEngine;
using System.Linq;

[ExecuteAlways]
[UpdateInGroup(typeof(PresentationSystemGroup))]
[UpdateBefore(typeof(RenderMeshSystemV2))]
public class RenderMeshFilteringIssuesSystem : ComponentSystem
{
    EntityQuery _RenderMeshQuery;

    protected override void OnCreate()
    {
        _RenderMeshQuery = GetEntityQuery(
            ComponentType.ReadWrite<RenderMesh>()
            );
    }

    protected override void OnUpdate()
    {
        // Issues exist in Test3 and Test4.

        /// [TEST 1]
        // A simple Entities.ForEach, counting up all materials. We're also running ResetFilter on the unrelated _RenderMeshQuery to avoid getting affected by it. That part will be demonstrated in Test4.
        CalculationResult test1 = Test_1_CalculateSimpleForEach_WithResetFilter();

        // 1571 RenderMeshes in Entities 0.1
        // 1571 RenderMeshes in Entities 0.2
        Debug.Log("[Test 1]" + test1.ToString());



        /// [TEST 2]
        // Iterates all chunks and fetches their entity count and render mesh to sum them all up.
        // Works as intended in both Entities 0.1 and Entities 0.2
        CalculationResult test2 = Test_2_CalculateIterateChunks();

        // 1571 RenderMeshes in Entities 0.1
        // 1571 RenderMeshes in Entities 0.2
        Debug.Log("[Test 2]" + test2.ToString());



        /// [TEST 3]
        // Fetching all Entities by using filtering.
        // This worked in 0.1, but no longer works in 0.2. It prints a very odd result, only matching a few select chunks.
        CalculationResult test3 = Test_3_CalculateWithSharedComponentFilter();

        // 1571 RenderMeshes in Entities 0.1
        // 84 RenderMeshes in Entities 0.2  <-- ERROR
        // All unique material instances are accounted for in both versions, but the filtered query result gives a 0 for most EntityCounts in Entities 0.2.
        // The filtering doesn't seem to match their chunks.
        Debug.Log("[Test 3]" + test3.ToString());



        /// [TEST 4]
        // Doing a simple ForEach over all entities, counting them all.
        CalculationResult test4 = Test_4_CalculateSimpleForEach_WithoutResetFilter();

        // Doesn't work properly in neither Entities 0.1 nor Entities 0.2.
        // The Entities.ForEach is seemingly to be affected by the filter set on the unrelated _RenderMeshQuery.
        Debug.Log("[Test 4]" + test4.ToString());
    }

    private CalculationResult Test_1_CalculateSimpleForEach_WithResetFilter()
    {
        // I noticed that if I don't reset the filter, this query seems to actually affect my ordinary Entities.ForEach. Odd.
        _RenderMeshQuery.ResetFilter();

        CalculationResult result = new CalculationResult();
        Entities.WithAll<RenderMesh>().ForEach((Entity e, RenderMesh renderMesh) =>
        {
            result.AddEntityCount(renderMesh.material, 1);
        });
        return result;
    }

    private CalculationResult Test_2_CalculateIterateChunks()
    {
        _RenderMeshQuery.ResetFilter();

        CalculationResult result = new CalculationResult();

        var renderMeshType = GetArchetypeChunkSharedComponentType<RenderMesh>();
        using (var renderMeshChunks = _RenderMeshQuery.CreateArchetypeChunkArray(Allocator.TempJob))
        {
            for (int i = 0; i < renderMeshChunks.Length; i++)
            {
                var chunk = renderMeshChunks[i];
                var renderMesh = chunk.GetSharedComponentData(renderMeshType, EntityManager);

                result.AddEntityCount(renderMesh.material, chunk.Count);
            }
        }

        return result;
    }

    private CalculationResult Test_3_CalculateWithSharedComponentFilter()
    {
        _RenderMeshQuery.ResetFilter();

        CalculationResult result = new CalculationResult();

        List<RenderMesh> renderMeshes = new List<RenderMesh>();
        EntityManager.GetAllUniqueSharedComponentData<RenderMesh>(renderMeshes);

        for (int i = 0; i < renderMeshes.Count; i++)
        {
            _RenderMeshQuery.ResetFilter();

            // In Entities 0.2, for almost all of the RenderMeshes I have, this matches zero entities.
            // The sum won't add up to the full number of Entities with RenderMeshes in the game.
            _RenderMeshQuery.SetSharedComponentFilter(renderMeshes[i]);
            int filteredEntityCount = _RenderMeshQuery.CalculateEntityCount();

            var material = renderMeshes[i].material;
            result.AddEntityCount(material, filteredEntityCount);
        }

        return result;
    }

    private CalculationResult Test_4_CalculateSimpleForEach_WithoutResetFilter()
    {
        // Note that in contrast to Test 1, we're not resetting the filter. This makes this Entities.ForEach use the filter from the EntityQuery created in OnCreate.
        CalculationResult result = new CalculationResult();
        Entities.WithAll<RenderMesh>().ForEach((Entity e, RenderMesh renderMesh) =>
        {
            result.AddEntityCount(renderMesh.material, 1);
        });
        return result;
    }

    /// <summary>
    /// Just a simple helper class to sum up the entity count for each material (grouped by material name)
    /// </summary>
    private class CalculationResult
    {
        public int TotalCount;
        public Dictionary<string, int> IndividualTypeCount = new Dictionary<string, int>();

        public void AddEntityCount(Material material, int entityCount)
        {
            TotalCount += entityCount;

            string materialName = material?.name ?? "";
            if (IndividualTypeCount.TryGetValue(materialName, out int existingCount))
                IndividualTypeCount[materialName] = existingCount + entityCount;
            else
                IndividualTypeCount[materialName] = entityCount;
        }

        public override string ToString()
        {
            var formattedTypeCounts = IndividualTypeCount.Select(x => x.Key + ": " + x.Value).ToList();
            formattedTypeCounts.Sort();
            return "Entity Count: " + TotalCount + "\n" + string.Join("\n", formattedTypeCounts);
        }
    }
}

Edit: Also reported this as a proper bug report. Case 1204153

What is the stating test scenario for you, number of spawned entities and so on?
I’ve tried it and it seems to be giving me the correct values.
I’ve spawned 20k RenderMeshes, 10k of each RenderMesh and all the tests come back with the correct 20k count.

I kind of assumed it was a generic problem after having taken the effort of verifying with friends in another project. Apologies for that. I have now tested some more, and I believe it might have to do with SubScenes.

I set up a sample project where I was able to reproduce this issue.

5262959–526550–RenderMeshIssues_SampleProject.zip (12.1 KB)

1 Like

I can confirm that it happens also in Entities 0.3.0 and has to be something related with SubScenes and RenderMesh.

EDIT: It seems to happens with all ISharedComponentData as long as they have managed types the Material type.

Thanks for the bug report. I tracked down the cause of the issue and it should be fixed in Entities 0.6.0. It should be available now, could you let us know if this fixes your issue?

The problem was due to an incorrect assumption we make about hashes for shared components during serialization and deserialization. If you can’t update to the latest package, you can work around it by manually setting/adding the shared component before filtering (should only need to do this once when the subscene is loaded).

Hi [mention|/tkkLtbm8EYgXNmJCGXuDg==]! I have since then changed approach in my project, eliminating the need for this code. I did however update the tiny sample project that I’ve attached previously in the post just above to Entities 0.6 and tested it out. Unfortunately it still doesn’t seem to work.

The example project has four test cases, where each test case is very simple and just iterates all entities with Render Meshes in a SubScene in different ways. Each test should return the same summed amount of entities. Unfortunately, test case #3 that uses SetSharedComponentFilter still doesn’t match any entities. If you need a test case, that sample project is extremely small and demonstrates the issue in an isolated manner.

Sorry, I think I was mistaken about when the fix would be available. It looks like it will be in the next release 0.7.0!