ModelSortGroupComponent support

I’m currently having a lot of trouble rendering UI (a custom UI system, but the end result is similar to Unity’s – a bunch of layered quads with alpha as separate GameObjects) without having elements drawing in the wrong order – it seems the Vision is unable to correctly Z-sort large quads if they are too close together, and this results in artefacts such as the background showing through.

It looks like ModelSortGroupComponent may be the solution I’m looking for, as this would allow me to directly control the draw order of overlapping quads, rather than relying on Z offset. Is this currently supported in PolySpatial, and if not, could it be added?


Not directly related to this problem, but just browsing the RealityKit documentation; a couple more components that look like they would be very easy to expose but very useful for us game developers:

We actually do have a proxy for ModelSortGroupComponent: PolySpatialSortingGroup (which should come up if you search components for “PolySpatial”). It appears that this component was not added to the documentation’s table of contents, but that link should explain its usage.

Supporting GroundingShadowComponent is definitely on our radar. Not sure about OpacityComponent.

Thanks, that is mostly working, but I am having a consistent issue where the ModelSortGroupComponents are not added sometimes when modifying the PolySpatialSortingGroup.renderers list, possibly relating to when the GameObject is created in the same frame.

Note that I am explicitly reassigning .renderers when I modify the list in order to mark the object dirty. Reassigning the .renderers list again the following frame fixes the issue (modulo the frame glitch).

I have traced the modification as far as PolySpatialSortingGroupTracker.TransferObjectData, which is picking up the change and adding the correct renderer (with a valid PolySpatialInstanceID) to engineData – it’s somehow being ignored at the other end when applying the change data to the RealityKit scene.

I haven’t been able to reproduce the issue in a minimal app yet, it’s very frustrating trying to figure out causes without access to source, so there’s a lot of guesswork about possible issues.

Understood. @vcheung-unity added our support for the ModelSortGroupComponent, so perhaps she has some ideas of why this would be happening.

The main thing I can think of is when the PolySpatialSortingGroup is processed on RealityKit platform, it looks at each renderer on the list to ensure that the renderer exists in RealityKit. If it doesn’t find that renderer on the list, it skips it and continues processing other renderers on the list.

In theory, PolySpatialSortingGroup processing should be last, after processing of renderers, so renderer creation should be handled before the PolySpatialSortingGroup processing, however, it is possible something could be delaying renderer creation for whatever reason.

Just to gather information, is the renderer being created via an async method or something like that? And what kind of renderer is it? Is it just a normal mesh renderer/mesh filter?

I can make a Jira bug ticket to track this issue and add some Log prints at areas where it looks for the existence of renderers to make further cases of these problems easier to debug. I tried to repro the issue by creating an object and assigning it to the list of renderers on the same frame, but was not able to repro.

Thank you! I isolated the issue to renderers that are disabled at the time the sorting group is updated, and then enabled later. I’ve submitted IN-57721 to demonstrate.

1 Like

Semi-related: it would be useful if the sorting group could be applied to any entity, not just those with a renderer. For example, colliders also use the sorting groups to determine hit priority, and I use these in UI to block input in some areas. I have tested that RealityKit respects the sort order when hit-testing colliders, regardless of whether they have a model attached or not.

I’ve updated the task with your findings, thank you!

As for applying sorting group membership to colliders, that’s interesting that it applies to CollisionComponents instead of just ModelComponents! The docs seemed to indicate it only applies to ModelComponents. I’ll make a task to expand coverage of the PolySpatialSortingGroup to colliders as well, and see if I can get confirmation from Apple that that’s intended behavior (and see if there are any other components it may affect as well). Thanks for all the info!