VisionOS Simulator Crash only in MR mode after loading simple model

Hi Guys,

Our Unity MR App is crashing the VisionOS Simulator after loading a very simple 3d model from a backend or either from a assetbundle compiled for visionOS.
A simple cube is working in unbounded mixed reality mode so its no the project setup.
However for VR Mode its worked, the exact same code loading the model is working in the simulator. but not in MR.

Unlikely the simulator console doesn’t give much feedback why its crashing.

Are there any Compiler Symbols or Debug Flags we can leverage to further find the root cause?
Any hints are appreciated.

When we just disable the single line to load the model the app is not crashing with an standard cube visible. And running assetbundle load in the Editor works, but is missing shaders even though the URP Lit Shader is present through the Cube placed in the scene.

Unity 2022.3.11f1
Polyspatial 0.4.1
Xcode 15.1 beta 1

We have tried already enabling in the polyspatial settings:
-Enable Statistics
-Enable Transform Verification
-Enable Mac reality Kit Preview in Play mode
-Transmit Debug info
-Debug Overlay Enabled
-Don’t abort on error
-stop on core exception

Nothing gave us more insights.

We have tried loading a very simple CAD file (10k Polys) containing MeshRenderer, BoxColliders and CustomScripts containing metadata via AssetBundle or Backend. So no unsupported components are used here.

The VisionOS Simulator Console, after our debug.log statement that the model is loaded:

[C:3] Error received: Connection interrupted.
[C:3-1] Error received: Connection interrupted.
nw_read_request_report [C1] Receive failed with error "No message available on STREAM"
nw_protocol_socket_reset_linger [C1:2] setsockopt SO_LINGER failed [22: Invalid argument]
apply fence tx failed (client=0x50fa6ffd) [0x10000003 (ipc/send) invalid destination port]
Failed to commit transaction (client=0xd73168c2) [0x10000003 (ipc/send) invalid destination port]
Failed to commit transaction (client=0x4e668f26) [0x10000003 (ipc/send) invalid destination port]
No resource connection exists for server <peer 15187104966748008135>

Message from debugger: Terminated due to signal 9

What is very strange unity is creating a canvas material when we load the model. However we have no UI Objects in the scene aswell as in the assetbundle or in the backend, so there shouldn’t be a need to create a canvas material.

[Diagnostics] Creating canvas material for 11344:0 (using texture: 00000000-0000-0000-0000-000000000000)
The referenced script (UnityEngine.Rendering.UI.DebugUIHandlerValueTuple) on this Behaviour is missing!
UnityEngine.Resources:LoadAll(String)
System.Lazy`1:ViaFactory(LazyThreadSafetyMode)
System.Lazy`1:ExecutionAndPublication(LazyHelper, Boolean)
System.Lazy`1:CreateValue()
Unity.PolySpatial.Internals.MaterialConversionHelpers:ToPolySpatialMaterialDataAndRegisterTextures(Material)
Unity.PolySpatial.Internals.LocalAssetManager:RegisterFirstReference(AssetRepresentation)
Unity.PolySpatial.Internals.LocalAssetManager:RegisterInternal(Object, PolySpatialAssetID)
Unity.PolySpatial.Internals.LocalAssetManager:Register(T)
Unity.PolySpatial.Internals.InstanceAssetToMaterialCache:GetOrMakeCacheMaterialForInstanceAndImage(PolySpatialInstanceID, PolySpatialAssetID, Func`2)
Unity.PolySpatial.Internals.InstanceAssetToMaterialCache:GetCanvasMaterialAssetIdForInstanceAndImage(PolySpatialInstanceID, PolySpatialAssetID)
Unity.PolySpatial.Internals.CanvasRendererTracker:TransferCanvasRenderData(CanvasRenderer, TrackingData`1*, IChangeListWritable`1)
Unity.PolySpatial.Internals.UnityObjectTracker`3:TransferObjectAndUpdateTrackingData(Object, TTrackingData*)
Unity.PolySpatial.Internals.CanvasRendererTracker:TrackNewAndModifiedObjects(Object[])
Unity.PolySpatial.Internals.ObjectDispatcherProxy:<.ctor>b__12_0(TypeDispatchData)
UnityEngine.<>c:<.cctor>b__54_0(Object[], IntPtr, IntPtr, Int32, Int32, Action`1)
Unity.PolySpatial.Internals.ObjectDispatcherProxy:DispatchTypeChangesAndClear(Type, Action`1)
Unity.PolySpatial.Internals.UnityObjectTracker`3:TrackObjectChanges(LayerMask)
Unity.PolySpatial.Internals.PolySpatialUnityTracker:TrackObjectChanges(LayerMask)
Unity.PolySpatial.Internals.PolySpatialUnitySimulation:Update()
Unity.PolySpatial.Internals.PolySpatialCore:PolySpatialAfterLateUpdate()

We see also this mysterious call in the simulator:

Unbalanced calls to begin/end appearance transitions for <UnityDefaultViewController: 0x1088ed6c0>.

Best regards
David

1 Like

If you could submit a bug report that reproduces the issue with the model in question and let us know the incident number (IN-#####), that would help us track down the problem.

@kapolka We got it now reproduced with the Polyspatial sample and have created a bug report with reproducible project

IN-57875

I hope QA passes it on to your team.

Thanks! I can access these reports directly, so I’ll take a look soon.

Well, the good news is that I can replicate the issue. The bad news is that I don’t know why it’s happening. Interestingly, if I modify AssetBundleLoad to walk the transform hierarchy after loading and replace all the meshes with copies (that is, by creating new meshes and copying the vertices, normals, uv, and triangles properties), then the crash doesn’t happen and the mesh loads correctly (but without the correct materials).

I will note that our support and testing for asset bundles in PolySpatial is basically nonexistent. We know that trying to use asset bundles causes issues with materials (as in the above case), but this example makes it seem like meshes in asset bundles may be problematic as well. I note that if I include the model prefab (I don’t want to say the name in case it’s confidential, but the one in the “Demo Models” folder) directly in the scene, it loads fine and no crash occurs.

thank you for looking into it. ok strange that copying the mesh data is solving that crash. However not a good solution. let me know when you have found the root cause

but the one in the “Demo Models” folder) directly in the scene, it loads fine and no crash occurs.

Same for me, but did you saw that the transformation matrices is wrong of the provided model in Simulator rendered by RealityKit. In Editor its rotated correctly (upwards) and in simulator its flipped (up side down).

This is because of an issue with negative scales (the Root - 1 node has a negative Z scale). There’s another topic discussing this, and there’s a partial fix in 0.4.3. However, while that fixes the orientation, you will probably still see issues with lighting and backface culling when using negative scales.

ok thank you for the clarification, we will checkout 0.4.3

This crash issue seems to only be resulted to just this model. Other models we have tried are working fine.

However, while that fixes the orientation, you will probably still see issues with lighting and backface culling when using negative scales.

Yes that’s true

Will Unity be supporting AssetBundles in PolySpatial? Is full support on your roadmap? Looking at porting a game which uses these, so this could be problematic.

1 Like

It’s on our road map, yes, but I can’t give you a time estimate at the moment.

@kapolka we get now the same crash issue when we spawn at runtime the same CAD model as prefab via C# Monobehaviour into the unity scene(GameObject.Institate).instead of using AssetBundles.
In contrast as you also mentioned placing the prefab in the unity scene and then compiling a build works in the simulator.

The Xcode console shows now before the crash:

Failed to add dependencies of asset ‘assetId:6295249664204647503.compiledmesh’ to the load request due to error: Failed to find AssetLoadDescriptors registered for assetId 6295249664204647503.

Is there a way to search for assetIds in the Editor, to track down which mesh has the assetId:6295249664204647503?

Or where you or your team already able to track the issue down as part of the bug report PSLB-54
Since we see now several CAD models crashing the Simulator however other similar models are working fine.

We have a fix in the pipeline for the asset bundle issue (pink material for URP/Lit), though full asset bundle support will require more work.

As for loading the prefab in that bug report using Instantiate, I was able to do so in the simulator without issue using 0.5.0 and the following script:

using UnityEngine;

public class SpawnObject : MonoBehaviour
{
    public GameObject prefab;

    void Start()
    {
        Instantiate(prefab);
    }
}

(with prefab set to the prefab asset). If you experience that crash in the same circumstances, it’s probably worth submitting another bug report reproducing the exact way in which you load the model.

@kapolka ok great to hear a fix is underway on missing materials with asset bundles.

As for loading the prefab in that bug report using Instantiate, I was able to do so in the simulator without issue using 0.5.0 and the following script:

I did the same thing with 0.5.0 and it still lets the unity visionOS App crash in the visionOS Simulator.
I created a bug report as recommend with reproducible project: IN-59772

Thanks! I was able to reproduce it with 0.5.0. It looks like this is the same issue with submeshes reported in another thread. We will be releasing a fix for this in the next bugfix version.

@kapolka thank you for looking directly into it.

We will be releasing a fix for this in the next bugfix version.

Great to hear!:smiley:

I forgot to reply, that Polyspatial 0.6.2 fixed this issue. Thank you:D

1 Like