Mesh updates not synchronized

It appears that when updating a Mesh (e.g., using AllocateWritableMeshData or the older APIs), the resulting change on the RealityKit side is not synchronized with other changes to the entities, resulting in flickering.

It looks like this is due to PolySpatial using MeshResource.replace(with:), which the documentation indicates is not synchronised. Using MeshResource.generate(from:) is synchronized, and eliminates the flickering.

I can work around this on my end by always generating new Mesh resources and destroying meshes from the previous frame, but I think it would make sense if PolySpatial handled this internally the same way (i.e., in the native layer, always generate new MeshResource and reassign ModelComponent.mesh, instead of using MeshResource.replace), as a more general solution – I see no benefit to defaulting to unsynchronized mesh updates.

Please file a bug with a small repro project and post the incident id here.

Thank you.

The reason we use MeshResource.replace is to avoid having to update the entities that reference the mesh (because they automatically incorporate the changes, whereas if we used MeshResource.generate, we’d have to reset the meshes in the ModelComponent). If recreating the meshes using MeshResource.generate avoids flickering, though, that’s certainly something we could switch to doing. As Joe says, a repro case would help make the argument to do so.

The synchronization that the docs to MeshResource.replace refer to is network synchronization, though (that is, RealityKit’s content synchronization), so I’m not sure it’s relevant in this case. Basically, it’s just saying that each mesh instance is only sent once over the network; updates to a given mesh aren’t synchronized.

I believe the network synchronization also applies to sending updates from the game process to backboardd, which is doing the actual rendering. Unfortunately I’m a bit short on time right now so won’t be able to put together a test case / incident report.

Hi, I’m having troubles with a mesh blinking when I update it (with SetVertices(), etc) in the Update() method.
It only happens on VisionPro with polyspatial v1.1 (not in VR), so I’m wondering if I’m actually in this same case of unsynchronized MeshResource.replace(with:)
I posted a thread in the forum with a simple example to reproduce it : https://forum.unity.com/threads/visionos-mesh-flickers-if-updated-on-every-frames.1561639/
Do you have news on that?

We tried using MeshResource.generate in all cases, but experienced significant performance regressions on meshes that change every frame (such as baked-to-mesh particles). However, if you generate new Unity Mesh objects (as opposed to modifying the contents of existing ones), you should be able to get the same result.

It’s probably worth noting that we use MeshResource.replace to update meshes every frame in many cases and haven’t seen the flickering you mention. I’m assuming it’s because of the combination of changing the mesh and switching to a new one, but I’ll try your example to see.

It’s probably also worth noting that when a mesh changes, we retransmit the entire mesh contents over PolySpatial. It doesn’t matter that only the vertices or only the triangles have changed. Because of that, you might have better luck just modifying the contents of the same mesh rather than switching between different meshes.

I can confirm that, with your example, either creating and setting a new Mesh each update or changing the contents of the same Mesh (that is, not changing the index) fixes the flickering issue.

However, I was unable to reproduce the issue in a non-Unity RealityKit project, which suggests that maybe we’re doing something that’s causing this issue. We’ll have to look into it further.