I’m trying to import a USDZ file at runtime for my Vision Pro app in Unity 2022.3, but the packages and plugins I’ve found don’t support these specific requirements. USD 3.0.0-exp5 package doesn’t support Vision Pro and USD Import 1.0.0-pre.2 package doesn’t support runtime importing. I’ve looked into using TriLib 2 plugin, but it doesn’t support USDZ files.
The last thing I’ve tried doing was making calls to native Swift code from Unity. I was able to dynamically import a USDZ file this way and view the model in my app, but the USDZ model did not appear in Unity’s scene hierarchy. The app I’m working on is highly dependent on Unity so accessing the USDZ model from the scene hierarchy like any other GameObject is really important.
It is possible to load a USDZ file using Swift into PolySpatial’s RealityKit scene graph, though it won’t create GameObjects on the Unity side (just Entities in RealityKit). I’m not aware offhand of a package that allows loading USDZ as GameObjects at runtime for visionOS.
If you look at the SwiftUI sample in the most recent version of PolySpatial (2.1.2), you’ll see that it includes an example of using PolySpatialWindowManagerAccess to get the RealityKit Entity corresponding to the Unity instance ID of a GameObject:
let entities = PolySpatialWindowManagerAccess.entitiesForUnityInstanceId(id: instanceId)
for entity in entities {
if let identifier = PolySpatialWindowManagerAccess.identifierForEntity(entity: entity) {
CallCSharpCallback("recolor", identifier.unityInstanceId)
}
}
(It returns an array of Entities because, if you have multiple volume cameras, there will be one Entity to represent the GameObject for each volume camera.)
With this Entity reference, you should be able to load a USDZ file (using Entity.init(named:,in:), for example) and set its parent to the Entity representing the GameObject. If the GameObject is destroyed, the child loaded Entity will be removed along with it.
My Unity project can only go up to PolySpatial (1.3.11) for Unity 2022.3. The APIs mentioned only exist for PolySpatial (2.1.2) on Unity 6000.0, which I can’t upgrade to since my project is heavily reliant on Unity 2022.3.
I did find the following APIs available in PolySpatialRealityKit.
I’m not sure how I can use functions found in PolySpatialWindowManagerAccess with GetPolySpatialIdentifier(GameObject) found in PolySpatialObjectUtils class. GetPolySpatialIdentifier(GameObject) returns a ulong, but entityForIdentifier(id:) expects a PolySpatialRealityKit.PackedIdentifier type. I can’t find any documentation on PolySpatialRealityKit library in Swift, only PolySpatial for C# side. Any idea how I make the connection between these APIs?
The PackedIdentifier comprises two UInt64s, where the first one (id0) is the Unity instance ID extended to 64 bits and the second one (id1) is zero. So, you can create one as PackedIdentifier(id0: unityInstanceId, id1: 0). You may run into issues with sign-extension, as discussed in this thread.