[Blocker][IN-68441] MR content rendered into Window gets scaled when moved

We render our game into a RenderTexture and show that on a plane inside a bounded VolumeCamera. What we do different then, is that in Swift we use a .windowStyle(.plain) instead of .windowStyle(.volumetric) to render our game inside a visionOS Window. This has worked in PS 0.4 to 0.7. Since PolySpatial 1.0.3, the content from the bounded camera gets scaled whenever the window is being moved along the z-axis. See this video for how this breaks our game.

We tried customizing the package and working around this ourselves, but we didn’t find anywhere where this scaling actually happens, so we assume it is happening somewhere in the closed source PolySpatial library. We know rendering into a Window is officially not supported, but it has worked so far.
Can someone point us to how we could circumvent or workaround this scaling as we would like to realease our Arcade game in March?
Maybe this issue was caused by this fix from PolySpatial visionOS 1.0.3:
Fixed issue with incorrect transforms on volumes when Display → Appearance → Window Zoom setting was changed.

PS:
We have also filed a bug (IN-68441) for this issue on February 12th, but it is still In Review….
We have also posted about this in VR category, but realised this might have been the wrong section, so I’m posting about it here again :wink:

Thanks! I’ve re-flagged the incident to our internal teams and we’ll take a look from there.

Thanks, Isaacs!
We also tried to find out where this message comes from as it seems related, but weren’t succesfull yet…
Volume 40028: window resized, uuid: CFE620C6-8843-491A-844B-63CB48473EE2 actualDimensions: SIMD3<Float>(4.0703845, 2.0351923, 1.3737547)

Hello Isaacs, I am the Swift developer in @mml’s team. Yesterday I finally found a way to workaround this problem, it’s not perfect so it still would be great if you could internally fix this issue, but at least it works for now. I wanted to share my solution here so it eventually also helps your developers.

So I moved PolySpatialWindow(…) window from the view to a variable and declared it as a @State variable so we can listen to changes of the window like this:

@State var polyWindow = PolySpatialWindow(UUID(),
                                          "Bounded-2.000x1.000x1.000",
                                          .init(2.000, 1.000, 1.000))

Then we are finally able to listen to the changes of the actualDimensions:

.onChange(of: polyWindow.actualDimensions) { oldValue, newValue in
    if oldValue.z == newValue.z {
        // This ensures that the normal window resizing with the handler works correctly
        // We use the difference of the starting window dimensions to the newly scaled window dimension and safe this as a scaling factor
        additionalResizeDimensions = startScalingDimensions / newValue
    } else {
        // When the window gets moved along the z-Axis, the window gets resized as well
        // We have to set a new start window dimension to calculate the difference of the starting dimension and the newly resized dimension correctly
        // Additionally, we also have to include the additional resize scaling that also already happened before
        startScalingDimensions = newValue * additionalResizeDimensions
    }
    
    actualDimensions = newValue * additionalResizeDimensions
}

And finally we can the apply the scaleEffect to the PolySpatialContentViewWrapper():

.scaleEffect(x: CGFloat(originalDimensions.x / actualDimensions.x),
             y: CGFloat(originalDimensions.y / actualDimensions.y),
             z: CGFloat(originalDimensions.z / actualDimensions.z))

As mentioned above, the solution works, but is not perfect as there is a short flickering when the window gets moved around as the content get’s scaled from PolySpatial and we re-scale it the opposite way again. Also, sometimes the onChange event does not get triggered correctly, ending up in a wrong scaling again - moving the window again fixes it then mostly.

It would be great to have some more options to pass for the window behaviour, like a preset of defined options like:

PolySpatialWindow(UUID(),
                  "Bounded-2.000x1.000x1.000",
                  .init(2.000, 1.000, 1.000)
                  options: [
                    .dynamicSizing: false,
                    // ...
                  ])

I guess internally this would then be just a simple if else handling for such scenarios and would be super helpful for others!

Additionally, I still would be very interested in how the PolySpatial package even detects the change of the window’s position - I couldn’t find any native way so far to listen to those changes - would be really interesting!

We saw that the Unity ticket already has been confirmed, so thank you for re-flagging the incident!