Error when not using Clean Build

Currently, in our project I can only build to the Vision Pro Dev Kit when I do a clean build (in unity), otherwise this error appears:

Notice: an Unbounded volume configuration is required for ARKit features when building for Mixed Reality
UnityEditor.EditorApplication:Internal_CallGlobalEventHandler () (at /Users/bokken/build/output/unity/unity/Editor/Mono/EditorApplication.cs:474)

I am using the Mixed Reality sample but I of course have it configured to an unbounded volume. It works why I do a clean build. It is just annoying because it takes a lot longer.

Anyone got the same issue? Any extra infos I can provide?

Using Uniity 2022.3.11f1 with 0.43 Vision Stuff