Key not found exception only for a single scene

I upgraded to the 6-1-1alpha version because to me, Unity 6 LTS was bugged when I was baking lights. Now all of a sudden, all of the objects in a scene are invisible and the following error message gets spammed in the console. If I load other scenes, they are invisible as well until I unload the one seemingly bugged scene. What’s going on?

KeyNotFoundException: The given key ‘28’ was not present in the dictionary.
System.Collections.Generic.Dictionary2[TKey,TValue].get_Item (TKey key) (at <321eb2db7c6d43ea8fc39b54eaca3452>:0) UnityEngine.Rendering.ProbeVolumeBakingSet.LoadStreambleAssetData[T] (UnityEngine.Rendering.ProbeVolumeStreamableAsset asset, System.Collections.Generic.List1[T] cellIndices) (at ./Library/PackageCache/com.unity.render-pipelines.core/Runtime/Lighting/ProbeVolume/ProbeVolumeBakingSet.cs:674)
UnityEngine.Rendering.ProbeVolumeBakingSet.ResolveSharedCellData (System.Collections.Generic.List1[T] cellIndices) (at ./Library/PackageCache/com.unity.render-pipelines.core/Runtime/Lighting/ProbeVolume/ProbeVolumeBakingSet.cs:911) UnityEngine.Rendering.ProbeVolumeBakingSet.ResolveCellData (System.Collections.Generic.List1[T] cellIndices) (at ./Library/PackageCache/com.unity.render-pipelines.core/Runtime/Lighting/ProbeVolume/ProbeVolumeBakingSet.cs:801)
UnityEngine.Rendering.ProbeReferenceVolume.LoadCells (System.Collections.Generic.List1[T] cellIndices) (at ./Library/PackageCache/com.unity.render-pipelines.core/Runtime/Lighting/ProbeVolume/ProbeReferenceVolume.cs:1427) UnityEngine.Rendering.ProbeReferenceVolume.PerformPendingLoading () (at ./Library/PackageCache/com.unity.render-pipelines.core/Runtime/Lighting/ProbeVolume/ProbeReferenceVolume.cs:1464) UnityEngine.Rendering.ProbeReferenceVolume.PerformPendingOperations () (at ./Library/PackageCache/com.unity.render-pipelines.core/Runtime/Lighting/ProbeVolume/ProbeReferenceVolume.cs:1554) UnityEngine.Rendering.Universal.UniversalRenderPipeline.RenderSingleCamera (UnityEngine.Rendering.ScriptableRenderContext context, UnityEngine.Rendering.Universal.UniversalCameraData cameraData) (at ./Library/PackageCache/com.unity.render-pipelines.universal/Runtime/UniversalRenderPipeline.cs:783) UnityEngine.Rendering.Universal.UniversalRenderPipeline.RenderSingleCameraInternal (UnityEngine.Rendering.ScriptableRenderContext context, UnityEngine.Camera camera, UnityEngine.Rendering.Universal.UniversalAdditionalCameraData& additionalCameraData) (at ./Library/PackageCache/com.unity.render-pipelines.universal/Runtime/UniversalRenderPipeline.cs:692) UnityEngine.Rendering.Universal.UniversalRenderPipeline.RenderSingleCameraInternal (UnityEngine.Rendering.ScriptableRenderContext context, UnityEngine.Camera camera) (at ./Library/PackageCache/com.unity.render-pipelines.universal/Runtime/UniversalRenderPipeline.cs:667) UnityEngine.Rendering.Universal.UniversalRenderPipeline.Render (UnityEngine.Rendering.ScriptableRenderContext renderContext, System.Collections.Generic.List1[T] cameras) (at ./Library/PackageCache/com.unity.render-pipelines.universal/Runtime/UniversalRenderPipeline.cs:484)
UnityEngine.Rendering.RenderPipeline.InternalRender (UnityEngine.Rendering.ScriptableRenderContext context, System.Collections.Generic.List`1[T] cameras) (at <6943fd8f6d32463ba3030a0774430673>:0)
UnityEngine.Rendering.RenderPipelineManager.DoRenderLoop_Internal (UnityEngine.Rendering.RenderPipelineAsset pipelineAsset, System.IntPtr loopPtr, UnityEngine.Object renderRequest, Unity.Collections.LowLevel.Unsafe.AtomicSafetyHandle safety) (at <6943fd8f6d32463ba3030a0774430673>:0)
UnityEngine.GUIUtility:ProcessEvent(Int32, IntPtr, Boolean&)

So after playing around a bit, doing the basic troubleshooting steps (like deleting the library folder to force Unity to reload stuff) I tried out different draw modes and weirdly enough, stuff appeared only in Debug Mode, so the engine recognised stuff was there in the scene. I did a few rebakes for both the lighting and the adaptive probe volumes and all of a sudden after a light bake, the issue was fixed. If anyone has any idea why this happened or why this fixed it I’m all ears.