Rendering stops/freezes on some mobile GPUs using Vulkan + Compute Shaders

Unity Engine
, , ,
1

For several weeks now, I’ve been having issues with vulkan rendering freezing sometimes (not always) once I start running a specific compute shader, and only on some mobile GPUs (seems to be Mali-GPUs). When it does, the last 3 frames are repeated over and over until the screen orientation changes or the app goes to the background, after that it’s just a blank screen. I have double and triple checked the compute shader, there’s no corrupt data in the buffers, no out of bounds reads or writes at any point, to the point where I suspect it’s not actually the compute shader that’s faulty, but something with how Unity uses it. I’ll attach the code just in case.

I’ve tried to diagnose the issue ofc, and there are certainly several things going wrong.
Firstly, a bunch of “gralloc4” errors show up right at the start:

2024-04-22 16:42:43.037 12826-12846 gralloc4                com.dev.indirecttest2             E  ERROR: Format allocation info not found for format: 3b
2024-04-22 16:42:43.037 12826-12846 gralloc4                com.dev.indirecttest2             E  ERROR: Format allocation info not found for format: 0
2024-04-22 16:42:43.037 12826-12846 gralloc4                com.dev.indirecttest2             E  Invalid base format! req_base_format = (<unrecognized format> 0x0), req_format = (<unrecognized format> 0x3b), type = 0x0
2024-04-22 16:42:43.037 12826-12846 gralloc4                com.dev.indirecttest2             E  ERROR: Unrecognized and/or unsupported format (<unrecognized format> 0x3b) and usage
                                                                                                    (CPU_READ_NEVER|CPU_WRITE_NEVER|GPU_TEXTURE|GPU_RENDER_TARGET|COMPOSER_OVERLAY 0xb00)
2024-04-22 16:42:43.037 12826-12846 gralloc4                com.dev.indirecttest2             E  ERROR: Format allocation info not found for format: 3b
2024-04-22 16:42:43.037 12826-12846 gralloc4                com.dev.indirecttest2             E  ERROR: Format allocation info not found for format: 0
2024-04-22 16:42:43.037 12826-12846 gralloc4                com.dev.indirecttest2             E  Invalid base format! req_base_format = (<unrecognized format> 0x0), req_format = (<unrecognized format> 0x3b), type = 0x0
2024-04-22 16:42:43.037 12826-12846 gralloc4                com.dev.indirecttest2             E  ERROR: Unrecognized and/or unsupported format (<unrecognized format> 0x3b) and usage
                                                                                                    (CPU_READ_NEVER|CPU_WRITE_NEVER|GPU_TEXTURE|GPU_RENDER_TARGET|COMPOSER_OVERLAY 0xb00)
2024-04-22 16:42:43.037 12826-12846 gralloc4                com.dev.indirecttest2             E  ERROR: Format allocation info not found for format: 3b
2024-04-22 16:42:43.037 12826-12846 gralloc4                com.dev.indirecttest2             E  ERROR: Format allocation info not found for format: 0
2024-04-22 16:42:43.037 12826-12846 gralloc4                com.dev.indirecttest2             E  Invalid base format! req_base_format = (<unrecognized format> 0x0), req_format = (<unrecognized format> 0x3b), type = 0x0
2024-04-22 16:42:43.037 12826-12846 gralloc4                com.dev.indirecttest2             E  ERROR: Unrecognized and/or unsupported format (<unrecognized format> 0x3b) and usage
                                                                                                    (CPU_READ_NEVER|CPU_WRITE_NEVER|GPU_TEXTURE|GPU_RENDER_TARGET|COMPOSER_OVERLAY 0xb00)
2024-04-22 16:42:43.037 12826-12846 gralloc4                com.dev.indirecttest2             E  ERROR: Format allocation info not found for format: 3b
2024-04-22 16:42:43.037 12826-12846 gralloc4                com.dev.indirecttest2             E  ERROR: Format allocation info not found for format: 0
2024-04-22 16:42:43.037 12826-12846 gralloc4                com.dev.indirecttest2             E  Invalid base format! req_base_format = (<unrecognized format> 0x0), req_format = (<unrecognized format> 0x3b), type = 0x0
2024-04-22 16:42:43.037 12826-12846 gralloc4                com.dev.indirecttest2             E  ERROR: Unrecognized and/or unsupported format (<unrecognized format> 0x3b) and usage
                                                                                                    (CPU_READ_NEVER|CPU_WRITE_NEVER|GPU_TEXTURE|GPU_RENDER_TARGET|COMPOSER_OVERLAY 0xb00)

Searching for them, I only found a few people having the same errors, but no resolution besides “don’t use vulkan” which isn’t really an option.

The app itself doesn’t actually crash, so there is no crash stacktrace. You can still attach a debugger and verify that it is running, and examine the graphics buffers used by the compute shader, and see that no invalid data was contained (They don’t update anymore though, once the rendering stops, so do the compute shaders).

So next I added the vulkan validation layers to my app, to see if I can get some error information that way. And there are tons:

2024-04-22 17:23:21.229 17480-17501 Unity                   com.dev.indirecttest2             D  Vulkan validation error./Runtime/GfxDevice/vulkan/VKDebug.cpp:144
2024-04-22 17:23:21.229 17480-17501 Unity                   com.dev.indirecttest2             D   #00 0x729a76d6e4 0x729a76d6e4 (libunity.so) vk::smile:ebugUtilsMessengerKHRDefaultCallback(VkDebugUtilsMessageSeverityFlagBitsEXT, unsigned
                                                                                                    int, VkDebugUtilsMessengerCallbackDataEXT const*, void*) 0x20c
2024-04-22 17:23:21.229 17480-17501 Unity                   com.dev.indirecttest2             D   #01 0x711c2fe130 0x711c2fe130 (libVkLayer_khronos_validation.so) ? 0x0
2024-04-22 17:23:21.229 17480-17501 Unity                   com.dev.indirecttest2             D   #02 0x711ba71d50 0x711ba71d50 (libVkLayer_khronos_validation.so) ? 0x0
2024-04-22 17:23:21.229 17480-17501 Unity                   com.dev.indirecttest2             D   #03 0x711bb17860 0x711bb17860 (libVkLayer_khronos_validation.so) ? 0x0
2024-04-22 17:23:21.229 17480-17501 Unity                   com.dev.indirecttest2             D   #04 0x711bd01764 0x711bd01764 (libVkLayer_khronos_validation.so) ? 0x0
2024-04-22 17:23:21.229 17480-17501 Unity                   com.dev.indirecttest2             D   #05 0x729a76855c 0x729a76855c (libunity.so) vk::CommandBuffer::Reset(unsigned int) 0x40
2024-04-22 17:23:21.229 17480-17501 Unity                   com.dev.indirecttest2             D   #06 0x729a754be8 0x729a754be8 (libunity.so) GfxDeviceVK::FlushPools_Internal() 0x22c
2024-04-22 17:23:21.229 17480-17501 Unity                   com.dev.indirecttest2             D   #07 0x729a74fc14 0x729a74fc14 (libunity.so) GfxDeviceVK::SubmitCurrentCommandBuffers(VkSemaphore_T*, bool) 0x1b4
2024-04-22 17:23:21.229 17480-17501 Unity                   com.dev.indirecttest2             D   #08 0x729a751b78 0x729a751b78 (libunity.so) GfxDeviceVK::PresentImage(bool) 0x228
2024-04-22 17:23:21.229 17480-17501 Unity                   com.dev.indirecttest2             D   #09 0x7299a31348 0x7299a31348 (libunity.so) PresentFrame() 0x58
2024-04-22 17:23:21.229 17480-17501 Unity                   com.dev.indirecttest2             D   #10 0x7299a2c418 0x7299a2c418 (libunity.so) PresentAndSync() 0x34
2024-04-22 17:23:21.229 17480-17501 Unity                   com.dev.indirecttest2             D   #11 0x7299a2c3c8 0x7299a2c3c8 (libunity.so) PresentAfterDraw(GfxDevicePresentMode) 0x10
2024-04-22 17:23:21.229 17480-17501 Unity                   com.dev.indirecttest2             D   #12 0x7299a2da20 0x7299a2da20 (libunity.so) InitPlayerLoopCallbacks()::PostLateUpdatePresentAfterDrawRegistrator::Forward() 0x80
2024-04-22 17:23:21.229 17480-17501 Unity                   com.dev.indirecttest2             D   #13 0x7299a1c3fc 0x7299a1c3fc (libunity.so) ExecutePlayerLoop(NativePlayerLoopSystem*) 0x84
2024-04-22 17:23:21.229 17480-17501 Unity                   com.dev.indirecttest2             D   #14 0x7299a1c43c 0x7299a1c43c (libunity.so) ExecutePlayerLoop(NativePlayerLoopSystem*) 0xc4
2024-04-22 17:23:21.229 17480-17501 Unity                   com.dev.indirecttest2             D   #15 0x7299a1c730 0x7299a1c730 (libunity.so) PlayerLoop() 0x144
2024-04-22 17:23:21.229 17480-17501 Unity                   com.dev.indirecttest2             D   #16 0x7299ec2040 0x7299ec2040 (libunity.so) UnityPlayerLoop() 0x3d0
2024-04-22 17:23:21.229 17480-17501 Unity                   com.dev.indirecttest2             D   #17 0x7299ec3350 0x7299ec3350 (libunity.so) nativeRender(_JNIEnv*, _jobject*) 0x54
2024-04-22 17:23:21.229 17480-17501 Unity                   com.dev.indirecttest2             D   #18 0x7333551e30 0x7333551e30 (libart.so) art_quick_generic_jni_trampoline 0x90
2024-04-22 17:23:21.229 17480-17501 Unity                   com.dev.indirecttest2             D   #19 0x58c00d04 0x58c00d04 (memfd:jit-cache (deleted)) ? 0x0
2024-04-22 17:23:21.229 17480-17501 Unity                   com.dev.indirecttest2             E  VULKAN: VALIDATION ERROR: Validation Error: [ VUID-vkResetCommandPool-commandPool-00040 ] Object 0: handle = 0xb400007345e64530, type
                                                                                                    = VK_OBJECT_TYPE_COMMAND_BUFFER; Object 1: handle = 0x70526c00000000f3, type = VK_OBJECT_TYPE_COMMAND_POOL; | MessageID = 0xb53e2331
                                                                                                    | vkResetCommandPool():  (VkCommandBuffer 0xb400007345e64530[]) is in use. The Vulkan spec states: All VkCommandBuffer objects allocated from commandPool must
                                                                                                    not be in the pending state (https://www.khronos.org/registry/vulkan/specs/1.3-extensions/html/vkspec.html#VUID-vkResetCommandPool-commandPool-00040)
                                                                                                      CommandBuffer labels: FinalBlit | ScriptableRenderer.Execute: URP-Balanced-Renderer | UniversalRenderPipeline.RenderSingleCameraInternal: Main Camera
                                                                                                    | FrameTime.GPU
                                                                                                      Objects:
                                                                                                        #0: { type = VK_OBJECT_TYPE_COMMAND_BUFFER, handle = 0xb400007345e64530 }
                                                                                                        #1: { type = VK_OBJECT_TYPE_COMMAND_POOL, handle = 0x70526c00000000f3 }
2024-04-22 17:23:21.229 17480-17501 Unity                   com.dev.indirecttest2             D  Vulkan validation error./Runtime/GfxDevice/vulkan/VKDebug.cpp:1
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #00 0x729a76d6e4 0x729a76d6e4 (libunity.so) vk::smile:ebugUtilsMessengerKHRDefaultCallback(VkDebugUtilsMessageSeverityFlagBitsEXT, unsigned
                                                                                                    int, VkDebugUtilsMessengerCallbackDataEXT const*, void*) 0x20c
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #01 0x711c2fe130 0x711c2fe130 (libVkLayer_khronos_validation.so) ? 0x0
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #02 0x711ba71d50 0x711ba71d50 (libVkLayer_khronos_validation.so) ? 0x0
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #03 0x711bc3e7ac 0x711bc3e7ac (libVkLayer_khronos_validation.so) ? 0x0
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #04 0x711bcf93c8 0x711bcf93c8 (libVkLayer_khronos_validation.so) ? 0x0
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #05 0x729a751528 0x729a751528 (libunity.so) vk::FrameTracking::GetFence() 0x58
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #06 0x729a74fbe4 0x729a74fbe4 (libunity.so) GfxDeviceVK::SubmitCurrentCommandBuffers(VkSemaphore_T*, bool) 0x184
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #07 0x729a747cf8 0x729a747cf8 (libunity.so) GfxDeviceVKBase::EnsureCurrentCommandBuffer(vk::CommandBuffer::Type, bool) 0x7c
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #08 0x729a74277c 0x729a74277c (libunity.so) GfxDeviceVKBase::SetShadersThreadable(GpuProgram**, GpuProgramParameters const**, unsigned char
                                                                                                    const* const*) 0x40
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #09 0x7299b3ee7c 0x7299b3ee7c (libunity.so) ShaderLab::ShaderState::ApplyShaderState(unsigned int, ShaderPropertySheet const*, ShaderPassContext
                                                                                                    const&, keywords::LocalKeywordState const&, Shader const*, ShaderLab::Pass const*, DynamicBranchState*, ShaderLab::SubPrograms*, DeviceRenderStateBlock
                                                                                                    const*, int, int, ShaderLab::Pass const*, ShaderLab::Pass const*) const 0x39c
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #10 0x7299b3ea7c 0x7299b3ea7c (libunity.so) ShaderLab::Pass::ApplyPass(unsigned int, ShaderPropertySheet
                                                                                                    const*, DynamicBranchState*, ShaderPassContext&, keywords::LocalKeywordState const&, Shader const*, int, int, ShaderLab::GrabPasses
                                                                                                    const*, ShaderLab::SubPrograms*, DeviceRenderStateBlock const*, ShaderLab::Pass const*, ShaderLab::Pass const*) 0xf0
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #11 0x729995faa8 0x729995faa8
                                                                                                    (libunity.so) RenderingCommandBuffer::ExecuteCommandBufferWithState(ShaderPassContext&, RenderNodeQueue&, RenderingCommandBufferState*, unsigned
                                                                                                    int, ComputeQueueType) const 0x24b4
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #12 0x729995cedc 0x729995cedc (libunity.so) RenderingCommandBuffer::ExecuteCommandBuffer(ShaderPassContext&, RenderNodeQueue&, unsigned
                                                                                                    int, dynamic_array<std::__ndk1::pair<ShaderLab::FastPropertyName, RenderTexture*>, 0ul>*, ComputeQueueType, RenderTexture*) const 0x50
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #13 0x7299976324 0x7299976324 (libunity.so) ScriptableRenderContext::ExecuteScriptableRenderLoop() 0xa3c
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #14 0x7292eef4d8 0x7292eef4d8 (libil2cpp.so) ScriptableRenderContext_Submit_Internal_mAE1E99BDF4C4E5A0FD9A0147E460E25ABB84BF14 0x40
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #15 0x7292ef24e0 0x7292ef24e0 (libil2cpp.so) ScriptableRenderContext_Submit_mF417223097ECEB2768EE736BE00A7F94DA3BDD26 0x138
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #16 0x7292ad9b40 0x7292ad9b40 (libil2cpp.so) UniversalRenderPipeline_RenderSingleCamera_m4886F9EF5EEF5C6A85E28E8A0D5E8CCC7809B49C 0x363c
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #17 0x7292acf8a8 0x7292acf8a8 (libil2cpp.so) UniversalRenderPipeline_RenderCameraStack_m47BBC0B4111D83BB6EE3275C9572BFBF2F5451B9 0x3fbc
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #18 0x7292ac8f08 0x7292ac8f08 (libil2cpp.so) UniversalRenderPipeline_Render_m15A42AB44C14AB4DCA7EF0B915964D46B643D50E 0x1140
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #19 0x7292ac7db8 0x7292ac7db8
                                                                                                    (libil2cpp.so) VirtualActionInvoker2<ScriptableRenderContext_t5AB09B3602BEB456E0DC3D53926D3A3BDAF08E36, List_1_tD2FA3273746E404D72561E8324608D18B52B533E*>::Invoke(unsigned
                                                                                                    short, Il2CppObject*, ScriptableRenderContext_t5AB09B3602BEB456E0DC3D53926D3A3BDAF08E36, List_1_tD2FA3273746E404D72561E8324608D18B52B533E*) 0x50
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #20 0x7292edd804 0x7292edd804 (libil2cpp.so) RenderPipeline_InternalRender_mCB77395158F4572F348D8227BA9127ABF1C9C5BE 0x324
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #21 0x7292ee5144 0x7292ee5144 (libil2cpp.so) RenderPipelineManager_DoRenderLoop_Internal_mB646C8738F4A9859101F3BE94809E2E10BBDB1FB 0x60c
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #22 0x7290e87c78 0x7290e87c78 (libil2cpp.so) RuntimeInvoker_FalseVoid_t4861ACF8F4594C3437BB48B6E56783494B843915_RuntimeObject_IntPtr_t_RuntimeObject(void
                                                                                                    (*)(), MethodInfo const*, void*, void**, void*) 0x44
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #23 0x7293e7a810 0x7293e7a810 (libil2cpp.so) il2cpp::vm::Runtime::InvokeWithThrow(MethodInfo const*, void*, void**) 0x58
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #24 0x7293e7a75c 0x7293e7a75c (libil2cpp.so) il2cpp::vm::Runtime::Invoke(MethodInfo const*, void*, void**, Il2CppException**) 0xb0
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #25 0x7293d2fd68 0x7293d2fd68 (libil2cpp.so) il2cpp_runtime_invoke 0x2c
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #26 0x7299dd41a0 0x7299dd41a0
                                                                                                    (libunity.so) scripting_method_invoke(ScriptingMethodPtr, ScriptingObjectPtr, ScriptingArguments&, ScriptingExceptionPtr*, bool) 0xb0
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #27 0x7299de87fc 0x7299de87fc (libunity.so) ScriptingInvocation::Invoke(ScriptingExceptionPtr*, bool) 0x88
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #28 0x7299efbe60 0x7299efbe60
                                                                                                    (libunity.so) Scripting::UnityEngine::Rendering::RenderPipelineManagerProxy::smile:oRenderLoop_Internal(ScriptingObjectPtr, void*, Scripting::UnityEngine::ObjectProxy, ScriptingExceptionPtr*) 0x78
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #29 0x7299976ae4 0x7299976ae4 (libunity.so) ScriptableRenderContext::ExtractAndExecuteRenderPipeline(dynamic_array<Camera*, 0ul> const&, void (*)(SceneNode
                                                                                                    const*, AABB const*, IndexList&, SceneCullingParameters const*), void*, ScriptingObjectPtr) 0x130
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #30 0x729a57a174 0x729a57a174 (libunity.so) RenderManager::RenderCamerasWithScriptableRenderLoop(int) 0x298
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #31 0x729a57a6e8 0x729a57a6e8 (libunity.so) RenderManager::RenderCameras(int, void (*)(), void (*)()) 0x38
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #32 0x7299a2c714 0x7299a2c714 (libunity.so) PlayerRender(bool) 0x2cc
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #33 0x7299a2d85c 0x7299a2d85c (libunity.so) InitPlayerLoopCallbacks()::PostLateUpdateFinishFrameRenderingRegistrator::Forward() 0x44
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #34 0x7299a1c3fc 0x7299a1c3fc (libunity.so) ExecutePlayerLoop(NativePlayerLoopSystem*) 0x84
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #35 0x7299a1c43c 0x7299a1c43c (libunity.so) ExecutePlayerLoop(NativePlayerLoopSystem*) 0xc4
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #36 0x7299a1c730 0x7299a1c730 (libunity.so) PlayerLoop() 0x144
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #37 0x7299ec2040 0x7299ec2040 (libunity.so) UnityPlayerLoop() 0x3d0
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #38 0x7299ec3350 0x7299ec3350 (libunity.so) nativeRender(_JNIEnv*, _jobject*) 0x54
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #39 0x7333551e30 0x7333551e30 (libart.so) art_quick_generic_jni_trampoline 0x90
2024-04-22 17:23:21.243 17480-17501 Unity                   com.dev.indirecttest2             D   #40 0x58c00d04 0x58c00d04 (memfd:jit-cache (deleted)) ? 0x0
2024-04-22 17:23:21.244 17480-17501 Unity                   com.dev.indirecttest2             E  VULKAN: VALIDATION ERROR: Validation Error: [ VUID-vkResetFences-pFences-01123 ] Object 0: handle = 0xab64de0000000020, type
                                                                                                    = VK_OBJECT_TYPE_FENCE; | MessageID = 0x68a5074e | vkResetFences(): pFences[0] (VkFence 0xab64de0000000020[]) is in use. The Vulkan spec states: Each element
                                                                                                    of pFences must not be currently associated with any queue command that has not yet completed execution on that queue
                                                                                                    (https://www.khronos.org/registry/vulkan/specs/1.3-extensions/html/vkspec.html#VUID-vkResetFences-pFences-01123)
                                                                                                      Objects:
                                                                                                        #0: { type = VK_OBJECT_TYPE_FENCE, handle = 0xab64de0000000020 }
                                                                                                  
                                                                                                    UnityEngine.Rendering.ScriptableRenderContext:Submit() (at \home\bokken\build\output\unity\unity\Runtime\Export\RenderPipeline\ScriptableRenderContext.cs:117)
                                                                                                    UnityEngine.Rendering.Universal.UniversalRenderPipeline:RenderSingleCamera(ScriptableRenderContext, CameraData&) (at
                                                                                                    .\Library\PackageCache\com.unity.render-pipelines.universal@16.0.6\Runtime\UniversalRenderPipeline.cs:798)
                                                                                                    UnityEngine.Rendering.Universal.UniversalRend
2024-04-22 17:23:21.244 17480-17501 Unity                   com.dev.indirecttest2             D  Vulkan validation error./Runtime/GfxDevice/vulkan/VKDebug.cpp:144
44
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #00 0x729a76d6e4 0x729a76d6e4 (libunity.so) vk::smile:ebugUtilsMessengerKHRDefaultCallback(VkDebugUtilsMessageSeverityFlagBitsEXT, unsigned
                                                                                                    int, VkDebugUtilsMessengerCallbackDataEXT const*, void*) 0x20c
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #01 0x711c2fe130 0x711c2fe130 (libVkLayer_khronos_validation.so) ? 0x0
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #02 0x711ba71d50 0x711ba71d50 (libVkLayer_khronos_validation.so) ? 0x0
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #03 0x711ba9a020 0x711ba9a020 (libVkLayer_khronos_validation.so) ? 0x0
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #04 0x711bd0204c 0x711bd0204c (libVkLayer_khronos_validation.so) ? 0x0
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #05 0x729a768824 0x729a768824 (libunity.so) vk::CommandBuffer::Begin(vk::CommandBuffer::Type, VkRenderPass_T*, VkFramebuffer_T*, unsigned int, unsigned
                                                                                                    long, bool) 0xa4
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #06 0x729a7867b8 0x729a7867b8 (libunity.so) vk::TaskExecutor::ExecuteCommandbuffers(vk::CommandBuffer**, int) 0x154
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #07 0x729a784b10 0x729a784b10 (libunity.so) vk::TaskExecutor::smile:oFlush(VkFence_T*, VkSemaphore_T*, unsigned long, bool) 0x38
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #08 0x729a74fc00 0x729a74fc00 (libunity.so) GfxDeviceVK::SubmitCurrentCommandBuffers(VkSemaphore_T*, bool) 0x1a0
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #09 0x729a747cf8 0x729a747cf8 (libunity.so) GfxDeviceVKBase::EnsureCurrentCommandBuffer(vk::CommandBuffer::Type, bool) 0x7c
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #10 0x729a74277c 0x729a74277c (libunity.so) GfxDeviceVKBase::SetShadersThreadable(GpuProgram**, GpuProgramParameters const**, unsigned char
                                                                                                    const* const*) 0x40
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #11 0x7299b3ee7c 0x7299b3ee7c (libunity.so) ShaderLab::ShaderState::ApplyShaderState(unsigned int, ShaderPropertySheet const*, ShaderPassContext
                                                                                                    const&, keywords::LocalKeywordState const&, Shader const*, ShaderLab::Pass const*, DynamicBranchState*, ShaderLab::SubPrograms*, DeviceRenderStateBlock
                                                                                                    const*, int, int, ShaderLab::Pass const*, ShaderLab::Pass const*) const 0x39c
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #12 0x7299b3ea7c 0x7299b3ea7c (libunity.so) ShaderLab::Pass::ApplyPass(unsigned int, ShaderPropertySheet
                                                                                                    const*, DynamicBranchState*, ShaderPassContext&, keywords::LocalKeywordState const&, Shader const*, int, int, ShaderLab::GrabPasses
                                                                                                    const*, ShaderLab::SubPrograms*, DeviceRenderStateBlock const*, ShaderLab::Pass const*, ShaderLab::Pass const*) 0xf0
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #13 0x729995faa8 0x729995faa8
                                                                                                    (libunity.so) RenderingCommandBuffer::ExecuteCommandBufferWithState(ShaderPassContext&, RenderNodeQueue&, RenderingCommandBufferState*, unsigned
                                                                                                    int, ComputeQueueType) const 0x24b4
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #14 0x729995cedc 0x729995cedc (libunity.so) RenderingCommandBuffer::ExecuteCommandBuffer(ShaderPassContext&, RenderNodeQueue&, unsigned
                                                                                                    int, dynamic_array<std::__ndk1::pair<ShaderLab::FastPropertyName, RenderTexture*>, 0ul>*, ComputeQueueType, RenderTexture*) const 0x50
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #15 0x7299976324 0x7299976324 (libunity.so) ScriptableRenderContext::ExecuteScriptableRenderLoop() 0xa3c
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #16 0x7292eef4d8 0x7292eef4d8 (libil2cpp.so) ScriptableRenderContext_Submit_Internal_mAE1E99BDF4C4E5A0FD9A0147E460E25ABB84BF14 0x40
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #17 0x7292ef24e0 0x7292ef24e0 (libil2cpp.so) ScriptableRenderContext_Submit_mF417223097ECEB2768EE736BE00A7F94DA3BDD26 0x138
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #18 0x7292ad9b40 0x7292ad9b40 (libil2cpp.so) UniversalRenderPipeline_RenderSingleCamera_m4886F9EF5EEF5C6A85E28E8A0D5E8CCC7809B49C 0x363c
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #19 0x7292acf8a8 0x7292acf8a8 (libil2cpp.so) UniversalRenderPipeline_RenderCameraStack_m47BBC0B4111D83BB6EE3275C9572BFBF2F5451B9 0x3fbc
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #20 0x7292ac8f08 0x7292ac8f08 (libil2cpp.so) UniversalRenderPipeline_Render_m15A42AB44C14AB4DCA7EF0B915964D46B643D50E 0x1140
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #21 0x7292ac7db8 0x7292ac7db8
                                                                                                    (libil2cpp.so) VirtualActionInvoker2<ScriptableRenderContext_t5AB09B3602BEB456E0DC3D53926D3A3BDAF08E36, List_1_tD2FA3273746E404D72561E8324608D18B52B533E*>::Invoke(unsigned
                                                                                                    short, Il2CppObject*, ScriptableRenderContext_t5AB09B3602BEB456E0DC3D53926D3A3BDAF08E36, List_1_tD2FA3273746E404D72561E8324608D18B52B533E*) 0x50
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #22 0x7292edd804 0x7292edd804 (libil2cpp.so) RenderPipeline_InternalRender_mCB77395158F4572F348D8227BA9127ABF1C9C5BE 0x324
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #23 0x7292ee5144 0x7292ee5144 (libil2cpp.so) RenderPipelineManager_DoRenderLoop_Internal_mB646C8738F4A9859101F3BE94809E2E10BBDB1FB 0x60c
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #24 0x7290e87c78 0x7290e87c78 (libil2cpp.so) RuntimeInvoker_FalseVoid_t4861ACF8F4594C3437BB48B6E56783494B843915_RuntimeObject_IntPtr_t_RuntimeObject(void
                                                                                                    (*)(), MethodInfo const*, void*, void**, void*) 0x44
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #25 0x7293e7a810 0x7293e7a810 (libil2cpp.so) il2cpp::vm::Runtime::InvokeWithThrow(MethodInfo const*, void*, void**) 0x58
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #26 0x7293e7a75c 0x7293e7a75c (libil2cpp.so) il2cpp::vm::Runtime::Invoke(MethodInfo const*, void*, void**, Il2CppException**) 0xb0
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #27 0x7293d2fd68 0x7293d2fd68 (libil2cpp.so) il2cpp_runtime_invoke 0x2c
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #28 0x7299dd41a0 0x7299dd41a0
                                                                                                    (libunity.so) scripting_method_invoke(ScriptingMethodPtr, ScriptingObjectPtr, ScriptingArguments&, ScriptingExceptionPtr*, bool) 0xb0
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #29 0x7299de87fc 0x7299de87fc (libunity.so) ScriptingInvocation::Invoke(ScriptingExceptionPtr*, bool) 0x88
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #30 0x7299efbe60 0x7299efbe60
                                                                                                    (libunity.so) Scripting::UnityEngine::Rendering::RenderPipelineManagerProxy::smile:oRenderLoop_Internal(ScriptingObjectPtr, void*, Scripting::UnityEngine::ObjectProxy, ScriptingExceptionPtr*) 0x78
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #31 0x7299976ae4 0x7299976ae4 (libunity.so) ScriptableRenderContext::ExtractAndExecuteRenderPipeline(dynamic_array<Camera*, 0ul> const&, void (*)(SceneNode
                                                                                                    const*, AABB const*, IndexList&, SceneCullingParameters const*), void*, ScriptingObjectPtr) 0x130
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #32 0x729a57a174 0x729a57a174 (libunity.so) RenderManager::RenderCamerasWithScriptableRenderLoop(int) 0x298
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #33 0x729a57a6e8 0x729a57a6e8 (libunity.so) RenderManager::RenderCameras(int, void (*)(), void (*)()) 0x38
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #34 0x7299a2c714 0x7299a2c714 (libunity.so) PlayerRender(bool) 0x2cc
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #35 0x7299a2d85c 0x7299a2d85c (libunity.so) InitPlayerLoopCallbacks()::PostLateUpdateFinishFrameRenderingRegistrator::Forward() 0x44
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #36 0x7299a1c3fc 0x7299a1c3fc (libunity.so) ExecutePlayerLoop(NativePlayerLoopSystem*) 0x84
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #37 0x7299a1c43c 0x7299a1c43c (libunity.so) ExecutePlayerLoop(NativePlayerLoopSystem*) 0xc4
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #38 0x7299a1c730 0x7299a1c730 (libunity.so) PlayerLoop() 0x144
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #39 0x7299ec2040 0x7299ec2040 (libunity.so) UnityPlayerLoop() 0x3d0
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #40 0x7299ec3350 0x7299ec3350 (libunity.so) nativeRender(_JNIEnv*, _jobject*) 0x54
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #41 0x7333551e30 0x7333551e30 (libart.so) art_quick_generic_jni_trampoline 0x90
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             D   #42 0x58c00d04 0x58c00d04 (memfd:jit-cache (deleted)) ? 0x0
2024-04-22 17:23:21.245 17480-17501 Unity                   com.dev.indirecttest2             E  VULKAN: VALIDATION ERROR: Validation Error: [ VUID-vkBeginCommandBuffer-commandBuffer-00049 ] Object 0: handle = 0xb400007345e64530, type
                                                                                                    = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x84029a9f | vkBeginCommandBuffer():  on active VkCommandBuffer 0xb400007345e64530[] before it has
                                                                                                    completed. You must check command buffer fence before this call. The Vulkan spec states: commandBuffer must not be in the recording or pending state
                                                                                                    (https://www.khronos.org/registry/vulkan/specs/1.3-extensions/html/vkspec.html#VUID-vkBeginCommandBuffer-commandBuffer-00049)
                                                                                                      Objects:
                                                                                                        #0: { type = VK_OBJECT_TYPE_COMMAND_BUFFER, handle = 0xb400007345e64530 }
                                                                                                  
                                                                                                    UnityEngine.Rendering.ScriptableRenderContext:Submit() (at \home\bokken\build\output\unity\unity\Runtime\Export\RenderPipeline\ScriptableRenderContext.cs:117)
                                                                                                    UnityEngine.Rendering.Universal.UniversalRenderPipeline:RenderSingleCamera(ScriptableRenderContext, CameraData&) (at
                                                                                                    .\Library\PackageCache\com.unity.render-pipelines.universal@16.0.6\Runtime\UniversalRenderPipeline
2024-04-22 17:23:21.246 17480-17501 Unity                   com.dev.indirecttest2             D  Vulkan validation error./Runtime/GfxDevice/vulkan/VKDebug.cpp:144
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #00 0x729a76d6e4 0x729a76d6e4 (libunity.so) vk::smile:ebugUtilsMessengerKHRDefaultCallback(VkDebugUtilsMessageSeverityFlagBitsEXT, unsigned
                                                                                                    int, VkDebugUtilsMessengerCallbackDataEXT const*, void*) 0x20c
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #01 0x711c2fe130 0x711c2fe130 (libVkLayer_khronos_validation.so) ? 0x0
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #02 0x711ba71d50 0x711ba71d50 (libVkLayer_khronos_validation.so) ? 0x0
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #03 0x711bbc1218 0x711bbc1218 (libVkLayer_khronos_validation.so) ? 0x0
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #04 0x711bbc1808 0x711bbc1808 (libVkLayer_khronos_validation.so) ? 0x0
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #05 0x711bbbe090 0x711bbbe090 (libVkLayer_khronos_validation.so) ? 0x0
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #06 0x711bbbce04 0x711bbbce04 (libVkLayer_khronos_validation.so) ? 0x0
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #07 0x711bcf5b20 0x711bcf5b20 (libVkLayer_khronos_validation.so) ? 0x0
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #08 0x729a784cec 0x729a784cec (libunity.so) vk::TaskExecutor::smile:oFlush(VkFence_T*, VkSemaphore_T*, unsigned long, bool) 0x214
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #09 0x729a74fc00 0x729a74fc00 (libunity.so) GfxDeviceVK::SubmitCurrentCommandBuffers(VkSemaphore_T*, bool) 0x1a0
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #10 0x729a747cf8 0x729a747cf8 (libunity.so) GfxDeviceVKBase::EnsureCurrentCommandBuffer(vk::CommandBuffer::Type, bool) 0x7c
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #11 0x729a74277c 0x729a74277c (libunity.so) GfxDeviceVKBase::SetShadersThreadable(GpuProgram**, GpuProgramParameters const**, unsigned char
                                                                                                    const* const*) 0x40
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #12 0x7299b3ee7c 0x7299b3ee7c (libunity.so) ShaderLab::ShaderState::ApplyShaderState(unsigned int, ShaderPropertySheet const*, ShaderPassContext
                                                                                                    const&, keywords::LocalKeywordState const&, Shader const*, ShaderLab::Pass const*, DynamicBranchState*, ShaderLab::SubPrograms*, DeviceRenderStateBlock
                                                                                                    const*, int, int, ShaderLab::Pass const*, ShaderLab::Pass const*) const 0x39c
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #13 0x7299b3ea7c 0x7299b3ea7c (libunity.so) ShaderLab::Pass::ApplyPass(unsigned int, ShaderPropertySheet
                                                                                                    const*, DynamicBranchState*, ShaderPassContext&, keywords::LocalKeywordState const&, Shader const*, int, int, ShaderLab::GrabPasses
                                                                                                    const*, ShaderLab::SubPrograms*, DeviceRenderStateBlock const*, ShaderLab::Pass const*, ShaderLab::Pass const*) 0xf0
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #14 0x729995faa8 0x729995faa8
                                                                                                    (libunity.so) RenderingCommandBuffer::ExecuteCommandBufferWithState(ShaderPassContext&, RenderNodeQueue&, RenderingCommandBufferState*, unsigned
                                                                                                    int, ComputeQueueType) const 0x24b4
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #15 0x729995cedc 0x729995cedc (libunity.so) RenderingCommandBuffer::ExecuteCommandBuffer(ShaderPassContext&, RenderNodeQueue&, unsigned
                                                                                                    int, dynamic_array<std::__ndk1::pair<ShaderLab::FastPropertyName, RenderTexture*>, 0ul>*, ComputeQueueType, RenderTexture*) const 0x50
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #16 0x7299976324 0x7299976324 (libunity.so) ScriptableRenderContext::ExecuteScriptableRenderLoop() 0xa3c
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #17 0x7292eef4d8 0x7292eef4d8 (libil2cpp.so) ScriptableRenderContext_Submit_Internal_mAE1E99BDF4C4E5A0FD9A0147E460E25ABB84BF14 0x40
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #18 0x7292ef24e0 0x7292ef24e0 (libil2cpp.so) ScriptableRenderContext_Submit_mF417223097ECEB2768EE736BE00A7F94DA3BDD26 0x138
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #19 0x7292ad9b40 0x7292ad9b40 (libil2cpp.so) UniversalRenderPipeline_RenderSingleCamera_m4886F9EF5EEF5C6A85E28E8A0D5E8CCC7809B49C 0x363c
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #20 0x7292acf8a8 0x7292acf8a8 (libil2cpp.so) UniversalRenderPipeline_RenderCameraStack_m47BBC0B4111D83BB6EE3275C9572BFBF2F5451B9 0x3fbc
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #21 0x7292ac8f08 0x7292ac8f08 (libil2cpp.so) UniversalRenderPipeline_Render_m15A42AB44C14AB4DCA7EF0B915964D46B643D50E 0x1140
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #22 0x7292ac7db8 0x7292ac7db8
                                                                                                    (libil2cpp.so) VirtualActionInvoker2<ScriptableRenderContext_t5AB09B3602BEB456E0DC3D53926D3A3BDAF08E36, List_1_tD2FA3273746E404D72561E8324608D18B52B533E*>::Invoke(unsigned
                                                                                                    short, Il2CppObject*, ScriptableRenderContext_t5AB09B3602BEB456E0DC3D53926D3A3BDAF08E36, List_1_tD2FA3273746E404D72561E8324608D18B52B533E*) 0x50
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #23 0x7292edd804 0x7292edd804 (libil2cpp.so) RenderPipeline_InternalRender_mCB77395158F4572F348D8227BA9127ABF1C9C5BE 0x324
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #24 0x7292ee5144 0x7292ee5144 (libil2cpp.so) RenderPipelineManager_DoRenderLoop_Internal_mB646C8738F4A9859101F3BE94809E2E10BBDB1FB 0x60c
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #25 0x7290e87c78 0x7290e87c78 (libil2cpp.so) RuntimeInvoker_FalseVoid_t4861ACF8F4594C3437BB48B6E56783494B843915_RuntimeObject_IntPtr_t_RuntimeObject(void
                                                                                                    (*)(), MethodInfo const*, void*, void**, void*) 0x44
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #26 0x7293e7a810 0x7293e7a810 (libil2cpp.so) il2cpp::vm::Runtime::InvokeWithThrow(MethodInfo const*, void*, void**) 0x58
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #27 0x7293e7a75c 0x7293e7a75c (libil2cpp.so) il2cpp::vm::Runtime::Invoke(MethodInfo const*, void*, void**, Il2CppException**) 0xb0
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #28 0x7293d2fd68 0x7293d2fd68 (libil2cpp.so) il2cpp_runtime_invoke 0x2c
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #29 0x7299dd41a0 0x7299dd41a0
                                                                                                    (libunity.so) scripting_method_invoke(ScriptingMethodPtr, ScriptingObjectPtr, ScriptingArguments&, ScriptingExceptionPtr*, bool) 0xb0
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #30 0x7299de87fc 0x7299de87fc (libunity.so) ScriptingInvocation::Invoke(ScriptingExceptionPtr*, bool) 0x88
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #31 0x7299efbe60 0x7299efbe60
                                                                                                    (libunity.so) Scripting::UnityEngine::Rendering::RenderPipelineManagerProxy::smile:oRenderLoop_Internal(ScriptingObjectPtr, void*, Scripting::UnityEngine::ObjectProxy, ScriptingExceptionPtr*) 0x78
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #32 0x7299976ae4 0x7299976ae4 (libunity.so) ScriptableRenderContext::ExtractAndExecuteRenderPipeline(dynamic_array<Camera*, 0ul> const&, void (*)(SceneNode
                                                                                                    const*, AABB const*, IndexList&, SceneCullingParameters const*), void*, ScriptingObjectPtr) 0x130
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #33 0x729a57a174 0x729a57a174 (libunity.so) RenderManager::RenderCamerasWithScriptableRenderLoop(int) 0x298
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #34 0x729a57a6e8 0x729a57a6e8 (libunity.so) RenderManager::RenderCameras(int, void (*)(), void (*)()) 0x38
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #35 0x7299a2c714 0x7299a2c714 (libunity.so) PlayerRender(bool) 0x2cc
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #36 0x7299a2d85c 0x7299a2d85c (libunity.so) InitPlayerLoopCallbacks()::PostLateUpdateFinishFrameRenderingRegistrator::Forward() 0x44
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #37 0x7299a1c3fc 0x7299a1c3fc (libunity.so) ExecutePlayerLoop(NativePlayerLoopSystem*) 0x84
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #38 0x7299a1c43c 0x7299a1c43c (libunity.so) ExecutePlayerLoop(NativePlayerLoopSystem*) 0xc4
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #39 0x7299a1c730 0x7299a1c730 (libunity.so) PlayerLoop() 0x144
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #40 0x7299ec2040 0x7299ec2040 (libunity.so) UnityPlayerLoop() 0x3d0
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #41 0x7299ec3350 0x7299ec3350 (libunity.so) nativeRender(_JNIEnv*, _jobject*) 0x54
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #42 0x7333551e30 0x7333551e30 (libart.so) art_quick_generic_jni_trampoline 0x90
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             D   #43 0x58c00d04 0x58c00d04 (memfd:jit-cache (deleted)) ? 0x0
2024-04-22 17:23:21.247 17480-17501 Unity                   com.dev.indirecttest2             E  VULKAN: VALIDATION ERROR: Validation Error: [ VUID-vkQueueSubmit-pCommandBuffers-00071 ] | MessageID = 0x2e2f4d65
                                                                                                    | vkQueueSubmit(): pSubmits[0].pCommandBuffers[0] VkCommandBuffer 0xb400007345e64530[] is already in use and is not marked for simultaneous use. The Vulkan
                                                                                                    spec states: If any element of the pCommandBuffers member of any element of pSubmits was not recorded with the VK_COMMAND_BUFFER_USAGE_SIMULTANEOUS_USE_BIT, it
                                                                                                    must not be in the pending state (https://www.khronos.org/registry/vulkan/specs/1.3-extensions/html/vkspec.html#VUID-vkQueueSubmit-pCommandBuffers-00071)
                                                                                                  
                                                                                                    UnityEngine.Rendering.ScriptableRenderContext:Submit() (at \home\bokken\build\output\unity\unity\Runtime\Export\RenderPipeline\ScriptableRenderContext.cs:117)
                                                                                                    UnityEngine.Rendering.Universal.UniversalRenderPipeline:RenderSingleCamera(ScriptableRenderContext, CameraData&) (at
                                                                                                    .\Library\PackageCache\com.unity.render-pipelines.universal@16.0.6\Runtime\UniversalRenderPipeline.cs:798)
                                                                                                    UnityEngine.Rendering.Universal.UniversalRenderPipeline
2024-04-22 17:23:21.248 17480-17501 Unity                   com.dev.indirecttest2             D  Vulkan validation error./Runtime/GfxDevice/vulkan/VKDebug.cpp:144
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #00 0x729a76d6e4 0x729a76d6e4 (libunity.so) vk::smile:ebugUtilsMessengerKHRDefaultCallback(VkDebugUtilsMessageSeverityFlagBitsEXT, unsigned
                                                                                                    int, VkDebugUtilsMessengerCallbackDataEXT const*, void*) 0x20c
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #01 0x711c2fe130 0x711c2fe130 (libVkLayer_khronos_validation.so) ? 0x0
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #02 0x711ba71d50 0x711ba71d50 (libVkLayer_khronos_validation.so) ? 0x0
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #03 0x711bc909bc 0x711bc909bc (libVkLayer_khronos_validation.so) ? 0x0
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #04 0x711bc90ba0 0x711bc90ba0 (libVkLayer_khronos_validation.so) ? 0x0
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #05 0x711bd1a78c 0x711bd1a78c (libVkLayer_khronos_validation.so) ? 0x0
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #06 0x729a7b1c58 0x729a7b1c58 (libunity.so) vk::OnscreenSwapChain::AcquireNextImage(vk::FrameTracking&, unsigned long, unsigned
                                                                                                    int*, fixed_array<VkSemaphore_T*, 2ul>*) 0x74
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #07 0x729a7518b8 0x729a7518b8 (libunity.so) GfxDeviceVK::AdvanceImageInternal(vk::SwapChainBase&, bool&) 0x60
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #08 0x729a7b1a6c 0x729a7b1a6c (libunity.so) vk::OnscreenSwapChain::GetBackBufferImage(bool&, bool) 0x3c
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #09 0x729a7b12ec 0x729a7b12ec (libunity.so) vk::PresentationSurface::GetBackBufferImage(bool) 0x28
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #10 0x729a7529bc 0x729a7529bc (libunity.so) GfxDeviceVK::EnsureValidBackbuffer(vk::PresentationSurface*) 0x38
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #11 0x729a7790a8 0x729a7790a8 (libunity.so) vk::RenderPassSwitcher::InternalApply(vk::CommandBuffer*) 0x94
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #12 0x729a747e68 0x729a747e68 (libunity.so) GfxDeviceVKBase::EnsureCurrentCommandBuffer(vk::CommandBuffer::Type, bool) 0x1ec
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #13 0x729a74277c 0x729a74277c (libunity.so) GfxDeviceVKBase::SetShadersThreadable(GpuProgram**, GpuProgramParameters const**, unsigned char
                                                                                                    const* const*) 0x40
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #14 0x7299b3ee7c 0x7299b3ee7c (libunity.so) ShaderLab::ShaderState::ApplyShaderState(unsigned int, ShaderPropertySheet const*, ShaderPassContext
                                                                                                    const&, keywords::LocalKeywordState const&, Shader const*, ShaderLab::Pass const*, DynamicBranchState*, ShaderLab::SubPrograms*, DeviceRenderStateBlock
                                                                                                    const*, int, int, ShaderLab::Pass const*, ShaderLab::Pass const*) const 0x39c
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #15 0x7299b3ea7c 0x7299b3ea7c (libunity.so) ShaderLab::Pass::ApplyPass(unsigned int, ShaderPropertySheet
                                                                                                    const*, DynamicBranchState*, ShaderPassContext&, keywords::LocalKeywordState const&, Shader const*, int, int, ShaderLab::GrabPasses
                                                                                                    const*, ShaderLab::SubPrograms*, DeviceRenderStateBlock const*, ShaderLab::Pass const*, ShaderLab::Pass const*) 0xf0
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #16 0x729995faa8 0x729995faa8
                                                                                                    (libunity.so) RenderingCommandBuffer::ExecuteCommandBufferWithState(ShaderPassContext&, RenderNodeQueue&, RenderingCommandBufferState*, unsigned
                                                                                                    int, ComputeQueueType) const 0x24b4
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #17 0x729995cedc 0x729995cedc (libunity.so) RenderingCommandBuffer::ExecuteCommandBuffer(ShaderPassContext&, RenderNodeQueue&, unsigned
                                                                                                    int, dynamic_array<std::__ndk1::pair<ShaderLab::FastPropertyName, RenderTexture*>, 0ul>*, ComputeQueueType, RenderTexture*) const 0x50
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #18 0x7299976324 0x7299976324 (libunity.so) ScriptableRenderContext::ExecuteScriptableRenderLoop() 0xa3c
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #19 0x7292eef4d8 0x7292eef4d8 (libil2cpp.so) ScriptableRenderContext_Submit_Internal_mAE1E99BDF4C4E5A0FD9A0147E460E25ABB84BF14 0x40
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #20 0x7292ef24e0 0x7292ef24e0 (libil2cpp.so) ScriptableRenderContext_Submit_mF417223097ECEB2768EE736BE00A7F94DA3BDD26 0x138
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #21 0x7292ad9b40 0x7292ad9b40 (libil2cpp.so) UniversalRenderPipeline_RenderSingleCamera_m4886F9EF5EEF5C6A85E28E8A0D5E8CCC7809B49C 0x363c
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #22 0x7292acf8a8 0x7292acf8a8 (libil2cpp.so) UniversalRenderPipeline_RenderCameraStack_m47BBC0B4111D83BB6EE3275C9572BFBF2F5451B9 0x3fbc
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #23 0x7292ac8f08 0x7292ac8f08 (libil2cpp.so) UniversalRenderPipeline_Render_m15A42AB44C14AB4DCA7EF0B915964D46B643D50E 0x1140
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #24 0x7292ac7db8 0x7292ac7db8
                                                                                                    (libil2cpp.so) VirtualActionInvoker2<ScriptableRenderContext_t5AB09B3602BEB456E0DC3D53926D3A3BDAF08E36, List_1_tD2FA3273746E404D72561E8324608D18B52B533E*>::Invoke(unsigned
                                                                                                    short, Il2CppObject*, ScriptableRenderContext_t5AB09B3602BEB456E0DC3D53926D3A3BDAF08E36, List_1_tD2FA3273746E404D72561E8324608D18B52B533E*) 0x50
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #25 0x7292edd804 0x7292edd804 (libil2cpp.so) RenderPipeline_InternalRender_mCB77395158F4572F348D8227BA9127ABF1C9C5BE 0x324
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #26 0x7292ee5144 0x7292ee5144 (libil2cpp.so) RenderPipelineManager_DoRenderLoop_Internal_mB646C8738F4A9859101F3BE94809E2E10BBDB1FB 0x60c
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #27 0x7290e87c78 0x7290e87c78 (libil2cpp.so) RuntimeInvoker_FalseVoid_t4861ACF8F4594C3437BB48B6E56783494B843915_RuntimeObject_IntPtr_t_RuntimeObject(void
                                                                                                    (*)(), MethodInfo const*, void*, void**, void*) 0x44
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #28 0x7293e7a810 0x7293e7a810 (libil2cpp.so) il2cpp::vm::Runtime::InvokeWithThrow(MethodInfo const*, void*, void**) 0x58
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #29 0x7293e7a75c 0x7293e7a75c (libil2cpp.so) il2cpp::vm::Runtime::Invoke(MethodInfo const*, void*, void**, Il2CppException**) 0xb0
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #30 0x7293d2fd68 0x7293d2fd68 (libil2cpp.so) il2cpp_runtime_invoke 0x2c
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #31 0x7299dd41a0 0x7299dd41a0
                                                                                                    (libunity.so) scripting_method_invoke(ScriptingMethodPtr, ScriptingObjectPtr, ScriptingArguments&, ScriptingExceptionPtr*, bool) 0xb0
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #32 0x7299de87fc 0x7299de87fc (libunity.so) ScriptingInvocation::Invoke(ScriptingExceptionPtr*, bool) 0x88
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #33 0x7299efbe60 0x7299efbe60
                                                                                                    (libunity.so) Scripting::UnityEngine::Rendering::RenderPipelineManagerProxy::smile:oRenderLoop_Internal(ScriptingObjectPtr, void*, Scripting::UnityEngine::ObjectProxy, ScriptingExceptionPtr*) 0x78
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #34 0x7299976ae4 0x7299976ae4 (libunity.so) ScriptableRenderContext::ExtractAndExecuteRenderPipeline(dynamic_array<Camera*, 0ul> const&, void (*)(SceneNode
                                                                                                    const*, AABB const*, IndexList&, SceneCullingParameters const*), void*, ScriptingObjectPtr) 0x130
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #35 0x729a57a174 0x729a57a174 (libunity.so) RenderManager::RenderCamerasWithScriptableRenderLoop(int) 0x298
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #36 0x729a57a6e8 0x729a57a6e8 (libunity.so) RenderManager::RenderCameras(int, void (*)(), void (*)()) 0x38
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #37 0x7299a2c714 0x7299a2c714 (libunity.so) PlayerRender(bool) 0x2cc
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #38 0x7299a2d85c 0x7299a2d85c (libunity.so) InitPlayerLoopCallbacks()::PostLateUpdateFinishFrameRenderingRegistrator::Forward() 0x44
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #39 0x7299a1c3fc 0x7299a1c3fc (libunity.so) ExecutePlayerLoop(NativePlayerLoopSystem*) 0x84
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #40 0x7299a1c43c 0x7299a1c43c (libunity.so) ExecutePlayerLoop(NativePlayerLoopSystem*) 0xc4
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #41 0x7299a1c730 0x7299a1c730 (libunity.so) PlayerLoop() 0x144
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #42 0x7299ec2040 0x7299ec2040 (libunity.so) UnityPlayerLoop() 0x3d0
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #43 0x7299ec3350 0x7299ec3350 (libunity.so) nativeRender(_JNIEnv*, _jobject*) 0x54
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #44 0x7333551e30 0x7333551e30 (libart.so) art_quick_generic_jni_trampoline 0x90
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             D   #45 0x58c00d04 0x58c00d04 (memfd:jit-cache (deleted)) ? 0x0
2024-04-22 17:23:21.251 17480-17501 Unity                   com.dev.indirecttest2             E  VULKAN: VALIDATION ERROR: Validation Error: [ VUID-vkAcquireNextImageKHR-semaphore-01779 ] Object 0: handle = 0x5a1e20000000248, type
                                                                                                    = VK_OBJECT_TYPE_SEMAPHORE; | MessageID = 0x5717e75b | vkAcquireNextImageKHR():  Semaphore must not have any pending operations. The Vulkan spec states: If
                                                                                                    semaphore is not VK_NULL_HANDLE it must not have any uncompleted signal or wait operations pending
                                                                                                    (https://www.khronos.org/registry/vulkan/specs/1.3-extensions/html/vkspec.html#VUID-vkAcquireNextImageKHR-semaphore-01779)
                                                                                                      Objects:
                                                                                                        #0: { type = VK_OBJECT_TYPE_SEMAPHORE, handle = 0x5a1e20000000248 }
                                                                                                  
                                                                                                    UnityEngine.Rendering.ScriptableRenderContext:Submit() (at \home\bokken\build\output\unity\unity\Runtime\Export\RenderPipeline\ScriptableRenderContext.cs:117)
                                                                                                    UnityEngine.Rendering.Universal.UniversalRenderPipeline:RenderSingleCamera(ScriptableRenderContext, CameraData&) (at
                                                                                                    .\Library\PackageCache\com.unity.render-pipelines.universal@16.0.6\Runtime\UniversalRenderPipeline.cs:798)
                                                                                                    UnityEngine.Rendering.Universal.UniversalRen
2024-04-22 17:23:21.253 17480-17501 Unity                   com.dev.indirecttest2             D  Vulkan validation error./Runtime/GfxDevice/vulkan/VKDebug.cpp:144

Vulkan-API-calls failing validation include vkResetCommandPool(), vkResetFences(), vkBeginCommandBuffer(), vkQueueSubmit(), vkAcquireNextImageKHR(). There doesn’t seem to be an initial error message, the rendering just freezes and I’ll get spammed with these messages until I close the app. I also tried disabling the compute shader again (via debugger), but the errors keep on coming. Again, this doesn’t happen every time I start the App, in fact most of the time, everything runs fine, and the compute shader does what it’s supposed to do. But “most of the time” isn’t good enough. Of course I tried looking for similar cases, but no luck.

Finally, here’s some information on the compute shaders themselves:

The first compute shader runs without problems, a direct dispatch call with a single workgroup, to calculate the workgroups for the second shader:

//this cbuffer is filled via CopyCount (from an appendBuffer)
cbuffer AppendInstanceCount
{
    uint _AppendInstanceCount;
}

cbuffer OtherStuff
{
    uint _MaxBoneCount;
}

#define BONECALC_GROUPSIZE 64

RWStructuredBuffer<uint> _boneArgsBuffer; //indirect args for the second compute shader
[numthreads(64,1,1)]
void SetWorkGroupCount(uint3 id : SV_DispatchThreadID)
{
    if (id.x > 0)
        return;
    _boneArgsBuffer[0] = ceil((float)(_AppendInstanceCount * _MaxBoneCount) / BONECALC_GROUPSIZE);
    _boneArgsBuffer[1] = 1;
    _boneArgsBuffer[2] = 1;
}

The second compute shader is an indirect dispatch call using the indirect args buffer filled by the previous compute shader. It computes bone-matrices used for skinning.

float4x4 TRSScale(float3 t, float4 r, float3 s)
{
    return float4x4(
        (1.0 - 2.0 * (r.y * r.y + r.z * r.z)) * s.x,
        (r.x * r.y - r.z * r.w) * s.y * 2.0,
        (r.x * r.z + r.y * r.w) * s.z * 2.0,
        t.x,
        (r.x * r.y + r.z * r.w) * s.x * 2.0,
        (1.0f - 2.0f * (r.x * r.x + r.z * r.z)) * s.y,
        (r.y * r.z - r.x * r.w) * s.z * 2.0,
        t.y,
        (r.x * r.z - r.y * r.w) * s.x * 2.0,
        (r.y * r.z + r.x * r.w) * s.y * 2.0,
        (1.0 - 2.0 * (r.x * r.x + r.y * r.y)) * s.z,
        t.z,
        0.0,
        0.0f,
        0.0,
        1.0
    );
}

cbuffer AppendInstanceCount
{
    uint _AppendInstanceCount;
}

cbuffer OtherStuff
{
    uint _MaxBoneCount;
}

StructuredBuffer<BoneBlendingData> _BlendingData; //Length: InstanceCount
StructuredBuffer<float4x4> _Instances; //Length: InstanceCount
StructuredBuffer<BoneData> _Bones; //Length: TotalFrameCount of all clips * their BoneCount
RWStructuredBuffer<float4x4> _BoneMatrices; //Length: InstanceCount * _MaxBoneCount
StructuredBuffer<float4x4> _Bp; //Length: total BoneCount of all meshes
StructuredBuffer<uint> _InstanceIds; //Length: InstanceCount

groupshared float4x4 _instancesLocal[10];
groupshared BoneBlendingData _blendingDataLocal[10];
groupshared uint _baseInstanceOffset;

[numthreads(BONECALC_GROUPSIZE,1,1)]
void CalcBones(uint3 id : SV_DispatchThreadID, uint3 gid : SV_GroupThreadID) //id.x = boneIndex
{
    uint baseInstanceId = id.x / _MaxBoneCount;
    if (gid.x == 0)
    {
        _baseInstanceOffset = baseInstanceId;
    }
    GroupMemoryBarrierWithGroupSync();

    uint instanceId = _InstanceIds[baseInstanceId];
    uint localId = baseInstanceId - _baseInstanceOffset;
   
    if (gid.x == 0)
    {
        _instancesLocal[0] = _Instances[instanceId];
        _blendingDataLocal[0] = _BlendingData[instanceId];
    }
    else if (id.x % _MaxBoneCount == 0)
    {
        _instancesLocal[localId] = _Instances[instanceId];
        _blendingDataLocal[localId] = _BlendingData[instanceId];
    }
    GroupMemoryBarrierWithGroupSync();
   
    uint i = id.x - baseInstanceId * _MaxBoneCount;
    BoneBlendingData blend = _blendingDataLocal[localId];

    if(baseInstanceId >= _AppendInstanceCount)
    {
        return;
    }
   
    bool transition = blend.blendFinal > 0;
   
    BoneData boneA1 = _Bones[blend.boneOffsetA1 + i];
    BoneData boneA2 = _Bones[blend.boneOffsetA2 + i];
   
    float3 positionA = lerp(boneA1.Position, boneA2.Position, blend.blendA);
    float4 rotationA = lerp(boneA1.Rotation, boneA2.Rotation, blend.blendA);
    float scaleA = lerp(boneA1.Scale, boneA2.Scale, blend.blendA);
   
    float3 position;
    float4 rotation;
    float scale;
   
    if (transition)
    {
        BoneData boneB1 = _Bones[blend.boneOffsetB1 + i];
        BoneData boneB2 = _Bones[blend.boneOffsetB2 + i];
        float3 positionB = lerp(boneB1.Position, boneB2.Position, blend.blendB);
        float4 rotationB = lerp(boneB1.Rotation, boneB2.Rotation, blend.blendB);
        float scaleB = lerp(boneB1.Scale, boneB2.Scale, blend.blendB);
        position = lerp(positionA, positionB, blend.blendFinal);
        rotation = lerp(rotationA, rotationB, blend.blendFinal);
        scale = lerp(scaleA, scaleB, blend.blendFinal);
    }
    else
    {
        position = positionA;
        rotation = rotationA;
        scale = scaleA;
    }
   
    float4x4 mw = _instancesLocal[localId];
    _BoneMatrices[instanceId * _MaxBoneCount + i] = mul(mw, mul(TRSScale(position, rotation, scale*100), _Bp[i + blend.bindPoseOffset]));
}

If you read this far, any input would be greatly appreciated, since I’m nearing my wits end here. Thanks in advance

PS: I’m testing on a Pixel 8 Pro, and built with several unity version (2022 lts, 2023.2), tried changing the vulkan-specific settings in the project settings, always the same errors. The project runs on URP, but that shouldn’t matter as I’m only using the Graphics-API. Also, the errors (and frozen rendering) happen even if I don’t execute the indirect rendering calls, where they are normally used.

2

If it happens in the latest lts and unity 6 beta file a bug report at least

3

Your other options are:

  1. Can’t you use GLES 3.2 instead? I think GLES 3.2 can do compute. I don’t know if gles 3.2 support is good with Unity, but it’s worth rolling the dice.
  2. Don’t do mobile stuff.
  3. Do a bug report, but if it doesn’t happen consistently and within 10 minutes I wouldn’t have high hopes of it going through the first phase of QA. And then even if it does, you need to wait at least a year for something to materialize, and then there’s a good chance that what materializes is “Won’t Fix”.
  4. Try Unity 6 in case they have fixed anything Vulkan related (although I don’t think they have).
  5. Don’t use Unity.
  6. Get enough money to get a source license so you can fix the issue yourself?

I’m hoping someone from Unity stumbles upon this and can tell you something useful… but I wouldn’t hold my breath.

4

Thanks for the input. I can reproduce the issue well before 10 minutes since you can just restart the app right away if it doesn’t happen, so I guess I’ll make a reproduction project tomorrow and file a report. As for the other suggestions…

  • I haven’t tried Unity 6 yet since it’s in beta, but a quick ctrl+f shows tons of “vulkan” hits in the release notes so I’ll check it out.
  • gles 3.2 works in theory, and has been more stable than vulkan in past projects. However, unlike vulkan, structured buffer bindings are very limited for both compute and vertex shaders. Some gpus (mali, if course) don’t support them in vertex shaders at all. With the amount of instances I want to render, I’d need to use so many constant buffers and dispatches… I’d really rather not.
  • As for not using mobile, or Unity, or getting source code access… I’ll ask my project manager but I don’t think he’ll be happy :smile:
1 Like
5

Maybe there is a UAV barrier missing between the two dispatch calls. Not sure if that’s something you have control over in Unity, though. I think, Unity is supposed to add those automatically.

You need a UAV barrier whenever you write something in a compute shader that you read in another compute or graphics shader - especially when the draw calls are right after one another. It makes sure that the draw calls don’t overlap and that all the caches are flushed.

I’m not super familiar with Vulkan or mobile development but if you have a way to take a GPU capture, you could check if there is a VkBufferMemoryBarrier between the dispatch calls.

6

Pre

Actually this sounds a lot like an issue we have…

7

You can see tech streams (2023.2 etc) as beta as well tbh. Most fixes land in beta first and are not always backported to tech streams.
So I’d say use the latest lts version or the latest beta, but not everyone might agree

8

I mean… you’re not wrong. The idea behind using the tech stream during development was that there will be an lts for it before we have to release. Anyway…

  • I’ve tried Unity 6, and it’s still the exact same issue, sadly

  • I’ve tried isolating the compute shaders in a reproduction project, but of course, now I can’t get it to (not) work. Seems like I’m still missing something. I’ll go double check the rest of the rendering code again

  • I’ve tried looking at the vk-commands using Android GPU Inspector, works fine when the issue doesn’t show, but if I try capturing a frame once the errors start, the profiler breaks with an out-of-bounds exception. Is it theoretically possible, that another, previously running compute shader accesses memory of the suspected CS above out of bounds, going unnoticed until it’s accessed again (correctly)? I have some more debugging to do

Thanks a ton for all the help so far.

9

You don’t have to have a broken frame to check whether the VkBufferMemoryBarrier command is there. It will always be there or never. So just take a capture after starting the game while everything still works.

I don’t think so. You can’t make out of bounds reads or writes in shaders. Those reads/writes be will simply ignored. You can have access violations, but it’s not so easy to get one, actually. They are usually caused by bugs on the CPU side like, for example, passing wrong GPU handles/pointers in descriptors.

If I am right (and there is a good chance that I am not since it is a long shot), the _boneArgsBuffer contents would be garbage due the missing barrier and the second dispatch call would run with an extremely big group count causing a TDR.

10

I have a similar issue on Mali GPU’s (pixel 6), except it’s a intermittent flashing, it doesn’t happen on the adreno.

That said I do have an error log from logcat…

“Compute Dispatch: missing input compute buffer ID xxxx”
Also
“Compute Dispatch: missing UAV ID xxxx (incompatible ComputeBuffer)”

9855135--1419195--Screenshot 2024-05-25 173043.png
9855135--1419195--Screenshot 2024-05-25 173043.png1267×517 105 KB

I assume this is what Unity means when they complain about compute shader support on mobile devices being spotty.

This is in unity 2022.3.20

11

Also btw, Unity technically doesn’t allow you to fix anything with the source license. You just get to “look at it”. It’s against their TOS to actually build with it… I am sure that doesn’t stop people, but it’s just typical Unity crap.

I would also love an update on this bug, as it’s really flaring up now it seems with our users.

12

I can give you an update but it’s not a great one, meaning I have not solved the issue. I’ve tried to isolate it in a reproduction project, but the only thing I learned from that is that out of bound writes or reads can indeed be a reason for these issues, like multiplying a buffer index by 10 makes it easy to reproduce. This does not seem to be the only cause though, as I’ve literally clamped the access of every single buffer (including the indirect args buffer) by now and the issue still persists.

I’ve checked the api calls in a graphics debugger as well, and yes, there are several VkPipelineBarrier calls in there which should prevent premature buffer access.

I suspect it has something to do with immediately using a buffer after creation, as I’ve noticed that

  1. The “crash” happens either on the same frame or the frame after I (re)create the buffers used in my compute shaders, or never
  2. Binding a buffer right after creation sometimes doesn’t work correctly, and I have to bind it again before executing the compute shader later in the frame

One thing I tried was putting all instructions into a command buffer and execute it slightly later in the frame, as well as waiting for its completion by polling a graphics fence before issuing more commands. This seems to alleviate the issues somewhat (though not completely) but also isn’t a great solution since it introduces a ton of latency.

That’s it for now, if I ever find the root cause I will of course post it here. In the meantime I’m thankful for more input as well.

13

If you’ve isolated the problem you should ABSOLUTELY do a bug report, isolating issues is what resolves most vague issues like this.

14

If you think out of bounds buffer access should not be causing these issues, yeah I can submit a bug report. However, from what I’ve been reading, accessing buffers out of bounds leads to undefined behavior, and may lead to losing the device. Vulkan even has an extension to safeguard against this called “robustBufferAccess”, which I’ve tried to enable on my Pixel 8 by writing a native plugin for it, but unfortunately it does not support this extension. So I thought, even though it leads to the same issue, out of bounds buffer access probably is not the root cause of my problems that I’m actually trying to isolate.

As for your problems, I think I’ve seen the same errors when trying to execute a compute shader with missing buffer bindings. If you aren’t doing so already, try binding your buffers (again) directly before executing the shader.

15

We were facing the same exact issue, occasionally on some specific devices (mostly with Mali-GPUs) we got the same GPU crash, 2-3 last frames repeating infinitely whereas the rest of the game was running normal on the background (input was still being detected + we were able to do everything as normally).

-We dealt with it for almost two weeks until we actually started to be suspicious on the GPU, and voilà here is this thread explaining everything in detail (kudos!) and having the same issue. This led us to dive deep into our compute shader(s) but mostly one that we pinpointed, and after some research on the Unity Docs and forums, we figured out that the Buffer’s amount IS THE REAL ISSUE.

-So as explained on the Unity Docs here specifically at the “Platform-specific differences”, under Android (OpenGL ES 3.1) we have:

  • "

  • OpenGL ES 3.1 (for (Android, iOS, tvOS platforms) only guarantees support for 4 compute buffers at a time. Actual implementations typically support more, but in general if developing for OpenGL ES, you should consider grouping related data in structs rather than having each data item in its own buffer.

"

and as also in your case, we surpassed the compute buffers amount by 1, so in total 5, and this led to the GPU crashing randomly or whenever it got overflowed.

-Resolution, make sure you combine/group related data in a struct and pass it as a single buffer in order to avoid surpassing this guaranteed threshold of 4 compute buffers at a time.

TLDR: Watch out for the compute buffers amount that you’re using inside your compute shader, since surpassing the minimum guaranteed support threshold is not viable and can lead to GPU crashes in those devices that don’t have the guts to process more buffers at a time.

P.S. This was our issue and it’s resolution (tested & confirmed) but would love to hear if this is also the same reason for you too (since most of the factors are there).

P.P.S. Also an interesting read is THIS thread where joshuacwilde was actually explaining these limitations.

16

In case it helps, for us the issue was related to Graphics Jobs (Experimental) enabled in project settings. This causes the issue on some android devices. We decide to fallback to GLES on problematic devices.

1 Like
17

It seems there’s probably several things that can cause this to happen.
But I’d just like to reassert that out of bounds buffer writes can cause this to happen.
I made a simple test compute shader and just told it to randomly write to like ID * 50 * rng0-1 in some buffer.
It’s not guaranteed to happen every time, as in one out of bounds write isn’t likely to cause a problem, and anything within like the overflow room when (numthreads * threadgroups) doesn’t line up with the size of the buffer, probably wont cause problems either.
But when there is a setup that causes the freeze, it does always happen. As in, if I re-run the app on my phone with a consistent rng setup that makes it freeze, it will always freeze.
No idea what’s happening behind the scenes when you try to do the buffer write that causes the issue though.

Anyway, hopefully that’s helpful to someone.
Preferably the GPU would just be more robust as others don’t even have this issue, and if I were a GPU I’d be feeling pretty embarrassed and inferior right about now.

18

Hi again, it’s been a while. I was busy with other parts of the project and haven’t been able to look at the CS crashes for some time. Then this week, I looked into it again when working on some adjacent stuff, and I’m happy to report that everything is resolved. The culprit? Out of bounds buffer access. Of course. Not in the shaders posted above (they have been almost completely rewritten by now anyway), but in another, seemingly small and innocent looking one that was used to simply copy over data between two buffers, where one of the buffers was not following the assumed size standards I thought I had set up for everything.

Why does accessing a buffer out of bounds cause the gpu to go up in flames (even bricked my device a few times)? The reason seems to be, that Mali GPUs do not have “robust buffer access” (I confirmed this by writing a native unity plugin where I tried to manually enable this vulkan extension), and you simply access memory belonging to some other buffer. If that buffer is in a pending state (or maybe the access causes it to enter the pending state when it’s not supposed to be? Who knows), vulkan complains and loses the graphics device, resulting in the errors I posted in my OP.

As for the four buffer binding limit per compute kernel, I believe this restriction only exists for GLES. If you’re building for Vulkan only, you can have more bindings than that. But check the compiler for warnings to be sure.

So if you have the same problems as me:

  • Select the compute shader asset and resolve all warnings listed there (also press “show compiled code” once to force compilation, just to be sure)
  • Make absolutely sure you do not have any out of bound accesses anywhere. It might not be obvious at first, since many platforms handle them silently in a way where there are no obvious signs of them occurring, until you test it on a platform that doesn’t and everything goes up in flames.
    • This also means, unless all your buffers align perfectly with your group size, you must do boundary checks in your kernels, e.g. if(id.x >= _BufferSize) return;, or place your code in an if-block if you need group syncs, since you are not allowed an early out in this case.

To help with preventing this in the future, I added some boundary tests to all my compute shaders:

#ifdef DEBUG
#define CHECK_BOUNDS(i, max) if(i < 0 || i >= max) InterlockedAdd(_Debug[0],1);
#define CHECK_VALUES(i, min, max) if(i < min || i >= max) InterlockedAdd(_Debug[1],1);
#else
#define CHECK_BOUNDS(i, max)
#define CHECK_VALUES(i, min, max)
#endif

RWStructuredBuffer<uint> _Debug;

I put this in an include file and use the macros every time a buffer is accessed, or I want to check a specific value for outliers. Of course, the _Debug buffer is only used and bound from code when debugging symbols are set, so it will have no impact on runtime performance. I use a simple vert/frag shader to display a small rect in the corner of my screen, which lights up when the debug buffer contains data. Hope this helps, or if you know of a better way to detect out of bound access from within a compute shader, feel free to post.