Unreasonably high GPU and CPU usage in VR

I see extremely high usage, even though the number of DrawCalls is only 442 and the verts are only 400k. This is completely unreasonable on such a powerful machine (I7-7700K, 1080Ti). The situation is the same when running from an editor and build.

Unity version: 2019.4.8f1, and all drivers are up to date.

Does anyone have an idea what I’m wrong with?

Ohh, and the profiler:
Only 320 draw call, almast 10ms time. I think it’s a problem with that taking so long.