Okay, found the source:
I was using a custom subclass of UniversalRenderPipelineAsset to be able to replace the grass shader.
Doing that prevents Unity from properly stripping shader variants somehow:
See this thread: 2019.2: Overriding shaders of Terrain-Grass page-3#post-9835866