Scriptable Render Feature to generate colorized outlines

I wrote a Scriptable Render Feature for generating outlines based on screen-space color, depth and normal thresholds to achieve a certain stylized look for my game.

This works great, but I’m running into the issue that these outlines are obscuring team-colors for different characters (vehicles) in the game, especially at large distances everything just turns into black blobs. I wanted to expand this feature to have colored outlines, red for red team objects, blue for blue team and black for everything else.

I was thinking to achieve this by rendering a sort of additional unlit opaque pass to a temporary RTHandle, this would give me the color information I need to get the right outlines, but I have no idea how to do that in the framework of a ScriptableRenderPass. Is this possible to do that or do I need to look for another approach?