Unfortunately there’s no way to control how vertex color gets applied for SpriteLit / SpriteUnlit within Shader Graph. What it does is apply that color (the related node is Vertex Color node) but unfortunately it applies it automatically after your Shader Graph runs, whether or not you want it.
See here: Interact with SpriteRenderer color in ShaderGraph
Consider adding your use case to the discussion so the developers are aware that people want more control over this.