Ok. Solved it. There are two approaches you can take here!
One is you can accept fate and move on. If you feed the original Alpha of your Main Texture’s Sample Texture 2D node to the Fragment Alpha then the artifacts are gone. (A user on twitter told me about this. Thanks again!) I did need to do some fancy stuff beforehand that so keep reading for a better solution.
My other solution is a bit convoluted but it works. Here we go:
Using this method you can clear the alpha completely and do other things to your texture!