Particle has 2 materials, overlay eachother

I have a particle system with 2 materials. A circle and a square. The problem is that when a particle spawns, it is assigned both materials so they stick to eachother and look wrong. Is it supposed to be like that or am I doing something wrong?

It kinda works like I want if I use 2 particle systems with different material each, but if I use 1 system with 2 materials, they just stick together.

If that's not supposed to happen, how do I "unstick" them?

You have to create two emitters if you want different textures.