Is there a way to have 0 reflection in reflection probe area?

I’m building a tile based 2D game that uses the standard shader for the tiles (using mesh not sprite renderer). I want to use normal and metal maps for the tiles, the latter to make certain parts of the tiles appear like metal. So my understanding of the metal map is, that as long as I have the color black (metalness=0) and alpha channel set to 0 at certain areas, there happens no reflection at all.
My problem now is, that when I just define a small area of my tileset to be metal (red=1.0, alpha=1.0) and leave the rest alone the black areas still have some reflection or at least these areas get lit up as soon as I place a reflection probe there. The metal part works fine, however, everything else within the reflection probe range is lighten up. How can I solve this? Reflection intensity in general lighting is set to 0.

Any help is appreciated, many thanks!

After further testing it looks like this is a problem that only arises when you use the metal workflow of the Standard Shader. It seems like there is always a minimum level of reflection as soon as you use this Shader. Fortunately, I was able to create the same material properties with the Standard (Specular) shader without the falloff from the reflection probe