GetKeyDown fires twice when KeyCode is Alpha number

I have the following piece of code in FixedUpdate()

if (Input.GetKeyDown(KeyCode.E) || Input.GetKeyDown(KeyCode.Alpha1))
{
    Debug.Log(string.Format("Down"));
}

When I press E, it only fires once no matter what, and if I press Num#1 it sometimes fires once and sometimes twice.
And what made it more weird, when I take this piece of code off and puts it in Update(), it no longer fires twice.

The script is attached to a simple cube object I created by 3D Objects → Cube

Use FixedUpdate only when dealing with physics:

FixedUpdate should be used instead of Update when dealing with Rigidbody. For example when adding a force to a rigidbody, you have to apply the force every fixed frame inside FixedUpdate instead of every frame inside Update.

If you really need to verify if a key is being pressed, set a boolean to true (when the key is pressed) and false (when the key is released) in Update.