Rigidbody changes position when I add torque. How do I stop this?

The script on my Rigidbody allows the user to change its position and rotate it (through AddForce and AddTorque, respectively). However, when I rotate it, its position changes which has a notable effect on the game. How do I make sure that the Rigidbody rotates around its center without changing its position at all? I thought about freezing the position, but that would bring the object’s velocity to zero, correct?

Upon closer inspection it seems that when I add torque in the x direction is when it moves. Adding a torque in the z direction does not. When the x torque is added, it seems as though the rigidbody rotates around a point outside of itself. I thought that adding torque made it rotate around the object’s position. Am I missing something?

This might help answer your question:

You might need to specify the center of mass.