Does your PlayerController manipulate the y component of the Rigidbody velocity? If so, fix that.
If you’re not sure, start the game, jump, then disable the PlayerController. Does it fall more reasonably?
Also, check the gravity setting. It’s obviously very tightly coupled to scale. Unity Physics expects 1 world unit equal to 1 meter
Also, make sure you’re not bypassing physics.
With Physics (or Physics2D), never manipulate the Transform directly. If you manipulate the Transform directly, you are bypassing the physics system and you can reasonably expect glitching and missed collisions and other physics mayhem.
Always use the .MovePosition() and .MoveRotation() methods on the Rigidbody (or Rigidbody2D) instance in order to move or rotate things. Doing this keeps the physics system informed about what is going on.