When two character controllers hit each other, one of them teleports away

Both the player and the enemy have character controllers, and the player is moving using the CharacterController.Move method. If i disable the character controller from the enemy, everything is fine but as soon as i enable it, when the player gets close to the enemy they zoom past them as if they got infinite move speed for a few frames.

Any fixes?

Video in the reddit link:

The player is moving using a state in a state machine where this is called:

private void HandleMove()
		cameraRelativeMovement = _ctx.ConvertToCameraSpace(_ctx.CurrentMovement);
		var movementSpeed = (_ctx.IsCrouching) ? _ctx.CrouchedMovementSpeed : _ctx.MovementSpeed;
		_ctx.CharacterController.Move(cameraRelativeMovement * movementSpeed * Time.deltaTime);

ConvertToCameraSpace just converts the input from horizontal and vertical to match the isometric style, and then i just use CharacterController.Move, and same thing with the enemies and NPCs.

This HandleMove is called in the Update of the PlayerMoveState