Stop Character Controller from falling down

Hello, My third person game has a simple ledge detect climb system. When it detects a ledge, it grabs using hand IK. My problem is the character controller attached to my Player pulls it down. Any way to snap it to the point where ledge is detected and not affected by gravity. (P.S Rigidbody is not an option because I use CharacterController.Move() for pretty much everything.) Thanks in advance.

Have you tried just disabling gravity on the rigidbody when on the ladder then re-enabling it when off the ladder?

@ jiak1
Yes. It just keeps moving up. Doesn’t stay where the ledge is.