Edge case code like this is always finicky. You basically have to play with things like friction or collider radius or other details until you get the feel you want.
If you want a basic full-featured controller, here is a super-basic starter prototype FPS based on Character Controller (BasicFPCC):
That one has run, walk, jump, slide, crouch… it’s crazy-nutty!!