Then the answer is probably no. It’s unfortunate that some of the early Unity tutorials make use of the physics engine, as it gives newbies the impression that you always need it, when in many games you do not… and using it when you don’t really need to definitely makes your life harder.
For what you’re trying to do, a better approach is probably to use a navigation mesh.
Agreed with Joe, no physics needed here for the avatar itself, also that glitch is actually pretty hilarious to see the avatar all squashed up like that above while still tracking haha. Either navigation mesh or make sure the avatar follows the position of the rest of the rig - as the head camera and armature obviously move down the stairs.