Character controller is broken at lower framerate

I have a huge problem with character controller component - with lower framerate it lets the character’s feet slip more into the slope when I try to climb it up which breaks my game.

I can ofc post my code if it would help but I think it’s caused by the character controller itself.

FixedUpdate seems to partially fix it but then it’s jittery and when I adjust the time step it sometimes works as before. So I think this isn’t really caused by not using FixedUpdate.

I would be really grateful if someone finds a solution or suggest what could be causing it. I haven’t even found much information about how CharacterController detects collisions etc.