Does this happen when you have a wall with a slight slope? As if the character is able to get a grip from the very slight friction? If so, you might try using a slippier physic material for the character’s collider.
I’m having the same problem as well. Using different collision material types doesn’t change it. It seems like it might have something to do with the collision making ground = true when you are next to a wall. You can just hold the spacebar and the rigidbody just moves upwards.