So as you can see in the image I would like to detect if there is a higher box than the y position of the player and if so make a slight jump to prevent collision malfunctions (I do not need the jump code as well, only the detection one) , I was thinking of using RayCast to solve the problem however I am not sure how to do this. I would really appreciate if the code is in C#, thank you in advance!
You could do a SphereCast with the appropriate radius.
I am new but what I would do is make a second game object attached to this one (or just moving with it) and make its collider trigger, make collider in front of the character so if there is something out of the ground for him to jump. If you want code just search for oncollisionenter tutorials.
If you just need your Rigidbody to climb stairs or go up small steps and stuff, you might want to consider using a Physics Material on your player’s collider. Then lower the “Dynamic Friction” property until you get the desired behavior. Also set the “Friction Combine” property to minimum.