Collisions and speed problems regarding a sphere and a controller

Hello !

I'm new to Unity so maybe I'm doing something wrong but I don't understand what's happening in my scene.

I created a plane with a single sphere (with Rigidbody) and used the FPS Player Controller Prefab. I'd like to move around freely and move the sphere if I'm touching it (like I'm pushing it).

So I put a Box Collider to my FPS Player Controller but then, I don't understand anything. My character becomes very slow (checking-unchecking the box collider while playing changes his speed) and I don't understand how I make the sphere move. Sometimes it works without problems, other time I need to force a "collision bug" to make it move and sometimes, the sphere doesn't move at all. I don't understand the logic behind this, it seems random. Also, with box collider checked, I can jump without stopping if I maintain pressed the space button. I'd like to know why my speed changes when I put the Box Collider and why is it so random to move my sphere.

Am I doing something really wrong or is it the Unity who's got a problem (I haven't tried to unistall and reinstall yet) or my PC who's doing something.

Thanks in advance, I can add more details if needed. Here's some pics if I can help. I tried messing with nearly all of them at least once but maybe I did wrong.
1943546--125647--Screen1.jpg 1943546--125648--Screen2.jpg

Here's a quick way to do this:

Add a empty gameobject (or cube) with a box collider and rigidbody. Make it a child of the camera and add tick all the constraints.

1945827--125801--Howtosetup.png

Hope this helps :) If you need any more getting-started help, PM me :3