Headtracking and camera collision

I got a character moved with the keyboard and a first person camera moved with a headtracker. The camera can go a small distance forward,backward, lean to the sides, up and down to simulate a character bending.
Now how would I prevent the camera to go through other objects? I tried using raycasting to keep the camera from going too near any other object, but then the center position of the camera gets misplaced.
The camera uses a MouseLook script.

Maybe you could just assume the characters collider is big enough to contain the extents for which the camera bobs with the tracker? That is, limit the movement so it doesn’t exceed the limits of the collider of your character/camera.