Preventing a Child object (eg. some weapon) from going inside other objects in the scene?

I have Cube with a sword like object as its child object. But when the cube moves in the scene the sword goes into the other objects in the scene. The sword have a BoxCollider attached to it but the scene contains objects that are stationary and don’t have a Rigidbody attached as I don’t want them to be controlled by physics, same for the sword. If I attach Rigidbody to it then it will be displaced from its main position on collision.

I tried to render only the Cube including the sword with one camera having higher depth value then the main camera in the scene. If my game was in first person then this works fine but In 3rd person when object get behind other objects in the scene we can still see the object.

How can I prevent the sword from going into other object without being controlled by physics?

Image:

I think the simplest solution is to have the capsule or box collider on the main player extend out further so that the sword won’t ever go into a wall. You could also still try to have a rigidbody on the sword, but just set it’s position and rotation to be frozen? Not sure if that will work, but it might.