I recently noticed that if one wants to freeze axis position of a spinning rigidbody object like a spinner that is hit continuosly with other rigidbody objects, using Constraints > Freeze Position X, Y and Z doesn’t work at all. Objects that are constantly hit over and over again start eventually move bit by bit for some reason, even if their position should be fixed using Freeze Position Constraint option.
In order to avoid this one MUST NOT use any Freeze Position options, but build mesh colliders that physically prevent objects from sliding off, like the green structure in the image below.

In order to keep the spinner in the image sliding off position on every axis, one must also add colliders to the end of the axis, but of course all these structures can be set editor so that all mesh colliders are not visible.
Note that if one uses Freeze position and these kind of mesh colliders at the same time, the same object slowly drifting issue occurs again. Rotating object somehow gets stuck inside the collider and keeps drifting while rotating still at the same time.
So the only working solution is NOT TO USE CONSTRAINTS on POSITION at all on Rigidbodies. I did test this with dropping objects repeatedly on the wheel making it turn from the force of dropping objects, and one could see the movement after 50 hits.
Some forums indicated that a higher DRAG value would help to prevent this, in my tests I did use the numeric max value one can set, and it still didn’t prevent the axis drifting slowly off position.