You’ve given us almost zero information… things generally don’t “begin floating off” unless something makes them do so, such as the physics system.
In any case, if you’re relying on floating point numbers for absolute perfection… don’t.
Perhaps you’re seeing this?
Floating (float) point imprecision:
Never test floating point (float) quantities for equality / inequality. Here’s why:
https://starmanta.gitbooks.io/unitytipsredux/content/floating-point.html
“Think of [floating point] as JPEG of numbers.” - orionsyndrome on the Unity3D Forums