It appears you are already doing it fundamentally correctly.
As far as controlling the size of the text, this is usually done either by a content size fitter control, or by manually limiting how large or small it can get. Or you can specifically leave it at a particular font size that works for all distances/positions. You can also hide the text part when it gets too far away or too close.
Without knowing more about your context (is it a 3D world canvas? Is it floating over the screen as 2D? etc.) it’s hard to say much more.
If you want just one number or a little more text, you may use TextMesh embedded into 3d character. It will just move with it as any other objects in hierarchy do and that’s it.