Concerns about performance issues with infinite sized game

I am starting to make a top-down procedural generated game. I am aware of the precision problems with very large floating point numbers, and have seen the ‘world moves, you dont’ solution. My concern is that you will end up with a lot of objects being moved (all the world game objects) instead of just one(the player) and i dont know how easily the engine will be able to cope with that? Has anyone implemented this type of system?

A side note, could you get this system to work for with unity physics?

Realistically though, Infinite Runners require Object Pooling.