Level bytes

There might not be an an exact answer to this but how many bytes would a scene have to be to start lagging?

Lagging has barely anything to do with how big your scene file is. It has more to do with how big your textures and meshes are (note that these are not stored in the scene file) and how you have coded your scene (note that there are many many possible ways your code can cause lag, and size of code is not an indicator either).

It is also true for programming in general that whether a program lags does not correlate to the size of the program. A good idea when programming is to not worry about lag until you see it. Make sure to test for it though on the slowest platform you wish to support before you release! Ideally test on all platforms.

If you are seeing lag then the Unity profiler window may help you deduce the cause.