unity 5.1 networking low fps in exe only

Hello I have a game I set up with the new networking components unity 5.1 offers.
Everything works great as long as I run it in the editor I get about 60fps as the host and client. The problem is that when I build it the exe gets about 5fps weather or not its connected.

This was not the case before I added the networking features.
I do have some rather expensive objects that have network identities just because I needed to be able to find them with gameobject.find.
I could fix this if that could be causing the problem.

The strangest thing is that the lag is local. It can’t render the proper amount of updates all of a sudden.
I do have a camera for each player with network identities as well.
I’m a little lost so I was wonder if anyone could shine some light on that.

bump bump bump