TcpListener with Network Manager

I have a game that uses the Unity Network system that uses their server to set up the game connections. This works fine.

I have a bunch of specific data that I need to transfer to each client when they connect. It is formated in such a way that using the RCP system would be both wasteful and cumbersome because the only data type you can send that would be even close is the Vector3. This would work, but as I said be very wasteful and cumbersome to pack my data into a bunch of Vector3s.

So I have created a TcpListener on the server and a TcpClient on the client. This works great except that you have to know the IP address and it has to be on the same network or be a public IP address.

I am guessing that my users are going to be behind NAT in most cases. The Unity network servers get around that, but as far as I can tell only for the game session that is managed by the Network plumbing.

I can get the IP address that is used for the NAT stuff and it looks like it is on the Unity servers somewhere with a special PORT.

My question is whether there is any way that I can set up a connection via the Unity server that I can use the IP address and my own assigned port that I can attach my TcpListener to that will accept incoming TcpClient requests?

After much research I have not found a way to make this happen. The Unity networking stuff seems to not allow direct TCP connections.