Typical bandwidth usage on an "MMOG"

Unity Engine
1

In my MOG each player sends 350 bytes per second while moving. So my questions are:

  1. How many bytes per second is sent by each player on a typical MMORPG? I know it varies from game to game, but generalize. 2) How many bytes per second is sent by each player on an MOFPS such as…Halo, etc.

I need to have a target to aim for in bytes per second so my game will run well as I expand it without using too much bandwidth. Even if each player on my game sent 1300 bytes per second that is 0.01 megabits per second and my connection is 3 megabits per second, doesn’t that mean I could have over 100 players on my server without any problems because I’m only using 1/3rd of my connection speed? That is hard to believe for me so I doubt that is correct. Thanks.

2

the connection speed will always depends on the users connection so there is not really a way to tell about numbers on how many can a person send, i’m using a 50mb connetion so i’m sending out more than that, at the same time it depends on the network solution wheter if you are using unity networking and or photon and the kinds of messages you are sending.

i’m using smartfox2x and my messages a re more different because they are being sent into a binary form besides compression if they are bigger than a set range from the settings.

3

I’m not a networking jedi by any means. But if you are sending 350 bytes per second, that leads me to believe you are sending character position and rotation 15 times a second (360 bytes a second). x,y,z and rotation for each axis = 6 floats, 24 bytes.

The big deal is…how many people does the server have to send that information to at the same time, if you are controlling this?

If you have 100 people in a zone, and the server needs to update all of them with each other’s data, this means (potentially) each person is sending out there data. 100 players talking to 100 players = 10,000 packets.

So for the server to send your character’s position to the other 99 (let’s say 100) people, that is 35,000bytes (3.5k) per second…just to send out YOUR information.
Now, it has do this for the other 100 people, to let each of them know about everyone else’s position right? That is 3,500,000 bytes per second (3.5 MegaBYTES, not megabits). 3.5 megabits is actually 437,500 BYTES per second. You are sending 3,500,000 bytes per second. Ouch. You would need a 30 megabit connection to handle that.

So you just blew your bandwidth totally out of the water. I think I’m doing this right…but maybe I need more coffee, somebody please correct me if I’m spouting nonsense :slight_smile:

Worse…if you are using tcp/ip…you have some overhead for each packet. I think it’s around 28 bytes or so. But’s pretend that doesn’t exist :slight_smile: The point is to trim this down as much as possible.

Ok, so I am just guessing at some of the details…like you are sending the x,y, and z rotations. But going forward with what I (think) I know about your data…
Most people don’t need the x and z axis rotations, they rotate around the Y axis. If someone needs to swim…just show a swimming animation, you don’t actually rotate the character. This can cut out 8 bytes right there.

For -my- situation, I don’t need a full float variable to hold my y rotation. I don’t even need a full 360 degrees of precision. With just using a byte (0-255), I can get 256 degrees of rotation. NOBODY will notice…at least in my situation, they might in yours. But if you can get by with that…that knocks off 3 more bytes. Also…the player itself -will- have a full 360 degrees, but the data sent over the wire will trim it down to 256. Again…nobody would probably ever notice or care if your character doesn’t have a full 360 degrees, because you are interpolating smooth movement between 256.

Just going with that, you would be sending 13 bytes instead of 24.
195 bytes a second instead of 360.
To all 100 players: 19,500/bytes/sec instead of 35,000.
For everyone to communicate: 1,950,000 bytes/sec instead of 3,500,000.
Sending out updates 10 times a second instead of 15 times a second would mean 1,300,000 bytes per second…still over your 3 megabit cap of 375,000 bytes per second :frowning:

But this is a worst case scenario (depending on your situation). How often is -everyone- moving around at the same time?
You could do other things. If someone is just standing there, not moving but looking around, you can only send their rotation…1 byte! that’s a huge savings. I think this happens a lot, especially in StormWind er whatever capital city you are in :smile:

Also…let’s say you split your world into zones, each 1km square, and your standard unit of measure is 1 meter.
Do you really need full floating point precision to track each player’s x,y, and z coordinates? Nah!

If your zone is 1km square…what if you used an unsigned 2 byte value (a ushort in C#) for each x,y, and z value? This would give you a resolution for each position of 0.015259 (1000 meters / 65536 possible values). In Unity, create a sphere and move it by increments of 0.015259. If you think that is enough precision to track someone…you chop your bandwidth in half for positions.

Again…that may not work for you. It may not be enough precision. The server would still have to track which zone they are in.But if you could do this, along with using a single byte for the Y rotation, you end up with this:

7 bytes for each update, 10 times a second for 100 people (but 100*100 = 10000 packets) = 700,000 bytes/sec (or 5.6 megabits).
At this point…there are other things you could do, like only send changes in position.

Let’s say the fastest something an move is 1 meter per second.

Divide that by 256 (1 byte) and you get this resolution: 0.003906. -That- is the precision you would get between moves, NOT BAD, but slow…3.6 km per hour. Walking speed is about 4km an hour, if I remember right. But the cool thing is…you can totally handle different movement types.

And…you can use different datapackets to handle different speeds:

For slow movement with more detail…send a “Low speed” datapacket. It’s for when the player is walking, we get high precision…which you would want with slow movement. 1m/sec = 1/256 = 0.003906 precision.

Medium speed datapacket - when the player is sprinting or falling. 5m/sec = 5/256 = 0.01931 meters of precision between position, 18km/hour. Not too bad I think.

High speed datapacket - you’re going at 10m/sec or 36km/hour = 10/256 = 0.0396063 meters of precision. We’re losing precision here…but this is probably going to be when you are on a mount. Do you -really- need a lot of precision when moving at high speed? Interpolation would totally hide that.

And if you want flying mounts…like a WoW taxi…even less. 30m/sec = 108 km/hour = 0.117188 meters of precision.

It depends on how fast players can move. But that means that probably 99% of your communication about player position drops down to this:

3 bytes for player position change, plus 1 byte for rotation. Heck, you could have packets that only sends position info when the rotation isn’t changed, but that’s up to you.
Anyway, let’s say 4 bytes.

send among 100 people moving all at the same time…that’s 10,000 packets…= 400,000 bytes/sec. Or 3.2 megabits.
And…you could make even more optimizations. If the player is facing away from someone, don’t send them a position update. I think this is why in some mmos, if you walk into a place and turn around REAL fast…it takes a while for people to show up.

If player B is a far distance from player A, send them updates less often and use interpolation to move the player between positions. Distant character movement is much less important than closeup. The downside is, you don’t want to bog the server down because it’s calculating distance and view angles.

Again…I am just rattling this stuff off my head. I’ve done some of this…but not all. If someone sees something I’m getting wrong, please post a correction.

Hopefully it’s of some use :slight_smile:

4

Very interesting, thanks. I can put about half of that to use, the other half I’ll have to learn more about.

5

We have cca 2Mbits/sec per each 150 players.

6

Even after trimming everything down as much as you can, you’re probably going to need to look into Movement Prediction, and variable update rates (i.e., updating less often based on how far a player is or where the character is looking).

7

Calculating message size (2 floats for position = 32bytes), 10 times per second, x 1000 players, means 1,250kb/sec incoming. Now work that out for any observing players and your outgoing is, on average 8 times that. And that’s just for movement.

Suddenly DSL doesn’t cut it even for testing.

8

yes yes, because the TCP header is typically 20 bytes… and do not forget retransmission of dropped packets, more brain pains for ya… you could try transmitting the change in position, rather than absolute position. and transfer it in a lower precision float, or maybe a fixed point number. but this could also introduce synchronization issues that may or may not be workable.

9

MMOs we send about 40 bytes per second for movement (when moving), 10 bytes (1 byte header, 3 bytes per component in fixed point form) 4 times per second, you can get away with less, depends on the game.

FPS you can get away with a lot more, the rule of thumb I have is to aim for 64kbit/sec. Roughly 256 byte packets, 30 times per-second. Mind you that has to include everything, not just movement. I use a priority updating scheme to work out which entities to squeeze into the 256 byte packet each time (when sending from the server to the clients, never had a problem fitting client data in to a 256 byte packet).

You don’t want to be using Unity’s built in networking (or middleware like photon IMO) for anything beyond prototyping, best to make your own networking protocol. Oh and use UDP for FPSes, TCP should be fine for an MMO.

10

I’ve been thinking of how useful some sort of batching or grouping mechanism might be. The idea is, it would batch or queue up certain packets of data for the player, and send them at regular intervals. Kind of like nagel’s, but handled on the server side.

Some types of packets (movement updates) would overwrite, only the latest values would be stored.
Others would actually queue up, like attack actions. You would want each one to be sent and performed on the client, unlike movement updates where only the latest would be included in the batch.
Every X milliseconds, it would send out the entire batch to the player.

This could cut down a lot on tcp/ip overhead, I think. Granted it would mean some more processing on the server…but maybe not too much if done right.

I’m not sure how that would also consider player distance, other than it would change the update interval depending on distance.

Different packets would have a property that dictated whether or not they overwrite previous ones in the batch.

On the server, for each client:
There is a dictionary of packet types. This stores the latest values for the packets that only store the latest information.
There would be a list/byte array/whatever that would store the other packets’ data.

When the server gets ready to send this information to the player(s), it would gather the byte data for the queued up packets, then add on the byte data for the “overwriting” packets. This entire byte data would be sent in one batch.

The more people you have around, the more it helps by removing tcp/ip overhead. I think the greatest traffic by far is movement updates. If you can squeeze this down into lower precision values (see discussion above) to 3 or 4 bytes per player, seems like it could save a lot of bandwidth.