I am the developer of a networking solution that has yet to be released. With the Server API completed and the Unity3D Client API currently in testing I’ve decided that I should create a game that uses my networking to help show off it’s capabilities. There’s only one limitation, I’m only a programmer and I don’t have a team.
Currently my networking solution has been tested to hold 2,000 concurrent connections without falter on a household machine, while this may not be a ridiculous statistic for most people, it’s more than enough to handle numerous different types of games. The software is configured to be scalable with hardware as-well, so pushing it beyond 2,000 connections shouldn’t be a problem, not to mention that 2,000 wasn’t the cap, however just the number I had decided to stability test with.
I keep focusing on numerous different tasks, and thinking of ideas for what would get people interested. I’ll probably release the project as open-source when it’s completed. The following options keep smashing through my head:
Create a “MMORPG”. - While this term is subjective, it can easily be accomplished. Things that make a MMORPG a MMORPG are far more than the basic game mechanics, but a cookie-cutter MMORPG without any lore or meaning is more than enough to prove the networking can hold up.
An open-world FPS - This is pretty much what is sounds like, take something like Call of duty for instance and put everyone on the same server, instead of multiple instances of a smaller server.
A MoBA - This one would be fairly simple to produce as-well.
There’s so many different popular game-types, but it’s hard to choose what you want to do when you’re trying to advertise a product. The more popular networking solutions such as SmartFoxServer have huge names behind them. So how should I bring myself into the market? While I already have numerous marketing tactics, including a cheap and affordable networking solution, I want to develop something that proves to my clients that it works. So what do you guys think?
The FPS game is only going to be semi-authoritative. I don’t have enough money to even think about justifying a fully authoritative version of this, the hardware required would be insane. The amount of messages being sent from the client/server aren’t the real problem in this case, but the vector calculation for every single bullet would turn your standard computer into a stove-top, I wouldn’t expect it to last over 250 CCU (And that’s with some pressure) on a low-end machine.
A large-scale FPS like this would require zone-based networking, which ultimately would require multiple server instances, and multiple instances of the client connected to a single server, per user. While there may be numerous users playing the game at the same time that can communicate with each-other, be in a “guild”, or party up, they would not all be in the same networking zone. This would mean that Zone A would have it’s own server to handle bullet calculations, player movement, etc, and Zone B would also have it’s own.
To combine this together we would have multiple servers, for things such as login, friend-server, chat-server, guild-server, etc.
This is the standard setup of most large-scale games anyway.
As for the back-end, it’s a modified version of the Netty networking library. I took the language that I was most comfortable with about a year ago and checked all of the networking solutions that were currently prospering. Netty usually contained the highest benchmarks, not to mention that it’s a NIO networking library which should always be used in this situation. I then checked out the white-paper on the SmartFoxServer benchmarks which showed Netty as their top competitor. While BitSwarm was always ahead of Netty, Netty was definitely surviving. I decided to use Netty and continue to make modifications to things such as encoding/decoding, as-well as to how the data was handle upon receipt. I rewrote fractions of code that netted fairly beneficial gains. Fixed quite a few resource leaks and removed a lot of overhead. While there’s still some there, it’s more than efficient enough to do what I’ve wanted it to do and considering my networking solution is hooked up to an update server, if I ever push a large-update through to it, all of my clients would be notified of the update.
These numbers are much more reasonable. I know it because i have done it ;). Droppde all middleware and wrote our network stack straight on top of UDP, minimal data copy after data is received from socket, aligned all data to match CPU cache lines, using lock-free data handling and so on. Because of that when presenting numbers about CCU, it’s good to add what kind of parameters make up this number ex. 2k ccu (client rate x, server rate y, bandwith z, allowed server side game logic time in ms per tick).
Well, as with any scenario it depends on how you’re optimizing the data as-well. If you have a double value of 16.0 and you send it over the network as a double, you’re effectively wasting 48 bits of data, however if you simply cast it over to an short, you’re effectively saving 48 bits of data. Small calculations like these while knowing the boundaries of the primitive data-types can save you a massive amount of data/second/client.
Also, when you’re dealing with an open-world shooter, you have to be a little more reasonable with how you handle data, don’t you think? All bandwidth aside, that’s enough math to make a bit-coin miner pause. I don’t really use UDP networking for much, while my networking solution does have an UDP implementation, I haven’t used it at all for any of my projects. I’ve never had to. While sending updates such as movement may not be as necessary as sending something like an attack command, I’ve always sent it all over the TCP stack, and it’s always done as it was supposed to.
The networking solution I made already uses a UUID to bind a UDP packet to a TCP Stack, or rather I should say the account that’s tied to the TCP Stack so it can update information on the server. The UDP and TCP listeners are actually all ran on the same application, this may or may not be changed.
You keep talking about a “Tick” and I’m almost confused as to in which way you’re meaning this, there are certain games which process a tick at a certain interval(Ex: Runescape - Once per 600ms), and some that limit ticks per second (Ex: Minecraft).
With my solution I don’t even think about “Ticks” considering it’s all Non-blocking IO. Instead, it’s more or less, “How fast can this hardware process the information given to it”. When dealing with a NIO networking solution, it’s almost impossible to decide what a server-tick is.
It should be something basic enough for new people to learn from but extendable enough for veterans to build upon. I recommend maybe a mini version of tabletop simulator where everyone connects to a scene where they can drag pieces around and have physics for those pieces apply across the network.
If you would prefer to stick to one of your original ideas, the shooter sounds like it would be an impressive demonstration. All of them sound a little complicated for someone to learn from, however
The art of my API is to simplify this to the point that anybody who spends a few hours will be able to write a multiplayer game of any type, granted the logic for each gameplay type is different, our goal is not to help players understand their gameplay logic, and instead to allow them to easily transfer it over a network.
For instance, if a player has already followed a tutorial to make a ghetto single-player game, he should be able to create a networked version of that game very easily using our networking solution.
I have agreed to work on an online TCG project with someone in the collaboration section, which will use my networking solution. This will show the capabilities of turn-based gameplay. (Although, one of the most basic kinds of gameplay)
Yes, this kind of basic data compression is obvious.
We built upon UDP because our tests showed that TCP get’s alot of buffering not only clint side, but alos routers, and if one message is lost all TCP stream stalls. This caused a lot of network jitter and more unpredictable results. We also supported ACK over UDP, more lightweight and faster than TCP (also lightweight congestion control). More latency and jitter means unresponsive and inconsistent gameplay most important in FPS games.
I try to explain what i mean with “tick”. In large scale thare are 2 kinds of server ticking, one for networking and one for server side gameplay (gameserver). To be efficient you have to buffer incoming messages for some time, then pass this data to actual gameserver running on top of your networking solution (what part does actually buffer it, networking side or game side does not matter, because this should be configurable).
Game tick is actual frequency game simulation is running on server, running simulation after every message, kills your performance instantly (buffer your data, run parallel SIMD optimized code etc.). Also gameserver have to take into account player latency and compensate for it. Ok, this is going to drift away from actual networking side of things :).
Network tick - how long you are buffering outgoing messages and grouping these messages per. client. This could run @ same rate as gameserver or slower. Also networking layer must support instant messaging, if needed.
NIO is way slower then BIO, all this context switching overhead is going to add up, fast.
Have you tested how much data and messages your solution can handle per second (from client to server side game logic and back to client)?
What i wanted to say is that, networking / server solution able to run MMO or FPS game with 2k CCU on household computer are bold statements and not easy to achieve.
Keep up the good work!
I guess my only question is… does it run offline? With power and a router but no internet, can people use your networking solution to play LAN without any server?
I’m interested in your single player → networked game claim. That sounds fantastic!
Thanks for breaking it down, I was a little confused on how you were using the word tick. As far as your question regarding messages/second I had tested using a 2,000 user simulation, sending a randomized about of messages per second, ranging from 10-15. Six of these messages were position updates, containing the x,y,z and rotation of the player. The others were just randomly generated data, for things such as spammed chat messages, friend requests, attacking, etc. While this wasn’t a legitement test of using 2,000 REAL users I did the best I could to simulate 2,000 active users playing on the game, walking around and updating a position. I was able to run this with no real effect to my machine, as-well as still play the game. I connected to the game over the internet, instead of a local connection and I noticed small Jitters throughout the testing, usually 3-7ms.
Most of the data was being handled locally at that point, as I don’t have the funding to get a 2,000 user test running. I only left the server running for about an hour to see if it would crash, but it stayed strong. This was tested on a Quad-core 3.6GHz AMD Processor. Physics calculations were disabled for this test, as all values were randomly generated.
As for the statement you made “and if one message is lost all TCP stream stalls”
–
I noticed this back in my early developments and I tried to find a way around this, most of this is due to TCP Segmentation. To prevent this what I have done is created a special buffer that handles data based on the packet. If the start of the packet is found again before the end of the packet then it will throw away the old packet and continue to read on. This is also the case for newer packets. If a packet with the ID of {1} was sent through to the server, and part of packet {1} was lost before the end of {1} was received, it will do a size-check on the buffer, and throw the data away if required. You can’t do much to restore lost data, besides for telling the client you didn’t get it.
On the flip side, if you receive the start for packet {1} and then immediately the start for packet {2}, all of the data for each packet has it’s own identifier, so the data won’t get mixed together.
Where on the internet though? My understanding is that a lot of the issues with TCP based networking are to do with internet infrastructure, if all your virtual clients and your server are hitting the same switch (for example) then your test is not very indicative of real performance over the net.
PS If you do set-up to run a more realistic multi-user test I’d be happy to run a client for a few hours for you.
Responding to the OP - clearly the most impressive display would be a openworld FPS with 2,000 concurrent sessions. This is something AAA games running on beefy servers have not managed to accomplish so would immediately get your solution noticed.
I’m little confused about this, if you are using TCP then TCP by itself handles all this kind of resend / duplication and lost data situations, why would you need to implement similar logic on top of TCP is confusing.
UDP in game networking have clear advantage over TCP because, when one “not important - ie. position/rotation update” package was lost, we just don’t care and keep handling next package. In TCP, stream is halted until lost package is received.
I would say that UDP is better for MMO games as most lost messages can be forgotten, and both client and server will continue with the simulation until new information arrives and the client simulation can be updated. Delayed messages can mostly be discarded, and important messages can be marked as such. A framework like Lidgren does most of this work for you. From what I understand about TCP, and it’s overhead - it just won’t work in an MMO/FPS scenario.
Ticks are a must - rather than sending many tiny messages with their overhead, it’s better for the server to build up a stack of messages for each client, and send the whole bundle once per tick.
Yay someone knowledgeable about network stuff! I have a question!
Can LAN games handle more network traffic? Do they handle network traffic better? I’m saying strictly LAN, like an adhoc network or router that is not connected to the internet.
I’m still trying to figure out why people ever in their life want to use a Local Area Network for a game? Back in the 80’s that was the thing, we’re in the new day in age, why would you simply want your game to be local when it can be world wide? That’s fun, but back in the 80’s of course LAN was fun because that was the new thing, or 90’s whatever it was.
If you’re gonna program a LAN game wouldn’t it be just as simple to use Photon Network and make it world wide? Even if it’s for just testing, at least your buddy can test it 300 miles away instead of in the same house/building.
I’m one of those die hard anti DRM people who want their games to be playable until the end of time. I’ll include both online and local play if a game gets popular enough, but I would never burden my cherished sheep with any kind of restrictions. I care about the customer experience first and my finances second. I also don’t plan on it being my primary source of income, but to avoid going off topic you can PM me for details on my ridiculous pricing plans.
Anyway, would you happen to know if local area networks with traffic going to 1 router or between computers can handle significantly more traffic than an online game?
Obviously!
For world-wide internet game, you have to optimize your low end potential player, who let’s hope do not have 56k modem :), but, with every network node packet latency is increased and overall player experience decreased. That’s why competitive multiplayer games are run in LAN and server tick rates are increased to sky high, it’s still possible only in LAN within controlled environment.
Big difference is fading away as 100Mbit home internet connections are getting more and more common, and network nodes faster and faster.