DELETE ME
If you need to ask these sort of questions, you are way in over your head.
With your level of skill, you should be looking at some pre-made architecture like uLink and PikkoServer. They handle this all for you.
Writing your own networking/MMO/socket system is no trivial task.
The question is…do you want to build an MMO, or do you want to spend many months building a basic networking library?
Lol, I was going to respond with this earlier.
Meltdown is correct, why reinvent the wheel? If you are planning a simple MMOFPS, photon might just be perfect for you.
HOWEVER if you really want to make your own server just combine them into 1 or 2 exe. like have the login as its own exe and then the game server itself as its own that handles party, chat and whatever else. Try to see if theres coding to some game servers like bukkit for Minecraft and use their source code as a referrence / example. Thats what I am currently doing to make my own java based server. BTW Hi from Ontario 
DELETE ME
If you can’t even afford the license Cost for Photon then your certainly not going to have the money to rent servers. You won’t be able to host a couple million players on a Single Physical Server you will need a whole Server Cluster with some unknown amount of Servers working together. I say unknown amount of Servers because that will largely be dependent on how much your Server Software needs to deal with.
oh would all you downers quit being douchbags please. He can get a cheap vps from amazon or other providers just to help get him going with live testings. Also ever heard of private servers noob? Minecraft doesnt host all the pc version servers, Notch has a download for the server jar and Dinnerbone is a contributor to the bukkit server. So this guy doesnt really have to worry about hosting massive players on his own (rented) hardware if he decides not to. MMO does mean Massive Multiplayer Online, doesnt mean hosted by just 1 company / person DUH!
The servers are the least of his worries.
He wants to write his own MMO architecture that can handle ‘millions of players’.
Firstly… nobody on these forums has the knowledge on how to write architecture like that and give you advice on it. If there is anyone on these forums that worked on the WOW/GW2 or the PikkoServer architecture, then feel free to chime in and correct me that I am wrong and help the OP with his question.
Secondly, if you’re worried about Photon/uLink licensing costs… then you certainly can’t afford to take the 6-12 months to write a simple MMO architecture that will be nowhere even near as polished as the Photon/uLink solutions that are available. These solutions have had many man-years of development by people vastly more experienced than the OP in network programming.
This thread is pretty much pointless. The OP wants advice on something that seems well beyond his reach, both financially and skill-wise.
Yes your right he COULD allow Players to host their own Servers… HOWEVER depending on what he’s exactly trying to achieve he has some issues he would have to solve. Once you allow the Server software to leave your hands you can no longer Trust anything that is going on on the different Servers. The Server can and WILL be hacked… If people Can hack the Servers then they could find Loop holes to also hack the Official Servers there fore any type of Cash Shop or even Subscription Service would no longer be able to work because people could hack the servers to bypass any subscription requirements or to give them selves whatever items from the cash shop they wanted to. He would just be setting him self up for a hard time were he to allow players to host the Servers. Also Servers hosted by other players would have to low System Requirements so that players could actually afford to host them. There for the Game would have to be designed around smaller matches/games with 50 to 100 players at most maybe as high as 200 on a Server. This would rule out normal MMO architectures where you host different Zones/Regions on separate Servers. Games like DayZ or Online FPS games would work great in this context but normal MMOs wouldn’t work in this environment.
untwist your panties please. you arent getting what we are saying, so i am going to lump you into the uneducated crowd. calling us douchebags for giving him good advice is pretty stupid on your part. neither of you seem to understand the complexity involved in writting a good networking backend.
you say you want to support up to a mil players, do you understand distributed computing / load balancing? do you know what fault tollerance is? do you understand that you cannot have only one cluster of servers in one place, and why? do you understand how tcp / udp works? do you know how to prevent cheating? can you explain interpolation or extrapolation? can you program c++?
these are just basic questions on networking. why in the world would you chose a system that will be subpar and buggy compared to real solutions out there that PROFESSIONALS have designed is beyond me. the licencing for these already built systems are not expensive at all. as meltdown has said before, if that is too much for you, you will never finish anything.
think about it like this, would you rather be building the game, or trying to figure out networking crap years down the road? if you intend to run this as a business, better drop the ego, as it will cost you a LOT more in the long run if you take this route
also, just to note: meltdown has finished projects and has published them. it would behoove you to listen to his advice.
i would also suggest checking out Badumna, pretty damn cool system, and might fit your needs. the arbitration server is a cool concept. it will allow the granularity you seek, while still covering the stuff we are advising you not to pursue.
Badumna is a pretty cool Networking Engine… Still a little too new and risky for my taste though for a MMO. I know they have done alot with security and I know you can run importing calculations on the Arbitration Servers but still just the idea of a P2P MMO still seems a little too risky for me personally.
i had the same thoughts, lol. i have actually been playing with this. i have a friend that loves to cheat in games. i gave him my code and tell him to break it. he has a few times, but that is because i didnt do the arbitration correctly. i found the trick is looking for expected values based on past values.
This was an article written by a developer for Guild Wars 2:
http://www.gamedev.net/blog/355/entry-2250155-why-you-shouldnt-be-making-an-mmo/
Whilst I don’t intend to belittle OP and say that their project is out of reach, I think they do need to open their eyes to just how big an undertaking an MMO engine designed to handle “10 million players” is along with what sort of skills you need to get this not only working but working efficiently (as the article notes, a server designed for a few thousand people is nothing compared to an MMO engine designed for millions of concurrent users).
I hope that what we are saying isn’t coming across as belittling. That was not my intention. As for the link, bingo, that sums it up very nicely! Now, I am a big believer in people being able to build an MMO by themselves. It has been done before, and it will be done again. But I also know that the ones that did it already have a good foundation to build from.
If you missed it,
There is the Pomelo NodeJS MMO server with Unity Client and Multiverse Java MMO server with Unity Client. Both are MIT licensed. Download the source code and study them 
Good luck
Since when does Multiverse actually have a publicly released Unity Client? I know some people were working on it but I haven’t seen anything released to the public yet.
I find it challenging designing code for an 8 player twitch gaming experience and I’ve been programming for around 30 years or so*. It’s a lot for me to take in and absorb and learn and bring to an acceptable standard. By acceptable standard I mean of comparable quality to what you would expect in a console game.
Bearing that in mind, I think you will have a lot of trouble making an MMO, as while it is simple on paper, there are all kinds of problems you’ll have to deal with on a financial and technical level so I’d advise you try a 2 player game then migrate to a 4 player game. If you can get that working perfectly, you probably stand a chance trying 16 players.
In short you’re going at top speed before you can crawl.
- This is just an indicator of experience, not my technical ability, I’m just speaking from a point of view where I think I’ve seen enough to know you’re leading yourself down the wrong path.
Hey hesa!
As a big data network developer and someone who is actively involved in developing an MMO, I sure can tell you it’s no easy undertaking, but I certainly would never say don’t do it. Yes there are many challenges, many of which you will never forsee, but what you learn from them is worth it.
Judging from your questions it seems this is coming more from a hobbyist/ why not give this a try point of view; in which case, good on you for going for an MMO! Here’s a few pointers that can possibly help you out.
First and formost, there’s two bits of an MMO that get overlooked all the time.
- Massively. The definition really isn’t kidding.
One thing that I can immediately point out is that each of your listed servers would have to be a cluster if you really do intend on getting players in the 6+ digit range. Even if your program is extremely streamlined, a server is limited in the amount of connections it can maintain by ports, so to handle lots of them at the same time you really must have multiple servers working together.
So you’re probably wondering; how do I make a cluster? Well, theres a bunch of different approaches. One of the most basic is whats known as round-robin DNS. Rather than having a DNS address such as server.example.com pointing to a single IP, it points to lots of them, and DNS will automatically shuffle them around each time someone wants to know an IP to use. The client then selects one from the set of IPs and you achieve a cheap method of load balancing. You’ll see round-robin DNS is employed by most large companies - an nslookup for google.com for example returns a set of IPs. Repeating the same nslookup will show them changing order. Neato.
What about the server end? This is where the hard part comes in. Your servers must then of course communicate with each other, working collectively as one. This can be done by each server knowing all other online servers in their local cluster (and informing all other servers when it starts up or goes down), and a simple technique from that point is just broadcast all messages to all the other servers; Ideally though if you can target a particular server with a message (e.g. you know for sure that this private message from player A on server 2 is to player B on server 4), do. Alternatively you can take the slave/master approach where there is one server in the cluster that tells all the others what to do; the technique you use can vary from cluster to cluster, depending on what the task is that it must perform. A great real life example of a master server is Tractor by Pixar.
- Online. It’ll be your duty to keep it so, 24/7.
It’s heartbreaking to see servers go offline. In gaming, it’s pretty fatal. This is where your monitoring and update systems should come in; In the biggest of systems, these are clusters too. You really have to consider how the game will get updated without causing too much of an interruption to your players, and without causing too much of a traffic issue at your server. Update servers are notorious for receiving massive volumes of requests, and then nothing for ages (expect it to be essentially DDOSED each time you update) - this is something that you must work with and even out.
Another note on staying online is you must attempt to remove any single points of failure in your network; thats any single servers which, if offline for any reason, would bring your whole game offline with it. This is almost always a database of some kind, so you must consider redundancy. Expect the harddrive to fail so your network doesn’t; Imagine what would happen if your database failed completely and you lost it all. At this scale, the wheels of a database routinely fall off anyway, so you must also employ techniques such as sharding to again spread the load from one single point to a cluster of servers. Dns can be excellently useful here too for keeping track of parts of your database. Locating your whole operation in a single datacenter should be avoided if possible, as if the datacenter goes down, you do too. Multiple datacenters each staying in sync with each other is exceptionally complex due to delays over the internet, but should certainly be taken into account if you wish to grow.
Security
Often overlooked, so please don’t underestimate it as it will form one of the largest components of the game. If somebody breaks past your games security and goes rampage gaining themselves a ton of advantages over everybody else, it has a significant impact on the overall mood of your players.
I could go on for hours about this stuff, so I’ll try summarising some extra points down below!
Other categories of server you may not have considered yet:
- Indexers. How do you find stuff, such as searching on a forum? When massive is involved, you can’t simply use in-database stuff anymore.
- Website forum database (if you really mean huge, this will get overloaded if its not seperated).
- Content servers. You have to structure your game in such a way so you can update it with minimal interruption, so splitting up your assets is vital.
- Billing server. The server you’ll probably love the most
Handles transactions, micropayments, recurring billing etc. - Security services. Servers which track and authorize progression through your network.
- Trade/Market servers. Can players trade in your game? If so, you’ll want to know how your games economy is fairing as a games economy can generate some incredibly insightful metrics, especially if it requires you stepping in and giving it a nudge every so often.
- User account control such as password recovery services, global emailing etc.
- Version control.
- Backup systems.
Other
- NPC and gameworld state syncronization. Everybody needs to see stuff in the same place.
- Support, Support, Support! Players hate awful support, so plan ahead for it. Consider various methods of communicating with your players and pick the ones you think will suit your game. Guides, forums, FAQ, tickets; all of which should be searchable.
- Tools and administration services. There will be times when you urgently need to interact with your servers. You’ll also want to provide simplistic methods of for example performing updates/ adding new content/ changing a players details manually. All of this should be provided to your staff so you’re not constantly tripping over each other or making mistakes, as a human error may just knock your network offline.
So, there’s a small summary of some things to consider! Making an MMO, or any large network for that matter, is a complex procedure filled with potential pitfalls but is well worth it once you’re done. Good luck and I hope this helps!
All the best,
Luke Briggs
Kulestar, UK
Thanks for the info Luke, I’d like to get your opinion on the following…
I’ll be launching my game soon, and hoping to get it involved with all sorts of marketing spikes and player usage highs and lows.
Can you give any useful tips on the best way for a small indie like me to scale infrastructure up and down as neccessary?
For instance if I have a single server that can handle 300 concurrent race sessions… but then I’m say in a Humble Bundle deal, and there is an influx of a few thousand players in the next week, what’s the best way to manage this? Are virtual servers the way to go?
I do plan on keeping a seperate forum server, and a seperate billing/database server too.
As an indie I need to keep costs down so any other advise regarding this sort of thing will be most welcome.
And security wise, my server will be purely authoritative, the only input the server will accept from players is keyboard input such as throttle (normalised from 0 - 1), and steering input (normalised too). All physics will be calculated by the server. Aside from this, are there any other security considerations I should think about?