Unity should be Open Source

News & General Discussion
1

Hi,

I use Unity since two years and release some games with it. The engine is easy to use and you can create games and you do not need all skills of a game developer.
But the problem is: The Unity developers starts new projects (like HDRP, URP, Network Gaming, Entity System, Job System, …) and do not finish theme! On top the old state of the art solutions will be deprecated and not supported anymore but the new solutions are very buggy and have a lot of technical problems.
So I believe there are too many projects for the Unity company and it should be better to make Unity completely Open Source to give the community a chance to make Unity one of the best Game Engines in the world.
Well, Unity is free for starters and all other licenses must check individual. So in this context it is not the point of interest that unity is a commercial product.
There are so much things I want to change via a Merge Request:

  • Package Manager Asset Store Solution

  • Play Scene with Base Scene

  • World System

  • Scene Global Components

  • Materials depend on target platforms

  • …

So I think a lot of users of Unity from the community would be made it better and let Unity grown up.

Maybe, dreams come true :).

Best regards!

2

The problem with Unity right now is that it is too much like an open-source project, with all sorts making an arrival via the package manager. I don’t see how actually making it open source would help anyone, quite the contrary.

3

Going open source will slow down things further.

Because hallmark of opensource approach is lack of general direction, starting many things and not finishing them.

4

Does Unreal move slower than Unity?

edit: I do not advocate Unity going open source. We dont have time to dig into their source anyway. But its still a interesting topic.

5

Unreal it’s not an has never been open source.
It is a propietary engine and if you tried to modify it and redistribute it, you would get a big fat lawsuit coming your way.

6

Unreal is proprietary. And not Open Source.

7

True, didn’t read the inital post good enough, most question like these wants equivalent to unreal were source code is available. But here they want more real open source like godot.

I’m a big advocate for open source, there are alot project that have very clear goals and great management.

Just look at.NET core, graphana etc

8

But then there are tons of them that have fallen by the wayside.

9

Thats also true. At my dayjob almost every SDK we use are open source so probably more success stories found in the corporate world

10

Not having (free or affordable) access to the C++ part of Unity source is, by far, Unity’s biggest pain in the ass for me compared to Unreal and the reason I will never willingly make any complex project using unity and strongly discourage people from doing so in every opportunity I get.

Whenever we get a hard crash, specially in built players, that originates from Unity code and not from our own code, it feels like having to disarm a bomb in complete darkness. We have to randomly poke and prod it, blindly trying to figure out which of our previous actions got Unity offended and what can we do to appease its anger. A process that can last hours, days or weeks, and could be greatly simplified if we could simply look at the crash site source while debugging.

The same happens when trying to optimize. There are some notoriously opaque systems in Unity, like the terrain, which do a lot of things in a very specific and undocumented way, and require serious trial and error to optimize.

I do get it: from the IPO financial reports we learned the bulk of Unity’s income comes from large clients who purchase source licenses. Also, making the source fully available would allow people to simply compile out license checks.

But could we at least get some of the C++ source available for reference and debugging purposes, like Unreal does for Launcher builds (which cannot be used to rebuild the engine)?

11

You know, having source code access does not exactly mean you’ll be able to solve the crash, though. Or that you’ll even have easier time resolving the problem. Something worth keeping in mind.

The reason for that is high complexity of game engines

12

Of course, not being able to actually re-compile does limit what can be done, but being able to see what the darn thing is doing to my inputs does help immensely. This patronizing “Unity knows best, you have no need to look behind that curtain” attitude is exactly the kind problem I’m talking about.

Let me give you real world examples:

  • We have been using SphereCastCommands with good results for a while, and decided to optimize the creation of the command lists and the processing of the results using jobs. Performance gains were significant, but we now get crashes on systems with 4 cores or less. The reason? Somewhere inside the SphereCastCommand batching system there is what we believe to be a fixed size buffer that limits the number of in-flight commands per job thread, which increased as a result of our optimization.

This is not documented anywhere and a quick glance at the source would cause us to promptly re-factor our system to respect this limitation. Instead, we spent two days running around doing CSI work to guess the cause of the problem.

  • We wrote a custom grass rendering system using DrawInstancedProceduralIndirect. Worked perfectly on PC, PS4 and Switch, but outright hard crashes on Xbox One. Crash site is inside the XB graphics libraries, callstack prior is full Unity code. This one took weeks of intermittent work to fix. There are other uses of procedural indirect drawing in the game, so it’s obviously supported. We tried all sorts of permutations on the shader, cutting it down to the bare minimum, to no avail.

I had to painstakingly read through the disassembly in Visual Studio, which fortunately retained function names and allowed me to place a breakpoint somewhere earlier in the crash call stack. Groking though the dissasembly, plus the few named variables that could actually be seen and keeping track of register and memory addresses, and comparing it with the execution of the other procedural drawing that didn’t crash, we figured out that the crash was caused by Unity telling Direct3D on Xbox to use the non-procedural draw path, which caused it to try to access the vertex input layout, which was null. We then just re-factored the procedural call into using a pre-generated mesh, since without the source it was impossible to find out why the draw call was not using the procedural path.

In both cases the time to figure out what was going on would be significantly shortened by being able to see the source code, even if we couldn’t be able to modify it. There’s no such thing as too much transparency. Might as well give up the reference C# source, the built-in shader sources, and the packages’ sources as you are at it. Those have made a heck of a difference in my workflow.

There are several parts of Unity which are absurdly opaque and do not have their implementation details documented, which would be much easier to work with if we could at least look at the source code.

13

And huge mess of a code base

14

This is an excellent example of why Unity will never be released as open source. Most if not all of it’s middle-ware is behind non-disclosure agreements as is all code related to console platforms.

I know what you’re going to say though. You’re going to mention that Unreal 4’s source code is available but the difference between the two engines is that Unreal 4’s middle-ware is implemented entirely through plugins while Unity’s middle-ware is built directly into the engine.

There are only two ways Unity’s source code would be able to be open source. One, Unity would need to refactor all of its middle-ware implemented with a plugin approach. Two, all of the middle-ware would have to become open source. Neither of these is likely to ever happen.

15

The platform support in UE4 are not plugins. I do have access to them all: they require you to unzip a folder full of source files on top of the engine source code and re-compile the whole thing. Meanwhile, all the NDA’d platform support in Unity are actual plugins. It’s actually a better implementation of multiple platform support: you cannot build for consoles on UE4 with the “launcher” version. In both engines you only get access to them after confirming your licensed developer status.

Also, for simple reference and debugging purposes, Unity doesn’t actually need to make NDA’d source available.

But you have a point in that Unity was way too happy to simply shop around for stuff like Umbra and Enlighteen (R.I.P.) until recently, and there might be stuff there that could require going through some red-tape even if their headers are not included.

16

Yes.

It is not “patronizing” or “Unity knows Best”, it is speaking from experience with Unreal engine where source code access is provided.

The main issue is that the engine is a Behemoth. It is a 300 megabyte body of program text. When you hit a crash, you do not always get a simple “oh, there’s a little bug here, you can fix it, if you change this parameter!”, but very often you get something that involves multiple threads, is spread through multiple components, and is for practical purposes, operating in ancient sumerian, using alien logic, and someone spent 10 years writing it.

Basically, while you can peek under the hood, below the hood, there will bee a swarm of writhing eldritch horrors and no documentation.

For example, off the top of my head, stuff I experienced:

If you create a landscape in currently open scene through C++ code, the editor will crash once it is done recompiling shaders with 75% certainty. The crash will lead to some sort of sanity check akin to “if shader.dirty == 0”, which won’t help you pinpoint a problem. To get a rough idea of what is going on, you’d need to comb through roughly half of the editor, as there’s no documentation and this is a multithreaded problem. Likewise, if it doesn’t crash, chances are grass layer won’t appear until you poke landscape with a brush at random point, which implies that there’s some sort of “dirty” flag… somewhere.

Or… trying to rebuild skin mesh from code past certain version will lead to a hard editor crash once the editor attempts to render said mesh for the first time. Investigating would lead you to certain render-related structure that has to be initialized for all meshes, but isn’t being initialized for your mesh specifically, and there does not appear any api function for initializing it.

Or… this is actually one of the easier ones. For example, when importing pngs, Unreal 4 at least in some versions sets RGB data to zero in transparent regions (A == 0). You’d expect there to be a switch for disabling this, but it does not exist, this is hardcoded. This is not what you want to do if you’re storing information in transparent region, but hey, at least in this case you can write your own texture factory, as the stuff which deals with direct png access is just a few levels deep within the original one.

Stuff like that.

Yeah, source code helps. No, it does not mean you’ll solve the problem.

17

But it at least makes it possible to fix the problem at all. Or at the very least make a more informed workaround if it happens somewhere too cryptic to touch.

With Unity the only option is to report a bug, upgrade your project, pray for the bug to be gone in the latest version, and for no new crippling bugs to come along. It’s the great gamedev equalizer: it doesn’t matter how much coding talent you have available, once you hit the closed source wall you’re as stuck as everyone else (unless you have enough budget to pay tens of thousands for a source license).

18

I believe that is untrue.

Having source code access does not mean that the problem can be fixed. And not having source code access does not mean the problem cannot be fixed. If you can sidestep it, then you can sidestep it without source. What’s more, having source code access does not even necessarily mean you’ll have easier time fixing it, because rather than poking it with a stick and figuring out what makes it die, you’ll spend time trying to fathom mysteries of alien logic used in the problematic spot, which can take longer.

Working with closed source platform is a very common thing, and when developing for windows, you CAN end up in a situation where you have a stack trace which ends up deep in the guts of ntdll. In this case you do not usually bemoan lack of source code access to ntdll, but look for way to figure out what happens and sidestep it. Because windows is even a bigger Leviathan than Unreal Engine (300 gigabytes repository, as they say), and you will not have time to do someone else’ job trying to fix it.

Likewise, if working on linux, you somehow hit a kernel level issue, if you aren’t already kernel developer, you won’t be fixing that. Because it takes prohibitive amount of time, and won’t be applied to every system you deploy on. So you won’t be diving into source, and instead will sidestep it.

That makes me wonder if you really need source code in the first place. Basically, the “under the hood” stuff does not have polish of the outer API, and is in “proceed at your own risk” category. Or even in “Abandon all hope” category.

It also makes me remember time of Accolade which reverse engineered a console to make games without acquiring proper license. Or when I, myself was writing a plugin for a closed source engine which implemented faux post-processing effect which the engine did not officially support.

In the end most likely scenario is that source code access matters when the source code is designed as a product, but in reality “under the hood” stuff is eldritch horrors, and is messy.

Also regarding this:

your other option is to replace faulty system with your own. That will be more efficient than praying, in my opinion.

I would advise to study techniques for debugging without source code access. Decompiling and disassembly are useful tools, same goes for tracing system calls, and so on, so it makes sense to add more items to your toolbox.

Rather than “I need source to fix it!”, I think “How can I diagnose and sidestep it if source code is not available” is a more useful approach.

19

I’m speaking from experience too. I use UE4. I’ve worked with it’s source since version 4.2. Modified a damn lot of it, did tons of feature back-porting, console SDK upgrades, and “we need to ship ASAP” hacks, all impossible in vanilla Unity. Between “eldritch horrors” and broken black boxes, I’ll take the horrors 100% of the time, thank you very much.

Just because it’s “possible” to debug things without source doesn’t make it nice.

Replace the faulty systems… sounds great, I would love to replace Unity with something else, but this isn’t my call, unfortunately. Enlighten, how should I replace the asset bundle async loading system by one that doesn’t hitches when it randomly decides to load stuff synchronously mid streaming? Or the shader compiler itself? Maybe it’s time to go nuclear and do all rendering via native plugins now?

And you said you were not patronizing. People will indeed defend anything on forums, damn. I should posting because if I continue I’ll get myself banned.

20

Frankly the middle path they have chosen seems like the worst of both worlds, like their development speed has already slowed to a crawl, since all the stuff they really care about are packages, but also there aren’t nearly enough open stuff that I could potentially decide to fix thing X that has bothered me for years if I really wanted to.