Unity 2021.2: Dedicated Server target and stripping optimizations now live! Please share feedback!

[UnityGio]

Last year, we shared our goals for 2021. Three of our key focus areas have been improving workflows and usability, improving performance and iteration speed, and focusing on multiplayer networking. As a part of this, we’ve developed a new way to allow developers to create Unity dedicated server builds regardless of the netcode you choose to use for your application.

What is Dedicated Server target?
Our new Dedicated Server is a desktop sub-target that improves server build efficiency by automatically stripping assets and code that are not necessary for server builds each time a project is built, providing increased space and processing efficiency. As a result, Dedicated Server builds also make it more economically beneficial for server operators to run these more-efficient server applications created with Unity.

Why did Unity create a Dedicated Server target?
Traditional Unity headless server builds often contain assets and compiled code which are not accessed by headless server processes. This might include artifacts such as audio files, textures, meshes, and shaders. In a multiplayer context, rendering and asset management processes occur unnecessarily when server runtimes are being built and executed. This even happens with traditional headless Unity servers since the rendering operations are still executed, though to a null-display target. When doing builds with all this additional overhead, longer build times occur and iteration speed suffers, too.

How do I use this new feature?
Starting in Unity 2021.2, when setting your platform target, you now have the ability to specify Dedicated Server as a target type with an option to choose for which Desktop platform. This not only makes it more clear what kind of build you are creating, but also allows Unity to know to apply these optimizations during the build process. Another advantage is you get to programmatically query if the target is set to Dedicated Server in order to program Dedicated-Server-specific behaviors.

How can I access this new feature?
Dedicated Server platform support is now available inside the Unity Editor as of Unity 2021.2.0b1. Before 2021.3 LTS, Unity will move the feature from the default Unity Editor installation and into a separate support installer via Unity Hub in a future release.

What feedback is Unity looking for?
We’ve reached the stage in our development where we’d like to get your feedback on this new functionality and how it performs when tested on real game projects. These changes are focused on the way we build dedicated server projects and the final content it is packed in it, therefore the types of feedback we’re looking for are:

• Data statistics on the projects you build with this version versus the results from build with previous versions of Unity. More specifically:

• Project size on disk

• Memory usage

• CPU usage

• Build time

• How this new Dedicated Server target feature impacts your workflow and iteration times

• If changes made by the Dedicated Server target could break or conflict with any previous approaches or features of your project

We welcome your feedback on ways we can improve this tool in the future, too! Please feel free to reply with feedback on the features that you would like to see in upcoming versions. If you would like to share your experience or provide any feedback or suggestions for us, please leave us a comment in this thread.

What about bug reporting?
Ideally we’d like any bugs reported through the built in bug reporter tool, as that will automatically provide us with some relevant context. For instructions on how to file bugs through the Unity editor, please go here.

Once you have submitted a bug report through the bug reporter, please feel free to refer us to it in your replies or new posts.

Changelog

• [UI] Dedicated Server target is now listed in the Build Settings window.
  

• [Memory Optimization] Pixel data for textures that are not marked as read/write is not included in Server builds.

• [Memory Optimization] Vertex data for meshes that are not marked as read/write and not used for internal systems, is not included in Server builds.

• [CPU Optimization] Audio system disabled in Server builds.

• [Scripting] UNITY_SERVER defined for Assemblies in Server builds.

Currently Known issues

• Error message “shader is not supported on this GPU” in Server Builds

[mischa2k]

@UnityGio

improves server build efficiency by automatically stripping assets and code that are not necessary for server builds each time a project is built

What code is stripped exactly?

If you don’t mind me asking, did you do any long running dedicated server tests?
For example, what about performance & memory leaks after several days / weeks?
Is there any data you could share with us?

Given all of Unity’s stability issues, people are often concerned about launching servers into the Unity black box.

[UnityGio]

@mischa2k

What code is stripped exactly?

No user code is affected. We follow the same stripping rules as the regular player. For now and in this first release, we’re only stripping and optimizing engine code around instance rendering and audio.

@mischa2k

If you don’t mind me asking, did you do any long running dedicated server tests?
For example, what about performance & memory leaks after several days / weeks?
Is there any data you could share with us?

Unfortunately, we don’t have long duration test outcomes, or performance improvement comparisons. This is the information we hope that you and other users can share back with us.

In terms of stability, there should be no crashes or memory leaks. As with any feature coming from Unity, we have a litmus of unit testing that we do on small samples and test suites. However, if any are discovered by the community, we do appreciate your bugs filed using the Bug Reporting tool.

[mischa2k]

@UnityGio

No user code is affected. We follow the same stripping rules as the regular player. For now and in this first release, we’re only stripping and optimizing engine code around instance rendering and audio.

Unfortunately, we don’t have long duration test outcomes, or performance improvement comparisons. This is the information we hope that you and other users can share back with us.

In terms of stability, there should be no crashes or memory leaks. As with any feature coming from Unity, we have a litmus of unit testing that we do on small samples and test suites. However, if any are discovered by the community, we do appreciate your bugs filed using the Bug Reporting tool.

Thanks for the reply.

Your community is more than willing to use Unity for their game servers.
Sometimes it’s not obvious if Unity is ready though.

For example, mention using Unity for a game server anywhere on the internet.
People just laugh at you. They laughed at us for years, still do.

Point being that nobody knows what’s going on inside the black box.

• Getting regular bugs fixed is virtually impossible. We’ve tried, a lot.
• If a production game encounters a leak, we can’t expect the poor QA guy to run it for a week with 100 CCU to reproduce it

Imho for Unity to become a serious option for game servers:

• Either the community needs to see inside the black box.

• Or the people that can see inside the black box need to show believable commitment to game servers.

Long running tests would be great.
Open sourcing more parts would be great.
Unity making their own multiplayer game would be even better.

[FakeByte]

Why is this called dedicated server build?
This would indicate it is building a server, but in fact it just builds a headless unity build.
So calling it Headless Build would fit much better. For example in network tests we also use headless clients that we can spawn, this means our headless clients would require a dedicacted server build?

[jesusluvsyooh]

Yeah might be time for name change, i build headless clients, and have had to explain in the past that headless server is for that too.

[andrews_unity]

@FakeByte

Why is this called dedicated server build?
This would indicate it is building a server, but in fact it just builds a headless unity build.
So calling it Headless Build would fit much better. For example in network tests we also use headless clients that we can spawn, this means our headless clients would require a dedicacted server build?

Well its not the same as the headless client entirely, previously we had the “headless” build option which depending on the platform meant different things. On Windows and MacOS this was just the normal players with a different entry point app and we just ran in batchmode. On Linux this option was a completely different binary from the standard player that actually had code differences for example we compile out things like input, audio, graphics backends etc… on top of running in batchmode. We also didn’t allow devs to provide specific player settings for headless vs the standard player.

This new target is new binaries for all platforms, meaning windows and macOS are also compiling out the un-needed code, we also defined a new sub-target that allows us to now to optimize specific to this target vs just hacking something on top of the standard player. This also enabled us to provide devs with the ability to have dedicated server specific player settings which means we can now customize that API where it makes sense to support the server target. So I would not say its the same as before because its not. Add to that as mentioned before we also do some stripping of content based on using this target that we know you could never use for example audio files, non-read/write textures etc… So it really does provide a better path for use to provide something more then we did before.

[JesOb]

May be you need to provide a way to keep audio files that can be read from cpu for audio centric game servers

[Neto_Kokku]

About time. No more having to deal with separate projects or juggling asset bundles to make builds without shaders, sounds, and textures.

Now find a way for us to test multiple clients on the same project without having to make builds or trick Unity with symbolic links and working with multiplayer in Unity can start looking less like a hack.

[andrews_unity]

@JesOb

May be you need to provide a way to keep audio files that can be read from cpu for audio centric game servers

I am curious the use case for this? I would argue more often then not folks do not want the audio files… we do keep some of the meta data linked to the AudioClip itself but just not the underlying raw audio data.

[JesOb]

You right
but there are games on your Engine (not my games) where audio is main part of game everything else is just environment around sound.
Such games can not be online with your new Server Target because you dont provide option to keep actual audio data

So building generic tech you must think about all usages not only most used ones.

[andrews_unity]

@JesOb

You right
but there are games on your Engine (not my games) where audio is main part of game everything else is just environment around sound.
Such games can not be online with your new Server Target because you dont provide option to keep actual audio data

So building generic tech you must think about all usages not only most used ones.

I don’t know if your statement is accurate as to people who have sound based games. I mean they could still throw their audio files in the StreamingAssets folder and load them and do whatever CPU foo they want to, it likely doesn’t make sense that they use AudioClips or the Unity Audio system on the server. I would almost argue it would be more performant that they not use AudioClips if all they are going to use that data for is their own Audio Processing.

[JesOb]

May be you right

But the same can be sad about textures or meshes that they dont need on server.

[Kyllthor]

That’s great, and why not take the idea a step further by developing a server engine isolated from the game engine. With an integrated and modular internet transport system, from there we could build all the multiplayer possibilities.

It would be very fast for multiplayer games. Allowing developers to use a server base with the whole structure of Unity is a big advantage. A market for high value AAA games.

Developing a server from scratch requires network developers, but starting from an empty solid foundation can be developed by game programmers, with less server training.

[jesusluvsyooh]

Looks like we can all agree, that customisable options is best way for the new headless feature, if for some strange reason people need to keep audio, or textures, shaders etc

Good luck [andrews_unity]( Unity 2021.2: Dedicated Server target and stripping optimizations now live! Please share feedback! members/andrews_unity.1004806/) , [UnityGio]( Unity 2021.2: Dedicated Server target and stripping optimizations now live! Please share feedback! members/unitygio.1847320/) and the rest of your squad!

[JagerZero]

Publish to Win And Linux, Crash during runtime.

Crash Report configuration:
* App Name: Island
* App Version: Unity 2021.2.0b6_2055a14f48ed
* Mono DLL: G:\Development\GameServWin\MonoBleedingEdge\EmbedRuntime\mono-2.0-bdwgc.dll
* Bug Reporter App Path: 
* Crash Report Path: C:\Users\JAGER\AppData\Local\Temp\Crashes
* Is Editor: false

[UnityGio]

@JagerZero

Publish to Win And Linux, Crash during runtime.

Crash Report configuration:
* App Name: Island
* App Version: Unity 2021.2.0b6_2055a14f48ed
* Mono DLL: G:\Development\GameServWin\MonoBleedingEdge\EmbedRuntime\mono-2.0-bdwgc.dll
* Bug Reporter App Path:
* Crash Report Path: C:\Users\JAGER\AppData\Local\Temp\Crashes
* Is Editor: false

Thanks @JagerZero . Because we make use of other metadata needed, and also leverage the system’s triage capabilities, may you please file a bug into our Bug Reporting system for this crash?

Instructions on how to do so are here: https://unity3d.com/unity/qa/bug-reporting

You can then follow-up with via direct-message to me with a link to the Fogbugz.

[Slandercakes]

This sounds interesting. I’m currently using Unity for a thin client for input + rendering that interacts with a authoritative C# server. It would be nice to take advantage of Unity code, like physics, to ensure the client + server simulation are the same, but I always thought the overhead of a headless client made it not worth it. This might change things for sure.

[MrG]

@UnityGio Is there a document that fully details what’s stripped? Lights aren’t mentioned above, and wouldn’t mesh colliders need the mesh? What can be configured?

[jesusluvsyooh]

hihi
Took me a few hours to figure out the problem, but got there eventually.

Mac building to Mac, gave this error, wouldn’t even run a blank headless scene.

Mac building to windows, gave this error, cannot build windows IL2CPP, even though it was set to MONO.

Removing Unitys ProjectSettings, close/reopen Unity, fixed both the above problems.

So something in 2021.2 or the headless feature doesn’t like Unity 2019 (and below) project settings files.

The following Audio error, when an AudioSource is on a prefab in a scene, removing the audio source from prefab makes the error go away.

And then i get spammed with .dll’s not found
(stripping is set to disabled, and allow unsafe code is true)
Might have to leave that part for another day, tired eyes.

(P.S everything worked and looked fine running in Editor)
Hope thats enough info to investigate with

Compression sizes:

2021.2, Non optimised headless build size = 212mb
^ Texture override to lowest “Max 64” = 105mb