Loading Times

Unity Engine
,
1

Hi all,
I would like to gather some feedback on loading times. It would be great if you could share some numbers, especially regarding asm.js vs wasm (regular build versus WebAssembly) on different browsers.
Regarding WebAssembly, keep in mind that browser vendors are still working on improvements, in fact you might notice differences if you run their latest versions: for example Chrome Canary build instead of latest stable Chrome (at this time, version 63 and 61 respectively).

Here is some code you can use to measure loading times:

   [...]
    <script src="Build/UnityLoader.js"></script>
    <script>
      var instantiateTime = Date.now();
      var gameInstance = UnityLoader.instantiate("gameContainer", "Build/<build name>.json", {onProgress: UnityProgress, Module: {
          onRuntimeInitialized: function () {
          var runtimeInitializedTime = Date.now();
          console.log("Download+Instantiation time: " + (runtimeInitializedTime - instantiateTime) + " ms");
          this.postRun.unshift(function () {
            var engineInitializedTime = Date.now();
            console.log("Engine initialization time: " + (engineInitializedTime - runtimeInitializedTime) + " ms");
            console.log("Loading time: " + (engineInitializedTime - instantiateTime) + " ms");
          });
        },
      }});
    </script>
   [...]

where “Loading time” is the aggregate download+instantiation and Unity engine initialization.

One note about Chrome, keep the DevTools closed while loading the page as I noticed that might affect loading times. In addition, remember to test using 64-bit versions of the browsers.

I’ll take this opportunity to summarize some best practices to reduce load times:

  • reduce code size

  • reduce data size

  • use gzip/brotli compression

  • enable compression support for .unityweb files on your server

  • make first scene small

I think these items have been discussed before so I am not going to add more detail but feel free to ask questions.
Thanks,
Marco

2 Likes
2

Here are some numbers from our benchmark project:

  • built with 2017.2
  • tested on Window 10 w/ 6 cores i7

FF55-asm:
Download+Instantiation time: 3481 ms
Engine initialization time: 937 ms
Loading time: 4418 ms

FF55-wasm
Download+Instantiation time: 1775 ms
Engine initialization time: 1468 ms
Loading time: 3243 ms

FF57-asm
Download+Instantiation time: 3339 ms
Engine initialization time: 1110 ms
Loading time: 4449 ms

FF57-wasm
Download+Instantiation time: 1581 ms
Engine initialization time: 1829 ms
Loading time: 3410 ms

Chrome 61-asm:
Download+Instantiation time: 6026 ms
Engine initialization time: 2268 ms
Loading time: 8294 ms

Chrome 61-wasm:
Download+Instantiation time: 2772 ms
Engine initialization time: 550 ms
Loading time: 3322 ms

Chrome 63-asm:
Download+Instantiation time: 5787 ms
Engine initialization time: 2364 ms
Loading time: 8151 ms

Chrome 63-wasm:
Download+Instantiation time: 2753 ms
Engine initialization time: 689 ms
Loading time: 3442 ms

We can gather the following:

  • both Chrome and Firefox wasm are faster than asm.js.

  • both Chrome and Firefox are faster at instantiating wasm than asm.js

  • Firefox wasm is slower at initializing Unity compared to asm.js

  • no much difference between current stable builds and nightly/canary builds

  • Chrome and Firefox are comparable when using wasm

  • Chrome is faster than Firefox at initializing Unity when using wasm

  • Firefox is faster than Chrome at initializing Unity when using wasm

of course depending on the project/device you will get different numbers so take them with a grain of salt and do you own profiling :slight_smile:

1 Like
3

I’ll try to push our test “loading project” to the cdn to share, but those findings roughly line up with what we observe. One thing you might want to note is that chrome doesn’t do AOT compilation for javascript/asm.js but does for wasm (I think eventually they plan to convert asm.js to wasm, but for now…). Thus if your primary bottleneck is the cpu, you can gain a decent performance boost by utilizing wasm on chrome.

Have you guys tested Edge (which will enable wasm by default for the fall creator’s update) and Safari (which also enables wasm by default for version 11)? I’m curious about your findings for those platforms.

4

Regarding 4th tip, enable compression support for .unityweb files on your server
I found out even I didn’t set anything on server(aws), it still uses gzip ( by logging in compress function in gzip in UnityLoader.js), in the code I found out it checked for data[0] and data[1] to see if it is certain value to determine if using gzip

5

Here are the loading times for my project:

Firefox 56.0 (64-bit):

  • Normal: DL+Inst=6715ms EngineInit=2572ms Loading=9287ms
  • Wasm: DL+Inst=17597ms EngineInit=3366ms Loading=20963ms

Firefox 58.0a1 (2017-10-11) (64-bit)

  • Normal: DL+Inst=3068ms EngineInit=2703ms Loading=5771ms
  • Wasm: DL+Inst=1622ms EngineInit=3198ms Loading=4820ms

Chrome Version 61.0.3163.100 (Official Build) (64-bit)

  • Normal: DL+Inst=11875ms EngineInit=4083ms Loading=15958ms
  • Wasm: DL+Inst=9071ms EngineInit=1512ms Loading=10583ms

Chrome Version 63.0.3236.0 (Official Build) canary (64-bit)

  • Normal: DL+Inst=10884ms EngineInit=6193ms Loading=17077ms
  • Wasm: DL+Inst=10796ms EngineInit=3063ms Loading=13859ms

I refreshed the browser a few times for each and took the fastest time. I also got these times from a webserver on my computer so download time is not a factor.
Unity version used to build this project is 2017.1.1f1.
Unity build settings: Non-development, 1024 memory size, explicitly-thrown exceptions only, gzip compression, debug symbols checked.
Computer: Windows 10 x64 i7-6800k 32gb ram

If anyone from the Unity team would like to see my project, let me know and I can provide it.

1 Like
6

Edge 16 loads in just under 4 seconds, Safari 11 in ~4.5s, however take into consideration that they have been tested on different hardware/os.

More recent tests show further improvements in Firefox on both Windows and Mac, and a possible regression on Chrome on Mac.

1 Like
7

Thanks for this script! We have these times for our project, built on 2017.1.0p4, and run on a i7-4790:

Chrome (62) :
ASM :
Download+Instantiation time: 8861 ms
Engine initialization time: 5940 ms
Loading time: 14801 ms

WASM :
Download+Instantiation time: 11478 ms
Engine initialization time: 3558 ms
Loading time: 15036 ms

Firefox (56) :
ASM :
Download+Instantiation time: 1637 ms
Engine initialization time: 7550 ms
Loading time: 9187 ms

WASM :
Download+Instantiation time: 3043 ms
Engine initialization time: 7662 ms
Loading time: 10705 ms

This is tested on a local server, so there is almost no download time. Clearly WASM is not worth it for our project right now. Any idea where this might come from? We have a memory size of 768MB.

1 Like
8

You could try Firefox Nightly and Chrome Canary. Likely, the versions you tried don’t have all the latest optimizations.

9

With the Nightly/Canary versions, I have this:

Chrome (Canary 64) :
ASM :
Download+Instantiation time: 6605 ms
Engine initialization time: 5676 ms
Loading time: 12281 ms

WASM :
Download+Instantiation time: 11994 ms
Engine initialization time: 2658 ms
Loading time: 14652 ms

Firefox (Nightly 58) :
ASM :
Download+Instantiation time: 1509 ms
Engine initialization time: 7125 ms
Loading time: 8634 ms

WASM :
Download+Instantiation time: 1091 ms
Engine initialization time: 8864 ms
Loading time: 9955 ms

This is better on every front, but mainly on Chrome with ASM build. But WASM still loads slower than ASM for me.

My build sizes are :
Data: 17951Kio
ASM (Code+Framework+Memory): 6733Kio
WASM (Code+Framework): 5061Kio

10

It sounds like on Chrome you will benefit a lot from the compiled WebAssembly caching we are working on. At least, that will speed up hot-reload by several seconds.

11

Are there any more details of this? What it does, how it works, etc?

12

This is what happens now:

  • download wasm code

  • compile wasm code

  • instantiate wasm module

however, it’s possible to store the compiled wasm into IndexedDB, so that the second time you open the same web page containing a wasm-enabled Unity build, you can skip #1 and #2.
This is something we are implementing right now and you won’t need to make any change to your project, apart from re-building.

1 Like
13

That sounds great! I’d love to hear before/after numbers for how much this decreases repeat load times. I’d also love to run my project through it once a preview becomes available, just to know how much that would help.

Right now I’m at the point where I’m trying to decide if I want to port my game into native Javascript, since my customers are regularly experiencing 30+ second load times trying to load Unity and I’ve exhausted all other options to speed it up. So knowing how much the caching will speed it up will help me make that decision.

14

That’s awesome!
We are currently working on a system to split a lot of our application into asset bundles, to speed up the “first draw” time. I’ll probably repost results of the difference between ASM and WASM when we have something working.

15

I should also mention a couple more optimizations that will speed up load times when the compiled wasm has not been cached yet. The first one is wasm tier-ed compilation (fast baseline + additional compilation passes for hot-paths). The other one is wasm streaming compilation which will allow wasm to be compiled while it is downloaded, especially useful for large code-bases. The former will be available to you as soon as it’s implemented by the browsers and will speed up existing wasm-enabled content. The latter requires both support in Unity and in the browsers, therefore it will take a bit longer before you can use it and it will need the Unity project to the re-deployed.

2 Likes
16

That all sounds great! Once you get to the point of measuring actual loading times, I’d love to hear the numbers.

17

I’d like to take advantage of caching the content, however I work on an enterprise-oriented application with continuous releases. Is there a preferred method of invalidating a client’s cache? Query strings aren’t ideal…