Web assembly benchmark

Has anyone done benchmarking of Unity web assembly builds compared to the js builds. Specifically I would like to know how web assembly compares with js in terms of:

Buildsize.
Initial parsing time.
Initial parsing in Chrome. Is the initial freeze still an issue?
Initial memory spike in Chrome. Does this still occur?
Runtime performance.

I would encourage people to share results too. Just bear also in mind that further optimizations by browser vendors are currently being implemented.

I have the same question, i did some testing and found some weird results.
I’v created an empty project that just shows “finished loading” after the unity splash screen.

Total startup time on a (not so good laptop) was slower using webassembly/firefox then js/chrome… ?
chrome : 9 seconds, firefox 12 seconds (this for an empty scene…)

On my computer i see little difference, webassembly is around .5 second faster.
chrome : 3 seconds, firefox 2.5 seconds

I’ll wait for 5.6 Release

Hi Marco,
Do you guys at Unity have some results that you could share? I think (by far) the most interesting benchmark would be Chrome as Firefox is doing pretty good with JavaScript already… I’ve done some very informal testing my self. Sadly the initial freeze in Chrome remains in the Tanks WebAssembly demo.

1 Like

didnt make any measurements, but tested old ludumdare webgl project on webassembly,

webgl: Unity WebGL Player | OneRoomLD37 *19mb 5.6.x beta
webasm: Unity WebGL Player | OneRoomLD37 *23mb 5.6.11 beta

…at least loading bar seems to work nicely :slight_smile:

*actually webasm gives some error after start moving (wasd keys)

I ran webasm and in log:
2blob:http://unitycoder.com/7c2ebc5a-6c16-4781-b0a8-707e5fd12bb8:6128 warning: 2 FS.syncfs operations in flight at once, probably just doing extra work
unitycoder.com/:1 WebGL: CONTEXT_LOST_WEBGL: loseContext: context lost

Hi,
We don’t have results to share yet. Last time we checked wasm performance in Chrome was definitely better than asm.js

Hi,

I have build the “Particles” example project that comes with Unity 5.6, and measured the performance with various build settings.

The only parameters I change is the “Graphics API” and the “WebAssembly”-flag.

I measured the:

  • Build size (.asm files are removed from wasm builds)
  • Memory consumption
  • Load time before unity splash screen is over

You can try out my builds for yourself:

My results from Unity 5.6.0f3 are shown in the figures below. “Auto” means that Unity could decide if the Graphics API should be Webgl1, 2 or both.

3058513--229719--Screen Shot 2017-05-05 at 15.13.30.png 3058513--229720--Screen Shot 2017-05-05 at 15.13.39.png

The load time surprised me the most, since I expected wasm to load much faster. I seems like Chrome is working on closing the gap, but there is still way to go.

Any idea how that might be?

1 Like

Thank you very much for taking the time to do this and share it!

One quick comment, yes Google is working on improving load times. As far as we know some of their optimizations should go in Chrome 59 and 60.
Any chance you could also profile Canary?

Sure, here it is.

I’m now on a cellular network, so to give a fair comparison to the chrome 60 (canary) I also recreated the chrome 58 figure.

Apparently Chrome 60 Canary is slower than Chrome 58… ?

These results match what we are seeing. The load time numbers for Chrome are somewhat disappointing indeed - but note that in all other browsers (Firefox, Safari, Edge) loading does become faster when switching to wasm. Likewise, runtime performance also improves in all browsers.

We expect that WASM will be faster and using low memory… 25% users of my Card game can’t run it and I don’t know why. I hope WASM help to me… when will it be?

Chrome is by far the most important browser, so this is very disappointing. Do you guys have any insights to when Chrome might surpass WebGl in terms of load time? Also: is there any improvements in runtime performance in Chrome?

Found a roadmap: https://wasmdash.appspot.com

1 Like

In Unity we use WebGL as a name for the build target, which is unfortunate, because WebGL is just a graphics API.

What we should really compare is asm.js vs. wasm. Google is working on improving loading times and you should see newer versions of Chrome getting better but we can’t really speak for them.

In terms of performance, Chrome definitely improved since they implemented wasm.

1 Like

hmm… It will be cool if we could to pack part of code in bundle… because it the biggest problem on start

Hi Marco,
I think it would be great if Unity could provide benchmarks. In my company it is quite a big decision to upgrade to Unity 5.6 and it would be extremely helpful with benchmark data to evaluate whether or not it makes sense at this time,

caozane1, you can use to 5.6 with asm.js (WebGL) and building with wasm only if you wish

Marco-Trivellato, you mentioned that:

You are very right that this is wrong, and I have therefore made a suggestion to how unity could solve this.

1. Suggestion:
In the Build Settings, rename WebGL to Web.
In the Player Settings, rename “Setting for WebGL” to “Setting for Web”.

Reason:
The platform is not “WebGL”, it’s Web. “WebGL” is as you mention only the Graphics API, and it’s not like you would call the Android platform “OpenGLES2”, or the Desktop platform “Direct3D/OpenGLCore”.

2. Suggestion:
Remove the WebAssembly toggle in publishing settings, and add a dropdown like this is instead:

Output format

  • Standard ECMAScript
  • Asm.js
  • WebAssembly

Reason:
As you mention we should really be comparing WebAssembly and asm.js. This would make it much more clear for your community that people choosing between asm.js and WebAssembly.

The current toggle makes it look like an addition to the build, and not a choice between something asm.js/webassembly.

Do you agree that this would make sense, and could you, in that case, push these suggestions further?

Thanks,
Birk