I’m starting this thread to investigate speed differences among the 3 languages Unity supports. This is an offshoot of a discussion that started here.
It’s been tossed about for a long time that the languages are all compiled and end up the same speed in the end, but I’ve never seen any real data to show this. I recently did a few tests and discovered that there are indeed real speed differences among the languages after compile and that the assumption they all perform equally is incorrect. I converted my game to C# from UnityScript and found I got a 14.5% boost in speed during loading/processing of a level in the standalone build. I want to try to understand where those speed differences lie.
A couple of things upfront:
- I want this thread to be sort of a database of people’s findings comparing the speed of the languages.
- This is not the place for bashing or defensive posturing. This should be for data gathering and analysis and nothing else. (Discuss the findings, but please don’t get into arguments about which language is “better”.)
- This is meant to investigate where speed differences occur among the languages and possibly answer why.
- This is the place for both synthetic and real-world benchmarks.
- Feel free to investigate any variable such as target platform, using compiled DLLs, different compilers, etc.
Please feel free to suggest or use a different format/methodology.
Here is a summary of some of our findings so far:
Real-World Example: Game converted from UnityScript to C#:
I prepared two identical projects of my game, one in C# and one in UnityScript. (Sorry, I don’t have time to try Boo.) No special language optimizations were made during the conversion to C#. Here’s what I found:
Loading and processing one level:
Editor:
C#: 7.08s = (baseline)
US: 6.20s = 114.19%
Standalone:
C#: 4.33s = (baseline)
US: 5.06s = 87.57%
Each test was run several times and averaged one after the other on the same boot. (The first load was discarded so that disk loading would not be a factor.) I even re-ran the tests again just to make sure there wasn’t any issue being run in some particular order. There was very little variance.
Two interesting findings:
- UnityScript version loads in the editor about 14% faster than C# in this test.
- C# version loads in final build about 14.5% faster than UnityScript.
I’m very surprised to see this, especially the 14% margins. There actually are differences in speed among the languages, and not that small either. But what could be causing the difference? It’s really strange one would run faster in the editor and the other would run faster in a final build. (I haven’t done enough testing to determine why… that will be an ongoing investigation.)
It got me thinking and I decided to do some benchmarks of some very basic processes:
Synthetic benchmarks
[NOTE: Some of these results below are Editor only results. I will update this post when I rebenchmark them in the Standalone build.]
Write to an int variable
RESULTS:
Editor:
C#: 37ms - 38ms - (baseline)
US: 37ms - 38ms - 100%
Boo: 52ms - 53ms - 71.70%
Standalone:
C#: 8ms - (baseline)
US: 8ms - 100%
Boo: 8ms - 100%
Sources:
// C#
using UnityEngine;
using System.Collections;
using System.Diagnostics;
public class SpeedTestCS : MonoBehaviour {
void Update () {
var sw = Stopwatch.StartNew();
int x;
for(int i = 0; i < 15000000; i++) x = i;
sw.Stop();
guiText.text = "C# " + sw.ElapsedMilliseconds.ToString() + "ms";
}
}
// UnityScript
#pragma strict
import System.Diagnostics;
function Update () {
var sw = Stopwatch.StartNew();
var x : int;
for(var i : int = 0; i < 15000000; i++) x = i;
sw.Stop();
guiText.text = "US " + sw.ElapsedMilliseconds.ToString() + "ms";
}
// Boo
import UnityEngine
import System.Diagnostics;
class SpeedTestBoo (MonoBehaviour):
def Update ():
sw = Stopwatch.StartNew()
x as int
for i as int in range(15000000) :
x = i
sw.Stop()
guiText.text = "Boo " + sw.ElapsedMilliseconds.ToString() + "ms"
Write to a string variable
RESULTS:
Editor:
C#: 37ms - 38ms - (baseline)
US: 37ms - 38ms - 100%
Boo: 50ms - 51ms - 74.51%
Standalone:
C#: 8ms - (baseline)
US: 8ms - 100%
Boo: 9ms - 88%
Sources:
// C#
using UnityEngine;
using System.Collections;
using System.Diagnostics;
public class SpeedTestCS : MonoBehaviour {
void Update () {
var sw = Stopwatch.StartNew();
string x;
for(int i = 0; i < 15000000; i++)
x = "test";
guiText.text = "C# " + sw.ElapsedMilliseconds.ToString() + "ms";
}
}
// UnityScript
#pragma strict
import System.Diagnostics;
function Update () {
var sw = Stopwatch.StartNew();
var x : String;
for(var i : int = 0; i < 15000000; i++)
x = "test";
guiText.text = "US " + sw.ElapsedMilliseconds.ToString() + "ms";
}
// BOO
import UnityEngine
import System.Diagnostics
class SpeedTestBoo (MonoBehaviour):
def Update ():
sw = Stopwatch.StartNew()
x as string
for i as int in range(15000000) :
x = "test"
guiText.text = "Boo " + sw.ElapsedMilliseconds.ToString() + "ms"
Append a string variable
RESULTS:
Editor:
C#: 60ms - (baseline)
US: 82ms - 73.17%
Boo: 82ms - 73.17%
Standalone:
C#: 18ms - (baseline)
US: 37ms - 48.65%
Boo: 37ms - 48.65%
Explanation:
alexzzzz: (see post) C#'s IL code calls System.String.Concat(string, string). US’s code calls Boo.Lang.Runtime.RuntimeServices.op_Addition(strin g, string) which then calls System.String.Concat(string, string)
Sources:
// C#
using UnityEngine;
using System.Collections;
using System.Diagnostics;
public class SpeedTestCS : MonoBehaviour {
void Update () {
var sw = Stopwatch.StartNew();
string x;
for(int i = 0; i < 5000000; i++) {
x = "";
x += "test";
}
sw.Stop();
guiText.text = "C# " + sw.ElapsedMilliseconds.ToString() + "ms";
}
}
// UnityScript
import System.Diagnostics;
#pragma strict
function Update () {
var sw = Stopwatch.StartNew();
var x : String;
for(var i : int = 0; i < 5000000; i++) {
x = "";
x += "test";
}
sw.Stop();
guiText.text = "US " + sw.ElapsedMilliseconds.ToString() + "ms";
}
// Boo
import UnityEngine
import System.Diagnostics
class SpeedTestBoo (MonoBehaviour):
def Update ():
sw = Stopwatch.StartNew()
x as string
for i as int in range(5000000) :
x = ""
x += "test"
sw.Stop()
guiText.text = "Boo " + sw.ElapsedMilliseconds.ToString() + "ms"
Write to a string array
RESULTS:
Editor:
C#: 73ms - 74ms - (baseline)
US: 94ms - 95ms - 77.89%
Boo: 102ms - 72.55%
Standalone:
C#: 35ms - (baseline)
US: 35ms - 100%
Boo: 49ms - 71.42%
Sources:
// C#
using UnityEngine;
using System.Collections;
using System.Diagnostics;
public class SpeedTestCS : MonoBehaviour {
private string[ ] sA = new string[100];
void Update () {
var sw = Stopwatch.StartNew();
string[ ] s = sA;
for(int i = 0; i < 50000; i++) {
for(int j = 0; j < s.Length; j++) {
s[j] = "test";
}
}
guiText.text = "C# " + sw.ElapsedMilliseconds.ToString() + "ms";
}
}
// UnityScript
#pragma strict
import System.Diagnostics;
public var sA : String[ ] = new String[100];
function Update () {
var sw = Stopwatch.StartNew();
var s : String[ ] = sA;
for(var i : int = 0; i < 50000; i++) {
for(var j : int = 0; j < s.length; j++) {
s[j] = "test";
}
}
guiText.text = "US " + sw.ElapsedMilliseconds.ToString() + "ms";
}
// BOO
import UnityEngine
import System.Diagnostics
class SpeedTestBoo (MonoBehaviour):
private sA as (string) = array(string, 100)
def Update ():
sw = Stopwatch.StartNew()
s as (string) = sA
for i as int in range(50000) :
for j as int in range(s.Length):
s[j] = "test"
guiText.text = "Boo " + sw.ElapsedMilliseconds.ToString() + "ms"
Add to generic string list (old test style, will update)
RESULTS (fps): EDITOR ONLY
C#: 26.5 - 28.0 = (baseline)
US: 26.5 - 28.0 = 100%
Boo: 26.6 - 27.6 = ~100%
Sources:
// C# Test
using UnityEngine;
using System.Collections;
using System.Collections.Generic;
public class SpeedTestCS : MonoBehaviour {
void Update () {
List<string> list = new List<string>();
for(int i = 0; i < 1200000; i++) {
list.Add("test");
}
}
}
// UnityScript Test
#pragma strict
import System.Collections.Generic;
function Update () {
var list : List.<String> = new List.<String>();
for(var i : int = 0; i < 1200000; i++) {
list.Add("test");
}
}
// BOO TEST (for)
import UnityEngine
import System.Collections.Generic
class SpeedTestBoo (MonoBehaviour):
def Update ():
list as List[of string] = List[of string]()
for i as int in range(1200000) :
list.Add("test")
Getting a component and casting to transform (old test style, will update)
RESULTS (fps): EDITOR ONLY
C#: 34.2 - 34.5 = (baseline)
US: 34.2 - 34.4 = ~100%
Boo: 34.2 - 34.3 = ~100%
Sources:
// C# TEST
using UnityEngine;
using System.Collections;
public class SpeedTestCS : MonoBehaviour {
void Update () {
for(int i = 0; i < 100000; i++) {
Transform t = (Transform)GetComponent(typeof(Transform));
}
}
}
// UnityScript Test
#pragma strict
function Update () {
for(var i : int = 0; i < 100000; i++) {
var t : Transform = GetComponent(Transform);
}
}
// BOO TEST (for)
import UnityEngine
class SpeedTestBoo (MonoBehaviour):
def Update ():
for i as int in range(100000) :
t as Transform = GetComponent(Transform);
Getting a component using generics (old test style, will update)
Interesting note: Using generics is slower than casting. Runs at 84.34% speed compared to casting.
RESULTS (fps): EDITOR ONLY
C#: 28.9 - 29.1 = (baseline)
US: 28.8 - 29.1 = ~100%
Boo: 28.7 - 29.0 = ~100%
Sources:
// C# TEST
using UnityEngine;
using System.Collections;
public class SpeedTestCS : MonoBehaviour {
void Update () {
for(int i = 0; i < 100000; i++) {
Transform t = GetComponent<Transform>();
}
}
}
// UnityScript Test
#pragma strict
function Update () {
for(var i : int = 0; i < 100000; i++) {
var t : Transform = GetComponent.<Transform>();
}
}
// BOO TEST
import UnityEngine
class SpeedTestBoo (MonoBehaviour):
def Update ():
for i as int in range(100000) :
t as Transform = GetComponent[of Transform]()
Getting a component by its base class (old test style, will update)
RESULTS (fps): EDITOR ONLY
C#: 40.4 - 40.5 = (baseline)
US: 40.4 - 40.5 = 100%
Boo: - = %
// C# TEST
using UnityEngine;
using System.Collections;
public class SpeedTestCS : MonoBehaviour {
void Update () {
for(int i = 0; i < 100000; i++) {
BaseCS b = (BaseCS)GetComponent(typeof(BaseCS));
}
}
}
// BaseCS.cs
using UnityEngine;
using System.Collections;
public abstract class BaseCS : MonoBehaviour {
}
// ExtCS.cs
using UnityEngine;
using System.Collections;
public class ExtCS : BaseCS {
void Start () {
}
void Update () {
}
}
// UnityScript Test
#pragma strict
function Update () {
for(var i : int = 0; i < 100000; i++) {
var b : BaseJS = GetComponent(BaseJS);
}
}
// BaseJS.js
#pragma strict
function Start () {
}
function Update () {
}
// ExtJS.js
#pragma strict
public class ExtJS extends BaseJS {
function Start () {
}
function Update () {
}
}
) And as for the synthetic bencharks, they are already compiled, and the standalone versions are compiled without debugging code, and yet show differences in many cases. And I know these simplistic benchmarks aren’t going to reveal a whole lot, but it’s just a start. I hope to expand this and add benchmarks from various portions of my game in C#/US as time permits. (Real-world is always better than synthetic.)