Using Data Types in C# for Job System Efficiency

Entity Component System (ECS) Job System
, , ,
1

Hello everyone,

I’m back with a few more questions, hoping for advice from those with more experience. I recall reading (or perhaps misreading) somewhere that using int is preferable over smaller data types because the C# compiler handles int more efficiently—hence why loops often use int (e.g., i < 100).

With this in mind, I used int2 to represent grid points. However, I was also advised that I could use byte2 for grids not larger than 10x10, as this would be more efficient in the Job System and allow me to store more data in FixedListXXBytes.

Does this mean I should start writing loops with byte, knowing it might be faster in this context?

Another assumption is that even though short is light, it wont translate well to SIMD and thats why unity math only has intx, floatx etc and not bytex or shortx.

I’d really appreciate your insights.

Thanks in advance!

2

For local variables and most things that live on the stack, you might get a little better performance using int over byte. That’s because the compiler doesn’t have to insert sign-extension instructions when doing specific math operations on the variable. Plus, if your code is small enough, the compiler may put these variables directly in registers, where there is no difference between byte or float.

However, for types stored in collections, smaller types mean you can fit more of them in a cache line. And depending on how heavy your memory demands are, that can often more than make up for the cost of a few extra instructions.

As for SIMD, most hardware can support smaller types. Someone even wrote a library for it for Burst: GitHub - MrUnbelievable92/MaxMath: A supplementary SIMD math library to Unity.Mathematics, extending it to all C# numeric types while adding many new types and functions. Written entirely with hardware intrinsics, using Unity.Burst.

Lastly, don’t forget the number one rule to performance. Profile, Profile, PROFILE! Don’t waste your time optimizing things that aren’t actually problems in your real project.

4 Likes
3

Thank you for detailed explanation, now i understand that where to use one.

Is there any reason unity officially didnt support this? and someone deploying on android any chances above library will crash on some random users? (Sorry again asking this noobish quesiton as i dont understand behind the scene stuff)

4

If you want the utmost performance you should consider explicitly write SIMD-friendly code. The loop control variable matters little, even if you use a byte counter the CPU will internally work with an integer because the index variable occupies a CPU register which is 64-bits so whether you use byte or ulong for “i” should not make any difference.

What may make a difference however is, if you pedal to the metal optimize your inner loop, is to transform iterating over a int2 variable to iterate over an int4 or even an int4x4. By that I mean something like this:

int2 grid = new (pos[i].x, pos[i].y);

vs

int4 grid = new (new (pos[i].x, pos[i.y), new (pos[i+1].x, pos[i+1].y));

Provided that you can express your code without using conditionals you can execute 2-8 times the number of operations effectively for free per iteration. Ideally with a power of two sized array (eg 10x10 is suboptimal, 8x8 or 16x16 would be better to avoid having to separately process “non-power of two remainders of the loop”).

However, be sure to check with the Burst Compiler debug window and yes, this may require learning a few bits about SIMD assembly instructions because the compiler may automatically SIMDisize your int2 loop into vectorized operations performing 8 times as many operations per iteration. You can also use the Burst compiler directive where you indicate that the resulting code is expected to be vectorized but if it isn’t, you’ll get a warning or error (see Burst manual for details).

1 Like
5

I can only speculate.

For ARM, the library says it uses general-purpose C# implementations and no SIMD, though in some cases Burst is able to extract SIMD out of those implementations.

1 Like
6

First off, thanks @DreamingImLatios for mentioning my library. In this regard I want to update that it now includes SIMD code for ARM processors as well.

As for the original question, I want to add to the great responses already provided by offering a different persepective.

As has been mentioned, you want to use 32 bit and 64 bit data types on the stack, mostly.
This is not always the case; some algortihms are dependant on the bit width of the underlying data type and can be many times faster on smaller types… In fact this is a little more nuanced and not easy to generalize, but the performance impact of decisions made in this regard will almost always be extremely minimal.

When it comes to anything related to RAM, though, the size of your data types has a huge impact on performance.
As a premise, the Von Neumann bottleneck states that computer performance will, ultimately, always be limited by memory bandwidth.
DDR4 RAM offers you a bandwidth of about 200MB read per Frame, if you target 60fps. The reality is, though, that this number is much lower than that. Much of it is already taken up by engine code I’d imagine, plus the fact that even within one frame, you have tens to houndreds of hardware interrupts which loads and executes operating system code, which might also read from memory, just to swap your current process for a different one to exeute, possibly reading tons of data from RAM and thrashing the cache etc.
Point is: memory bandwidth is extremely limited and this becomes more relevant the larger the datasets you operate on are.
Given that data oriented design (“DoD”) is particularly effective when iterating over large entity bases, we can assume people who use DoD are interested in having large entity counts. Additionally, DoD offers excellent support for multithreading by design which is what we want to use excessively. So if many threads read(/modify)(/store) relatively large datasets simultaneously, that memory bandwidth becomes the bottleneck extremely quickly.
Given the history of hardware, this will only become worse, since CPU core counts tend to increase and memory has only ever fallen further behind CPU speed over time in the past.

Using smaller data types also gives you tremendous speed ups when cosidering SIMD. I have written a lot of assembly code (and equivalent); I can confirm that both x86 and ARM have native support for 8bit and 16 bit data types in SIMD registers. That in turn means that if you take the width of an int4 SIMD register, which is 4x4 = 16 bytes, and fill the entire register with bytes instead, you can operate on… 16 bytes at once! Operating on 16 bytes rather than on 4 ints is the same theoretical performance difference between scalar int code and SIMD int4 code.
In fact, Burst actually does SIMD-fy your byte code to use these 16-at-once operations if you have an easy to vectorize scalar loop where you operate on bytes. This is one of the things I meant when talking about the fact that the bit-width of variables on the stack and its performance impact is rather nuanced.

As a final note I want to mention bit packing.
Using 8 bit data types where appropriate will always objectively be the best solution with regards to performance. But you can go wayyyy beyond that and pack your data into bitfields.
I want to mention two real world examples:

  1. Bit Arrays: Since CPUs can only address individual bytes in RAM rather than bits, a bool will always take up an entire byte and waste 87.5% of space and thus memory bandwidth when read.
  2. An enum with the possible four values “Excellent”, “Good”, “Poor” and “None” only takes up 2 bits, but when stored as a byte wastes 75% of space and thus memory bandwidth when read.

Packing and unpacking these adds an additional, approximate 5 clock cycles to reading and writing individual values from/to a contiguous block of RAM respectively, but can almost paradoxically increase performance by orders of magnitude. Math on the CPU is very cheap; RAM bandwidth is extremely limited in comparison.
You also don’t have to do it for data types smaller than 8 bits either; if you know any value in a given array can only have up to 18 bits taken up let’s say, you would, rather than storing those values in ints, store each value contiguously in memory in order not to waste 14 bits = 43.75% of memory bandwidth per element, provided your array is large enough or, at least, use a 24bit = 3-byte software impremented integer type as a compromise.
These problems have also been solved by a library of mine (although it is currently being reworked).

TL;DR: Use the least amount of memory possible.

6 Likes