Where does it all go?

News & General Discussion
1

In theory we have CPU’s that can do millions of things every frame.

e.g:

A CPU running at 3.5 Ghz has time to do about 58,333,333.3 operations a frame.

And when you run a modern multi core cpu you can multiply that by the number of cores or threads.

Who’s stealing/squandering all that gaming processing power?

You could say hey it’s a 3D game you’re reducing your processing power by a third but chips have instruction sets to handle vector/matrix operations even batches of them (MMX, SSE, SSE2, SIMD ect).

And even at a third you still have about 19,444,444 3D operations a frame?

Example test code:

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using System.Diagnostics;

public class OneFrameTest : MonoBehaviour {

    List<int> counters;
    Stopwatch timer;

    void Start () {
        counters = new List<int>(frames);
        timer = new Stopwatch();
      
    }

    int frames = 100;
    int counter;
  
    void Update() {

        if (frames > 0)
        {
            timer.Reset();
            timer.Start();

            counter = 0;

            do
            {
                counter++;
            } while (timer.ElapsedMilliseconds < 16l);


          

            counters.Add(counter);
        }
      
        if (frames == 0)
        {
            for (int i = 0; i < counters.Count; i++)
            {
                print(i + " : " + counters[i]);
            }
        }

        if (frames >= 0) frames--;
    }
}

Results on my machine in Unity:

23 : 191496
24 : 191706
25 : 191835
26 : 191953
27 : 191625
28 : 191891
29 : 191696
30 : 187944
31 : 191885
32 : 190361
33 : 190641
34 : 191200
35 : 191646
36 : 191955
37 : 191502
38 : 190452
39 : 189946
40 : 203891
41 : 191700
42 : 191511
43 : 192046
44 : 191733
45 : 188930
46 : 197201
47 : 189692
48 : 190824
49 : 195638
50 : 187880
51 : 187643
52 : 190388
53 : 190285
54 : 189610
55 : 189504
56 : 198415
57 : 190303
58 : 191159
59 : 188409
60 : 189778
61 : 196251
62 : 191237
63 : 198426
64 : 188969
65 : 190379
66 : 188860
67 : 191096
68 : 191153
69 : 188934
70 : 191743
71 : 190792
72 : 190204
73 : 188921
74 : 191005
75 : 188810
76 : 188735
77 : 189483
78 : 190817
79 : 189095
80 : 187751
81 : 189009
82 : 198746
83 : 188778
84 : 188894
85 : 190369
86 : 191731
87 : 190610
88 : 191025
89 : 189469
90 : 188812
91 : 189051
92 : 190186
93 : 191331
94 : 189507
95 : 189101
96 : 194340
97 : 190848
98 : 190904
99 : 190922

Around 190,922 additions and comparisons per a 16ms frame or about 381,844 ops.

Which is a lot less than the 58,333,333 ops a 3.5 Ghz cpu should be able to do in 16ms.

OK there is the OS, Unity, .Net Mono then the CPU but this looks like I’m only seeing 1/152th of the raw gaming power I should be getting.

So, where does it all go?*

Or is there a good reason MS are bringing out a game mode to the OS?

2 Likes
2

Or What are we doing with the spare 141,173,635 ZX Spectrums? [Silly Thought]

If you take a modern CPU running at 3.5 Ghz with about 1.2 billion transistors, you have about 141,176,000 ZX Spectrums [ZX] (Z80 8,500 transistors @ 3.5 Mhz).

Now the ZX had a 256 x 192 x 3 bit display.

To run a screen a 1920 x 1080 @ 32 bits would only take about 473 ZX’s.

The ZX display ran at 24 Hz. To run at 120 Hz we need 5 x.

So that’s 2,365 ZX’s to run a modern game display.

That leaves us with 141,173,635 ZX Spectrums.

So what are we doing with all those spare ZX Spectrums?

3

Between software bloating and the focus of rapid development over efficient development you have wasted resources.

https://en.wikipedia.org/wiki/Software_bloat
https://en.wikipedia.org/wiki/Rapid_application_development

Microsoft? Good reason? What is this madness?

4 Likes
4

We’re making games that are not restricted to character graphics (the original ZX had a resolution of 256 by 192 but it was restricted to a color grid of 32 by 24 meaning you were drawing modified characters for your game), sound that is not restricted to ten octaves of beep, games that are no longer restricted to less memory than a smart card, etc.

https://en.wikipedia.org/wiki/ZX_Spectrum
https://en.wikipedia.org/wiki/Graphic_character

Memory card with 256KB. Just for comparison the Commodore 1541 gave 170KB of storage per side.

http://www.acs.com.hk/en/products/307/acos3x-express-microprocessor-card-contact/

5

Number of CPU operations is not directly correlated to clock rate. An operation may take more than one than clock cycle, on other hand, cpu may perform multiple operations ahead of time , then there are delays when dealing with IO, fetching data from memory, etc.

You can’t say “CPU can do 50 million operations per second!”.

Also, you don’t really know into what kind of code your program is turned after compilation. Measuring just addition and floating point comparison (which is what your code does) is rather pointless.

Modern computers are not zx spectrums.

ZX spectrum (64k version) didn’t really have a beeper, if I remember it correctly. There was a speaker, and you could set its level to high (1) or low (0), changing the state produced a click. If you wanted to beep, you’d to do it manually, meaning you’d need to busywait required number of milliseconds before toggling high/low state of the speaker. Which means you couldn’t really do anything useful while your program tried to play any music. Spectrum also didn’t have a system clock, meaning you couldn’t just measure how much time has passed, and needed to first prepare a table for notes (how many loops should I busy wait to produce a note of a correct frequency). Also, obviously, no polyphony of any kind. You could, of course try to put some sort of short “wave” into memory, but with 64k bytes of memory total with 16k of it being reserved for the ROM, and another chunk of ram reserved for the screen, this wasn’t a good idea.

6

Not sure about the 58 million operations per frame estimate but surely yes these modern machines should be able to do far more than we are getting from them. That’s just common sense really.

I think @Ryiah hit it when she mentioned the bloat factor. Over the years as computing power has increased the software side has just slapped on layer upon layer.

First, we have all of the layers in the OS itself. Basically something like Windows is just a bunch of layers upon layers from the front end people work with all the way down ultimately to the old DOS and device drivers in simplified terms. And there is just so much of it not just vertically but also horizontally which leads to…

Second, we have all of the crap going on in the OS itself. This is why gamers often use things like Razer Cortex: Game Booster to squeeze out a little more performance by just temporarily disabling some of the many unnecessary services that are running on the typical computer. Things just taking resources (RAM, CPU time and perhaps even performing disk IO).

Third, using something like Unity again you are dealing with layers of abstraction. And overhead from not only the layers themselves (who knows if you are working 10 layers of abstraction away from the actual end result or 50 layers or more) but also the overhead from the C++ → C# calls (such as Update which can be seen the more gameobjects you have each having their own monobehaviour with an Update called once per frame… basically think back about the Space Invaders games we each made last summer to compare)… and then physics and other things that are more complex than is needed in many cases. All of this stuff takes time. And this is not just Unity but most modern game engines.

Finally, in addition to all of that you have the layers upon layers of abstraction that most developers swear by. Meaning anything that is happening is abstracted away to some degree.

It’s basically the difference of… you want to take a load of limbs out of your yard and haul it off a 1/2 mile away. Instead of you simply loading the truck and driving it over and being done with it you drive the truck a block down the road and pull into a neighbors driveway. You then wait as they hop in the truck and head down the road another couple of blocks where they pull into a driveway. Get out. Another person gets into the truck and drives off. And this continues until the destination is reached and finally the truck is unloaded. Then everything is reversed until finally the truck is back by you, you jump in and return to your home.

Of course, all of these things are happening incredibly fast unless there is some kind of resource waiting or other delay going on… and there may well be real benefits gained from using such an indirect process if only to make the dev feel like they engineered a beautiful system… but the point is it certainly isn’t efficient.

Basically, the best you can do is to focus on the last two. Choose as streamlined of an engine / framework as possible and design as streamlined of an architecture for your games (or other software) as possible. And just doing those things can give you a hell of a performance boost.

If Microsoft, etc did it at the OS level we would see a pretty nice performance boost I think. If they had some kind of true GameMode that kill off say 95% of the crap while a game is running… I’d love to see that boost. Of course, they will still have all of the layers of abstraction to pass through. To really see you would need to get a modern machine and code up some very minimal OS. Or basically be something like how consoles used to be. These days even the consoles seem to have these huge bloated Operating Systems (comparatively speaking).

@Arowx why not try using the Razer Cortex: Game Booster and see how performance is boosted. On your standalone exe. Although it should be able to be used to boost even the Unity Editor performance I would think. Don’t expect some massive boost like tripling the speed but there definitely should be a boost just from killing all of that unnecessary crap while your perf test or Unity is running.

3 Likes
7

There have been a lot of higher-level advancements which seem like they are nothing major but actually take a lot of processing power and levels of abstraction to accomplish. I remember when I first saw that Unity was doing all of its collision detection using full-blown physics calculations, I thought to myself, oh my God, that is such a huge overkill, so extremely wasteful… especially lets say to do 2D games, even with the 2D physics now, some years ago if you told someone you’d be simulating an entire environment with realistic physics calculations for collision detection, they’d think you were nuts. It’s a lot of processing required to pull that off. And yet here we are now just taking it for granted that you can ‘easily’ achieve collision detection by adding a simple physics collider to something. And this is now used in games where physics isn’t even really needed, like for collisions on a tile map where everything is rectangular. So we’ve gotten really used to having this high-level developer-friendly features and ways of doing things that save time, but then to pull of that savings in time there is more realtime calculations needed. There’s a whole lot of convenience going on there and ‘ease of use’ and ‘taking the pain out of development’ etc.

That said I am still puzzled why certain operations in any software are taking so damn long when the cpu should be cranking out billions of calculations per second. As more cpu resources become available, they seem to be mostly gobbled up by lazy programming, a complete ignorance about optimization “because why bother, cpus are fast”, and very high-level functioning, feature bloat, generalization and abstraction etc, which all add up to huge amounts of ‘unseen stuff’ that has to be done to pull that off.

2 Likes
8

You could always ditch Unity and go program a large 3D game in raw assembly language, and enjoy the 10-100x performance boost, plus take many many many years to finish it.

6 Likes
9

Exactly.

10

That’s an extreme… be fun to do just from a technical perspective but there is a large area in between the two extremes.

First it’d be interesting for @Arowx to disable as much stuff in the Unity project as possible… if possible. Then test again.

Next use C++ and AGK library or C and Allegro library etc to perform the same test and see what the difference is.

Then maybe just create a straightforward C/C++ console app and perform the test.

Actually I’ll convert @Arowx 's code to a couple of different things and see how it does.

But basically it is just the layers. More specifically what I mean by layers… are unnecessary code outside of our control being involved (in some way like just it is also running to some degree even if just performing checks to decide nothing needs to be done) and abstracted (over generalized) code being involved.

I have Blitz 3D installed on my new laptop… and GLBasic which is very fast. And Monkey X and AGK. These are all just straightforward programming languages with game oriented APIs. Although one or two have built-in physics support I don’t believe it is engrained to the core and just there all the time. Might be wrong about that. But let’s find out what if any differences there are between a lighter weight API (that is likely using minimal, if any, abstraction layers) and a full blown game engine when running code that would seem to have nothing to do with using a game engine’s features at all.

EDIT: Actually the better thing to do is @Arowx just create your test program purely in C# outside of Unity. And see what difference, if any, that makes. You’re dealing with generic stuff and overhead layers from both Unity and C# at the moment. See if just using C# on its own has any impact on performance. If performance is basically the same then the issue is likely C# or just all of the other crap mentioned before in the OS.

11

Not sure exactly why you’re doing things as you’re doing them.

I just wrote a little CPU test doing the same variable counter looping operation in Blitz3D.

It just times for 1 second to see how many loops are performed and stores that value in an array.
It only does this 5 times.
Then the average of those is displayed at the end.

I then ran this test 5 different times and took the worst and best cases.

WORST / SLOWEST RUN 37.6 million

BEST / FASTEST RUN 37.9 million

Which gives us an average (based on those two runs alone) of 37.75 million loops per second incrementing a single variable each loop.

This means if targeting 60 fps we can do only about 629,000 operations (equivalent to the overhead of 1 loop iteration plus incrementing 1 variable) per frame.

I’ll knock out a C# test doing the same thing next.

1 Like
12

For C#, I just created a simple Windows form application with a button to click to start the timing test and a label to display the average of the 5 runs.

I actually should have declared the variables at the form level to stay consistent with the B3D test but it is close enough I am not worrying about it.

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows.Forms;
using System.Diagnostics;

namespace CPUperfTest
{
    public partial class frmCPUTest : Form
    {
        public frmCPUTest()
        {
            InitializeComponent();
        }

        private void btnStartTest_Click(object sender, EventArgs e)
        {
            Stopwatch oTimer = new Stopwatch();
            long[] aCounters = new long[5];
            long counter = 0;
            long index = 0;
            long total = 0;
            long average = 0;

            lblResult.Text = "calculating...";
            lblResult.Refresh();
            Application.DoEvents();

            for (index = 0; index < 5; index++)
            {
                oTimer.Reset();
                oTimer.Start();

                while (oTimer.ElapsedMilliseconds < 1000)
                {
                    counter++;
                }

                aCounters[index] = counter;
                oTimer.Stop();
            }

            for (index = 0; index < 5; index++)
                total += aCounters[index];

            average = total / 5;

            lblResult.Text = "AVERAGE: " + average.ToString() + " per second";
        }
    }
}

I wasn’t surprised to see that C# actually performed better than the old Blitz3D. I was impressed however that Blitz3D actually wasn’t that far behind the C# performance!

WORST / SLOWEST RUN ~44.3 million

BEST / FASTEST RUN 44.4 million

Which gives us an average (based on those two runs alone) of 44.35 million loops per second incrementing a single variable each loop.

This means if targeting 60 fps we can do only about 739,000 operations (equivalent to the overhead of 1 loop iteration plus incrementing 1 variable) per frame.

I should mention both the B3D and C-sharp tests were done under the Release profile not the Debug profile.

Tested on my laptop that has an Intel Core i7 CPU @ 2.50 GHz

I’d already say we can see we are obviously losing a hell of a lot of performance someplace. At the same time, both of these B3D and C-Sharp seem like they are capable of performing enough CPU processing per frame to handle games.

Yet we can also see we might not have nearly as much room as some people think we do on the CPU side… and this is why it is important to reduce the overhead to the bare minimum and that includes the callbacks, the abstraction generalization overhead, etc.

Especially since we know the graphics upload to the cards, rendering, etc needs some time and for many games these days the bulk of the time most likely.

2.50 GHz… and yet the original Doom was doing it all actually implementing its own 3D rendering engine plus the gameplay and running on what… ah here we go… 486 processor operating at a minimum of 66 MHz or any Pentium /Athlon processors

66 MHz … so let’s say it needed a 90 MHz to be largest window / full-screen and smooth. 90 MHz and it was doing it all including the rendering (building the display slice by slice). And these days we have 2.50 GHz (2,500 MHz) (multi-core which you’d think would at least cover the OS overhead if nothing else) CPUs and dedicated Graphics cards to handle all of the rendering as well. And this is why @Arowx is rightly questioning… where in hell is it all going. lol

I will continue on and profile a couple of more languages.

It would be great if someone converted the C# test to use Unity and then it will be clear once and for all how much overhead, if any, is added simply by using Unity.

1 Like
13

Someone who is extremely familiar with assembly (eg Chris Sawyer) might be able to pull it off in a reasonable time frame but for just about everyone else a good compromise would be C++ (modern C++ compilers are great at optimizing) with custom assembly for the worst performing portions of the program.

No Man’s Sky is one of the more recent examples I know of that used a custom C++ engine with custom assembly for the procedural generation (we found out because the Phenom II lacks SSE4 and that’s what they were trying to use for all CPUs causing it to hard crash at the splash screen).

14

I think this benchmark provides no useful ifnormation. Mostly because it measures one very specific operation, out of infinite number of combinations of instructions a compiler could produce.

The good compromise would be C++ maybe with C-style code for portions that need high performance. NO assembly. Assembly is not portable, and because of it, relying on it is a bad idea.

2 Likes
15

You could always do both but I agree that assembly is best avoided unless you know exactly what you’re doing and are only going to support a very limited selection of platforms.

16

I agree sort of. I look at it like the test @Arowx came up with represents about the minimum CPU-intensive task. Such a loop should be converted into highly optimized code that is (or was at one time anyway) cached inside the CPU allowing it to run at maximum speed. So looking at it from this angle… this represents the absolute best case scenario possible.

So there is value in that as it definitely illustrates the whole where does it all go point.

However, there is the always the consideration of just how much time is being used up to simply get the time value from the system clock. I have no idea if that is part of the CPU these days or truly its own little chip with a line to the CPU. Perhaps that is a bottleneck that is larger than we’d suspect. I am beginning to suspect this may be the case. I just don’t know how much overhead it adds.

17

The minimum cpu-intensive task is busy waiting through the message loop every modern program has.
I believe last time I checked the “framerate” of this on a simple windows application … that was a LONG time ago on a single core machine, the result was something around 40 000 (forty thousands) “frames” per second. This is a program doing absolutely nothing aside from calculating fps.

The problem with benchmarks (both yours and ArrowX) is that they are written pretty much with “real mode” thinking in mind. The program does not exist in vacuum, it is not the only thing running on a computer, so the whole “X millions of instructions per second” metric is not very useful.

Instead there’s a message loop, and when the program is not processing any OS message, it has a chance to do something useful.

1 Like
18

IIRC on windows platform the proper way to get precise timing is using QueryPerformanceCounter routine. There was a popular cpu instruction for measuring time, but it went bust when CPUs became multicore and true parallelism became possible.

Querying the time may actually be an expensive and an imprecise operation on its own (using GetTickCount on a fast machine could actually tell you that “zero milliseconds passed since the last frame” on some occasions). Rather than verifying “elapsed time”, what you’d need to do is to run the loop thorugh, say, one hundred million iterations and see how long it took. Without measuring elapsed time within the iteration.

1 Like
19

Oh yeah I get that. That is just a big part of the overhead what I call the horizontal stuff. Basically computer OS is always doing something. If our program is waiting then other stuff happening… then when ours wants to run may actually need to wait for the other crap to finish up first. And even when we are not waiting Windows is still going to take some slices… timer goes off for this thing or that thing… oh better check for Windows updates or maybe defrag the hard drive, etc.

Reason I wish we were using like DOS or C64 style approach still. lol

Yeah this is what I am thinking… previously when I timed stuff I always approach it from the other direction… I get the start time… I then do about a million (or in this case probably a billion) iterations and then get the end time.

But I wanted follow pattern @Arowx was using. I think we need to change it though because I really think it is this constantly checking the time that is the real bottleneck here.

20

Those high level instruction sets are just protocols and the operations they are talking about, which those higher level protocols get turned into, are still binary operations and more transistors means more binary operations in parallel but still nowhere near as fast as you think although they are making the displays bigger and bigger too using up much of the expanded transistor capacity added.