Could games project their simulation a few milliseconds into the future so players see now?

Your game is running at 100 hz so you have 10 ms to

  • Update your games simulation on the CPU (7ms)
  • Pass the rendering information to the GPU (3 ms) [Your games 10 ms update time]
  • Send the pixel information update to the Monitor (5 ms)
  • The monitors display elements to change. (2 ms)

If you know the complete time it takes for the player to see the update instead of showing them the world from 17 ms ago you could time shift the simulation so they get a view of the world as it is when they see it.

Could this forward time shifting of your game work or would it add more problems than it solves?

I always love your threads, because at the same time they get me thinking while also scratching my head why you’d want to do this in the first place :slight_smile: :stuck_out_tongue:

But I would think you’d attack this from the reverse, where you wouldn’t simulate into the future but cache the state from the past. You could store the values of everything relevant to your game as it was 17 ms ago (2 frames?), which you could treat as your ā€œpresentā€ (even though it is really the past) for whatever you want that information for, and everything getting processed/rendered this frame, in the actual present, you consider the future in the context of your idea.

3 Likes

and it is great up until the point where we add real time interaction…

1 Like

What does this solve and how it is different than Stadia’s proposed predictive input?

Well it’s not about input lag it’s about output or display lag, gamers often want faster hz displays because the time gap between what the game simulation is doing and the display is showing are reduced often halved or even quartered. Therefore they don’t need to ā€˜aim off’ to compensate for display lag.

60 hz - 16.66…ms
120 hz - 8.33…ms
144 hz - 6.94…ms
240 hz - 4.16…ms
360 hz - 2.77…ms

Actually Stadia should have a system like this that compensates for network to display lag for each gamer so what they see and when they see it is not too out of date (I’m not familiar with Stadia’s predictive input system).

So gamers have to compensate for the fact that what they are seeing is a past snapshot of the games simulation time and adjust their actions to compensate.

If we gave gamers a real time view of the world where they see the simulation rendered to represent the time it is displayed that would mean they would not have to compensate as much.

And in theory if we combined that with input lag compensation they could respond in real time to the game.

To fully do it accurately we would probably need some hardware signal back from the monitors to ensure we have the correct timing information to calculate the display time offset. And from input devices to provide accurate lag compensation calculations. Or accurate timing information from display drivers.

Imagine games where regardless of frame rate it’s an accurate representation of now.

Could this actually also help in VR games with VR sickness, maybe output lag compensation could help?

I wonder if times that small really matter considering that the delay in the human nervous system is quite a bit larger.

https://pdfs.semanticscholar.org/fac5/9922c79f5e78f658d8c312d127a9346f2792.pdf

2 Likes

The issue is letting people interact with that system. The real concept of lag is between what the player sees on screen, the player’s reaction to what was seen, and then what is rendered next based on the player’s actions. The player is obviously part of that loop. Predictively rendering some future point in time does not take the player into account.

2 Likes

A bit more fun but similar results around 200 ms.

It’s not so much the absolute times as the offset from the simulation which is dependent on the movement speed of the player and game objects in the scene. So if you imagine a game running at 30 hz or 60 hz and your looking at 32 or 16 ms offset plus change. Which is not a lot if the game objects are static but with moving objects and distance to factor in you can start to grasp how quickly it becomes an issue in games especially ones with longer range targets.

And arguably a good player will be adapting their play to allow for their own reaction time offset. How else would we be able to play real world fast paced sports with high speed objects or even catch a ball. So your point could be a moot point.

Your idea would definitely make VR motion sickness far worse. Basically your idea completely ignores player input. Imagine rotating your head while wearing a VR headset, but imagine the in-game VR camera just randomly moving without any regard for your actual head tracking. That is what you are actually advocating in this thread. Basically guaranteed puking in VR.

2 Likes

To get an idea of how incredibly small 17 ms is, try to start and stop this timer in that amount of time. I know that elite gamers say it makes a difference, but I’m not convinced it does.

Click on the left side of the start button so that your second click is in place to stop the timer.

1 Like

As opposed to rendering some historic point in time that then players have to compensate for by predicting the future?

Lets say the following dotted line with chevrons represent a target.

What the players sees:
…>…

Where the game simulation is at the point in time the player sees the scene

…>…

By compensating for output lag the player sees the simulated world as it is now and not how it was.

Consider Peeker’s Advantage, it’s an FPS trick where the peeker knows they have a split second window of time before the game updates to show them to the enemy player. Just enough time to peek and shoot.

Good video on Peeking…

https://www.youtube.com/watch?v=5JaJcV26bpE

Of course this is often used in network play but how much of Peaker’s Advantage could be reduced with a output lag compensation system?

Why would a person need to compensate for a difference in time that they can’t even detect?

ā€œA response time of 100ms is perceived as instantaneous.ā€

https://www.pubnub.com/blog/how-fast-is-realtime-human-perception-and-technology/



https://blurbusters.com/nvidia-study-reveals-240hz-gives-you-edge-in-battle-royale/

Nvidia’s study seems to indicate a correlation between gaming refresh rate hz and performance for players.

Therefore if there is a jump in performance going from 16 ms to 7 ms and again at 4 ms then we could also see a display output offset impacting gaming performance.

PS John Carmack says this output lag compensation is built into VR gaming systems as it reduces VR sickness.

Now, I’m not saying that they’re lying, but they do have a monetary motive to push the idea that their more expensive hardware matters a lot. I’d like to see the data from a neutral party.

There is a point at which our technology is good beyond the ability to matter to the human brain. The numbers say that in many cases we’ve reached that. Now, if your game is slowing down to an actually perceptible FPS issue, that’s an issue. But if the difference is far smaller than we can perceive and far smaller than the delay our own neurons insert into the equation, then the differences advertised might only be a well done marketing gimmick.

https://www.youtube.com/watch?v=OX31kZbAXsA

1 Like

The first thing that video says is ā€œsponsored by Nvidiaā€. Not neutral. Don’t get me wrong, I love my Nvidia graphics. I’ve been a fan forever. All of my graphics cards have come from them since the 90’s. No hate on them. They’re just not neutral.

I’d like to see a study done by neurologists, perhaps from a top research university. But that’s me. Maybe I’m weird.

1 Like

Feel free to build a prototype of a game to test your idea. If you do that, then you will see what I mean. Your idea does not take player inputs into account. Your idea basically amounts to trying to predict what the player will do before the player does it, and then rendering that automatically before receiving the player’s input. The idea would be fantastic if and only if you are able to reliably predict every player input correctly in advance. If you fail even once to correctly predict what the player was going to do, then the game will instantly feel disconnected from the player inputs.

Arowx, feel free to completely ignore this fact and just post a bunch of unrelated videos and images.

2 Likes

Your premise is flawed. It’s people who are competitive who invest in the hardware necessary to reduce frame time.
So you can argue that performant players buy the pro hardware and not the other way around.

1 Like

What about the data they discovered on how total play time boosted performance and time using higher refresh rates improved performance faster…

And you seem to be base your counter-premise on how good players are, based on how competitive they are and their disposable hardware income.