Should we be rendering to curved frustums for curved monitors?

[Arowx]

Frustums are flat and tend to be oblong like flat panel monitors.

However if the player is using a curved monitor should we also be curving the rendering frustum to match the viewing surface?

[bgolus]

@Arowx

However if the player is using a curved monitor should we also be curving the rendering frustum to match the viewing surface?

Technically, yes.

The next question is: why aren’t we already?
Well, because the OS doesn’t know the monitor is curved. AFAIK it’s not something the monitor data (EDID) has any information about, so unless someone builds and maintains their own database for, or it’s left as an option for users to enable, there’s no way for a game to know it should be curved.

If you do know the curve the next problem is GPUs aren’t capable of rendering using a curved frustum. Rasterization’s whole thing is predicated on having a “flat” frustum. This is a similar problem to what VR headsets have, as they need to render a very wide view that’s getting distorted by the lenses in front of each eye. If they could render with a curved frustum they could match the resolution and curve needed for the physical display. Instead they render at a significantly higher resolution than the physical display and distort that image to match the curve needed.

So why don’t more games do that? Because modern games have a hard enough time as is running at decent frame rates at the resolution of most ultra wide monitors. Also because most curved monitors aren’t curved enough for this to be obviously wrong.

@Arowx

Until now that is… That Samsung is a 1000R monitor. That means the optimal viewing distance is 1000mm (1 meter, or just over 3 feet) away. This fits well with how far a lot of people already sit from their monitor, but it means for the 32:9 49” Samsung G9 pictured above, that curve is most certainly going to be a noticeable distortion when playing a game.

[Arowx]

@bgolus

Technically, yes.

The next question is: why aren’t we already?
Well, because the OS doesn’t know the monitor is curved. AFAIK it’s not something the monitor data (EDID) has any information about, so unless someone builds and maintains their own database for, or it’s left as an option for users to enable, there’s no way for a game to know it should be curved.

If you do know the curve the next problem is GPUs aren’t capable of rendering using a curved frustum. Rasterization’s whole thing is predicated on having a “flat” frustum. This is a similar problem to what VR headsets have, as they need to render a very wide view that’s getting distorted by the lenses in front of each eye. If they could render with a curved frustum they could match the resolution and curve needed for the physical display. Instead they render at a significantly higher resolution than the physical display and distort that image to match the curve needed.

So why don’t more games do that? Because modern games have a hard enough time as is running at decent frame rates at the resolution of most ultra wide monitors. Also because most curved monitors aren’t curved enough for this to be obviously wrong.

Until now that is… That Samsung is a 1000R monitor. That means the optimal viewing distance is 1000mm (1 meter, or just over 3 feet) away. This fits well with how far a lot of people already sit from their monitor, but it means for the 32:9 49” Samsung G9 pictured above, that curve is most certainly going to be a noticeable distortion when playing a game.

What if we just rendered to multiple fustrums that make up a polygonal ‘curved’ surface, 3 might be the minimum but higher poly-ustrum counts might be needed depending on curvature and resolution.

[Antypodish]

I see this like that

So to me, that is quite a bit of overhead, to even trying tackle such technical challenge.

[bgolus]

Nvidia and AMD both have hardware features for doing multi-frustum rendering. This is used for VR rendering already, and used by literally one game to support multiple monitors. It indeed could be used to help with this. It still means rendering “too much” and distorting the result, though potentially much less. Almost no modern VR game uses this though as other options are much simpler to implement and offer similar performance benefits. Rendering to multiple frustums seems to have some overhead that just rendering to a higher resolution with some of the area known to not be needed depth masked out. For super wide FOVs (>140 degrees or so) multi frustum rendering becomes really useful again. The game that uses the Multi-view rendering is a racing game so you can have three monitors with the side two basically as side window views.

Here’s the thing though. That 49” 1000R monitor viewed from the “optimal” 1 meter distance is still only filling roughly 68 horizontal degrees of your actual view. Most likely you’re going to play a game on that with a higher vertical FOV, and a horizontal FOV in the 130~150 degree range (60~90 vertical FOV). PC gamers are generally pretty used to playing with a game rendering using an FOV that is way, way wider than the actual FOV the “window” of the monitor would produce unless they mash their face in their monitor (and even then…).

So, sure, we could take the time to get the monitor to perfectly match the FOV of the monitor based on some optimal view distance, just like they do for VR HMDs, but no one would play with those settings since the FOV would be way more narrow than most people like playing at.

Basically, gamers already play with weird distortion issues from playing games with FOVs that don’t match the one the monitor would produce if it were a real window, and generally just don’t care. Fixing the additional distortion from curved monitors is a bunch of work most people won’t notice the benefit of.

[Jeff-U1]

Does anyone know if Unity supports Nvidia’s Simultaneous Multi-Projection Engine? If it did, couldn’t it be used to render sections of a curved monitor?