[RELEASED] Camera Projection Rendering Toolkit (Non-linear perspectives, oblique projections, SSAA)

Camera Projection Rendering Toolkit is a plugin that bring some projection & anti-aliasing functionalities:

  • Wide field of view perspective with non-linear projections techniques

This plugin introduce Stereospherical (Fisheye) and Pannini perspective technique that solve the deformation that can occur with a wide field of view on objects far from the center of the screen.
I let you judge with these comparative 110° FOV screenshots:


StereoSpherical “Fisheye”: this perspective projection is the mathematically most accurate imitation of what the human eyes see.

Pannini: Inspired by the painter Giovanni Paolo Pannini, this perspective projection isn’t very realistic but allows the drawing of wide panorama and gives a pretty good feeling of being small relative to the architectural structures. It has the advantage of keeping the linearity of vertical lines. This projection can be free or horizon-aligned to prevent strange deformations.

These non-linear perspective projections gives more space to the objects in the center, which has the effect of “zooming” the center pixels. This is why I needed to oversample the rendering with SSAA to avoid blurred/pixelated image. You can expect FPS drops if you use very wide FOV because of the oversampling (110° needs x1.6 SSAA, so a 1920x1080 rendering viewport will become a 3072x1728 one).
Warning : this is not intended for VR.

  • Super-Sampling Anti-Aliasing (SSAA) / Pixelization

The plugin allows you to add SSAA to take high resolution screenshots and to add high quality Anti-Aliasing even in deferred rendering (which isn’t supported by Unity’s MSAA).

It can be GPU-heavy but this Anti-Aliasing improve image quality on the entire surface, including edges and opaque/transparent textures.

SSAA quality is currently limited to 2x, which is equivalent to a “better” MSAA 4x. It’s quite enough for real-time purposes and can be improved with other Anti-Aliasing techniques.

It can also pixelize the image to give it a classic “Quake-like” feel.

  • Oblique orthographic projections

If you’re not looking for perspective, this plugin can produce oblique orthographic projection and will help you give a more classic look to your game/project.


This screenshot shows you the 4 obliques preset, but you can customize them.

  • The plugin hasn’t been tested with VR yet

  • It can take high resolution screenshots.

  • The whole plugin is compatible with image effects (glow, FXAA, SMAA…)

  • The final resolution is limited to 16384x16384 pixels (8192x8192 on older hardwares).

  • Interactive inspector window (automatic buttons, warnings)

  • Works with multiple cameras, render textures, forward and deferred rendering, linear and gamma color space, post-processing effects (HDR, Bloom, FXAA…)

  • Tested with Unity 5.6.1 and 5.3.0, not compatible with Unity 5.6.0

  • Comes with PDF documentation and exemple scenes (those seen in screenshots)

  • Supports picking

Asset Store Page

ChangeLog

-1.6

  • Added the pseudo orthographic projection which mimics an orthographic projection with a perspective in order to allow deferred rendering with an orthographic projection.
  • removed a bug when disabling the CPRT component.
  • removed bugs in the CPRT editor.
    -1.5.1 :
  • Improved CPRT compatibility with various external post-process plugins
  • Added an optional CPRTFrustumUpdater script :
  • Some external plugins can interact with the camera during the “OnPreCull” Unity callback, and may be in conflict with CPRT. CPRTFrustumUpdater allows to reorder the CPRT camera handling, and should typically be first so the other scripts can work on CPRT camera settings.
  • Oversampling setting interface has been slitghly reworked :
  • You’ll likely need to reconfigure the oversampling in your scenes,
  • To make the setting clearer you now have two sliders : you can either set the render size multiplier or the Min Pixel Samples Count (minimal oversampling in the screen, 1 ensures that the image won’t be downsampled at any place even with non-linear projection, 4 produce SSAAx2),
  • programatically, oversamplingFactor stays unchanged but is deprecated, please use renderSizeFactor instead from now on. The minimal oversampling in the screen can be set with the MinPixelSampleCount property,
  • Those changes applies to screenshot settings.
    v1.5 :
  • New “Screenshot Pass Count” settings :
  • Use screenshotPassCount to do successive rendering to help troubleshooting post-processing effects which accumuate samples over frames (like some ambient occlusion or global illumination effects do).
  • Set screenshotPassImproveSSAA to true to gather more antialiasing samples (for a maximum of 64 samples with SSAAx2 enabled).
  • Fixed a compatibility issue with SonicEther’s SEGI effect.
  • Enabling / disabling the script should no longer make the camera unusable.
  • FOV can now be changed without calling RefreshEffect and RefreshViewport.
  • Added a CPRT.WorldToViewportPoint method (equivalent of Camera.WorldToViewportPoint), it will help you to project a world point into the screen.
    v1.4 :
  • Added support for AdaptivePannini camera-picking.
  • Adaptive Pannini has been fixed to keep viewport center at screen center, and to feel more natural.
  • CPRT control panel doesn’t overstep the inspector width anymore.
    v1.3 :
  • Added support for camera-picking :
  • ScreenPointToRay and ViewportPointToRay are two new method that can help you to pick object from your camera while taking into account the projection distortion,
  • New showcase scene “MultiCamera Picking”,
  • Supported projections are the “Stereospheric” and the “Pannini”, support for AdaptivePannini will come later.
  • New script “AutoOversamplingScript” which adapt oversampling quality in real-time in accordance with the framerate,
  • Added Diagonal FOV setting (real-life camera common characteristic),
  • New property “RenderingFramerate” which gives the duration between two actual rendering.
    v1.2 :
  • fixed bugs when releasing executable with the plugin
  • made the documentation more user-friendly
  • added a TempRenderTextureManager.cs script that can help you to setup post-CPRT effects
  • added two test scenes
  • Surveillance Screens: demonstrate how to setup realtime render texture
  • Post-CPRT FX: demonstrate how to setup a post-CPRT chromatic aberration effect
    v1.1 :
  • added Horizontal FOV setting,
  • fixed some issues with render textures, you can no longer use the classic “Camera.targetTexture” parameter, you must use the script’s “targetTexture” one,
  • fixed a memory leak and did some optimisations.

I’m trying to make this plugin as simple as possible. Don’t hesitate if you have suggestions, questions, requests or if you encounter any issue.

1 Like

Hi there!

I’m interested in using your asset in VR (Oculus or Vive), and I was wondering if its geometric transforms are going to work just as fine in VR as well (I’m particulary interested in the Pannini one).

Do they have any perceivable performance impact?

What about SSAA? Is it going to work in VR as well, both single pass and multipass? Linear and Gamma color space, deferred or forward rendering?

Hello NeatWolf !

The geometric transformations part of the pluggin hasn’t been specifically created with VR purposes in mind so I can’t tell you with certitude what it will look like in a VR environment (I don’t have a VR headset to test any of this at the moment, sorry)
Same answer for SSAA, but my intent is to look into it in the near future, since the SSAA compatibility seems to be a popular demand.

About the performance impact, since you need to oversample the image to do these transforms, the more you ask from the plugin, the worse the performances will get.
For exemple, 110° FOV projection will need as much processing as a x1.6 SSAA one (a 1920x1080 rendering viewport become a 3072x1728 one).

To end on a high note, the whole plugin has been tested with Linear and Gamma color space, deferred and forward rendering and Unity post-processing effects (HDR, Bloom, FXAA…) so you can already roll with theses.

Note : I just updated the main post with these informations

1 Like

Thanks for your answer, for some reason I never received the notification.

I’m going to buy it right now, and will send you some feedback about VR with different headsets as soon as possible.
The Pannini projection is really interesting :slight_smile:

I’m not sure on how you’re handling the projections since the asset is currently only in the cart.
Do all of them require upscaling to look good?
Are they fullscreen postFX?
Which is the correct order in the FX chain? Absolute first?
Does it interfere with standard screen-space-coordinates raycasting?

Sorry for all the questions :slight_smile:

Take my money! Please! These projections look great! And they work in deferred? You’ve saved me a ton of time. Thanks!

@Sluggy1
Thanks for the kind message !

@NeatWolf
Not receiving notifications either, no worries !
Don’t hesitate to ask as many questions as you want, I will try to answer them as quickly and precisely as I can.
So, about your previous questions :

  1. Yes, all the projections require upscaling to look good or else the center of the rendered view will be blurry
  2. The pannini and stereospherical projections are fullscreen postFX but the oblique ones (iso, top down…) are not
  3. For a given camera, the correct position is absolute last (I’d like to change this to be more flexible in the future, I don’t know if this will be feasible yet). That said, you can still add your effects (like some glow) on the render texture (Camera.targetTexture) you’ll use to draw the screen, effectively applying it after the transformation of the plugin.
  4. Yes it unfortunately interfere with standard screen-space-coordinates raycasting and that’s the top item on my todo list. At the moment, the farther you get from the center pixel (or the 4 in the corners) the more error you will get.

Anyway thank you for your interest and don’t hesitate to contact me about the pluggin behavior in a VR environment, I will gladly look into it :slight_smile:

PS : I will update the plugin on monday with a fix for a small bug I noticed

1 Like

Hi guys,

Version 1.1 is now available, here are the changes :

  • added Horizontal FOV setting,
  • fixed some issues with render textures, you can no longer use the classic “Camera.targetTexture” parameter, you must use the script’s “targetTexture” one,
  • fixed a memory leak and did some optimisations.
1 Like

I bought this asset and I’m using it in Unity 5.6.3.p1.

It’s working perfectly in the editor but when I build my project it doesn’t work. There are no errors in the log but what I see on the screen is messed up. There is like a blue haze at the bottom of the screen and the actual fov and zoom settings on the camera are all incorrect …

Seems my output log didn’t update correctly, I’ve run it again and now I’m getting a message saying CPRT shader is both missing and not supported on my platform.

I believe I found the problem. Nothing references the shaders for this toolkit, so even if you add the script for it to your camera it will not include the shaders in a build, although it will work perfectly fine in the unity editor. The solution was to add the shaders under the “Always Included Shaders” list in ProjectSettings/Graphics.

Hello catfink

Many thanks for your support and having pointed out that issue. I’m currently working on a fix for it. I’ve already identified where the problem lies in the code and have found a way to solve it, but for now it’s a bit clumsy. I’ll update the plugin as soon as I have something proper for you guys.

I also just fixed a bug that prevented scripts from working outside dev builds and editor. That will come in the same update (hopefully this week).

Sorry for the inconvenience.

Hi there Melcx,

Did you find a workaround for this? Not being able to interact/Raycast from screen space to world space is quite a big limitation for those, as it would be useful just for showcasing. Or, maybe, there is a simpler way to do it that I am missing?
Having at least a custom “bending” Raycast solution for each non standard visualization would be nice, but I’m not aware of the math required to implement it.

Cheers,
Alessandro

Thanks for the reply. Just for info I also had to play around with the #if UNITY_EDITOR usage inside the scripts as it was throwing an error on the “using UnityEditor” includes. It’s not a big deal but it may confuse some less experienced devs.

@catfink
Yes, that’s exactly the fix I was talking about that will come with the next patch.

@NeatWolf
I’m still working on this issue, which is - as you mentioned - quite big, an also not a simple one to solve. This is still my top item on my TODO list, I just took the time to fix the “easier” issue risen by catfink, since I knew right away how to handle it.

Still have to update a bit the documentation to reflect the recent changes, and I’ll upload everything.
Then I’m back on the raycasting issue :slight_smile:

This is a very interesting asset for making a better rendering without distortion. I have 3 questions:

  • Is a post process effect? it renders into a mesh like other solutions ? or is a new way to rendering in the camera, and how is the component attached to the camera?
  • How is the shape of the near and far limits? are just distance plain limits (so object enters and escapes in the center of far objects) or by distance ray length (radial or radius)?
  • Do you think this will work in Unity 2017.2 (or else) OTOY rendering?

It is a post effect and uses a shader to render the camera differently. The effect is attached to the camera using a script.
I know you raised this particular bug to Unity (I voted on it 9 times :smile: ). I am using this asset as a workaround to that problem.

https://feedback.unity3d.com/suggestions/correct-camera-distortion-issue-on-the-side-by-using-spherical-clipping-planes-instead-of-flat-near-far-clipping-plane

The scriopt oversamples the screen and allows you to get a much wider field of view than a normal unity camera and without all the horrible distortions. The downside is it is GPU heavy as you have to oversample the screen to achieve the results.

The author will have to answer your other questions as I haven’t looked deeply into how it works although all the source is included in the asset.

1 Like

@AlanMattano

  • Yes it is indeed a post-processing effect, asking the camera to render the scene in a different resolution according to the settings.
  • In world space, the near and far limits are planes, like regular cameras, but the perception of a plane is distorded by non-linear projections.
  • I just tested it in Unity 2017.2, it works fine. However, OctaneRender (if that’s what you’re talking about) is based on a completly different renderer, and I highly doubt about its compatibility. Moreover OctaneRender already implements those non-linear projections.

About the spherical clipping planes, I’m not completely sure how you’d like to achieve this kind of projection, but what I know is that if you try to use the current Unity rendering engine (which uses DirectX and OpenGL) with the GPU rasterizer (without CUDA/OpenCL custom solution) you’d run into two issues:

  • First of all, matrix math can only produce linear transformations, and that’s what creates the rectilinear projection you see in nowadays games. A workaround would be to create a new transformation type to achieve the spherical projection.
  • The second and biggest issue is that a spherical/cylindrical projection will transform some lines into curves. The Unity rendering engine is only able to produce triangles and lines (if we don’t take post-processing effect into account). Meshes are shaped by Vertex Shaders, and Pixel Shaders only setup triangles content. You can’t customize anything between these two steps.

If you want to change these rules, you’ll have to use or create another rendering engine, not based on DirectX or OpenGL, probably in ray/path tracing to ease the calculations. OctaneRender is a good exemple.

I just uploaded the CPRT 1.2 version, it will be available as soon as Unity validate it.

Have a nice weekend!

Edit : CPRT 1.2 version is now online, here is the changelog :

  • fixed bugs when releasing executable with the plugin
  • made the documentation more user-friendly
  • added a TempRenderTextureManager.cs script that can help you to setup post-CPRT effects
  • added two test scenes
  • Surveillance Screens: demonstrate how to setup realtime render texture
  • Post-CPRT FX: demonstrate how to setup a post-CPRT chromatic aberration effect
2 Likes

Thx @Melcx & @catfink for the deep explanation. Now I will include this asset and let the player choose it by using the rendering settings.

@NeatWolf
Hi there, version 1.3 has just been released and picking is now supported for main projections (Stereospherical & Pannini). Support for AdaptivePannini will come in the next version.

Here is the complete changelog :
v1.3 :

  • Added support for camera-picking :
  • ScreenPointToRay and ViewportPointToRay are two new method that can help you to pick object from your camera while taking into account the projection distortion,
  • New showcase scene “MultiCamera Picking”,
  • Supported projections are the “Stereospheric” and the “Pannini”, support for AdaptivePannini will come later.
  • New script “AutoOversamplingScript” which adapt oversampling quality in real-time in accordance with the framerate,
  • Added Diagonal FOV setting (real-life camera common characteristic),
  • New property “RenderingFramerate” which gives the duration between two actual rendering.
1 Like

Yay! Thanks!

Pannini is my favourite so far so I’m looking forward the next update as well! :slight_smile: