Well, the ultimate objective is to extract any data the client wants and capture it, while repeating the simulation in a deterministic fashion, so that we can extract different data, allowing us augment the previous data-set while keeping all the data in sync because the simulation runs in a deterministic fashion. And, if the client wants to be able to extract a different kind of data, then we can just throw that bit of code in without have to re-write any of the previous code.
When I implemented a Custom Pass that displayed the Motion Vectors after post-processing, all the negative values were clamped to zero by the AOV Recorder. It is possible that post-processing did this, but how it did this to something that ran after post is beyond me. The first attempt to get around that was to 01 compress the motion vector values like we do for normals, but 16 bit floating point numbers only hold 11 bits of precision, so “(value / 2.0) + 0.5” causes most values to drop to zero at 120 frame per second.
A quick example, lets say that our register can only hold 4 digits:
0.5 becomes 5.000 * 10^-1
0.00001 becomes 1.000 * 10^-5
0.5 + 0.00001 becomes 5.000 10^-1 + 0.0001 * 10^-1, but we can only hold 4 digits, so the 1 gets dropped.
0.5 + 0.00001 becomes 5.00010^-1 + 0.000 * 10^-1
Instead of getting 0.5001, we got 0.5.
That +0.5, was wiping out most of our motion vector data.
The next attempt around this was to just re-render (to a RenderTexture) at the spots the camera stops when the Recorder stops the camera to render, and it sort of worked. But, when we pulled the frame number from Time, it didn’t match up with the frame number the recorder was stamping into the images. Also, we didn’t have the time to check if the difference in frame number would be consistent across multiple machines. On the machine we tested this work around, the frame number in Time, was 4 frames larger than the one in the Recorder’s output images from the same location.
We (well, I) built my code on top of managing the AOV Extension Recorder because the base Recorder didn’t have exr output and it wouldn’t capture all of the lights and colors when set it to capture a series of images (it worked great for video capture). The AOV Recorder (Extension) plug-in had exr output and it was able to capture the entire output image when set to capturing a series of images. I still need to re-check this now that there’s newer versions of the base Recorder plugin. I know the base Recorder can now capture exr for 2020.3, but I haven’t check for 2019.4.
Yet, the reason why we’re using the Recorder is because one of our projects pre-dates the Timeline tool. It also pre-dates the Standard Surface Shader used in the Standard Render Pipeline, but it can’t predate it by too much, because it was using prototype Stand Surface Shaders. Despite getting the project to work in Unity 2019.4 and HDRP 7.5, getting the scene to run under the Timeline tool is more work than I can do with the time constraints.
Yet, one of the things that the Unity Recorder excels at, is getting the simulation to run the same way every time we run the simulation (as long as scene doesn’t contain non-deterministic random particles floating around like Morgan in the Heretic VFX scene).
So, one of the things I noticed about the new AOV API, is that it executes a delegate. This made me think, wouldn’t it be nice if we could just tell the Recorder to execute this bit of code when it tries to capture an image. So, I guess my question should have been, how does one create an extension plug-in for the Unity Recorder plug-in. I know it’s possible (or used to be possible) to create an extension plug-in for the Unity Recorder plug-in because the now defunct AOV Recorder is an extension to the Unity Recorder. The old documentation (actual documentation that tell me how the code interacts with the rest of the code) that I found for the Unity Recorder was for the Recorder that was used in Unity 2017.4. But, the Recorder gained a major performance boost in Unity 2018.4, so I don’t think I can trust that old documentation. Also, I lost the link.