The Recorder team would like to get your take on how/if you use Accumulation with HDRP and the Recorder. Accumulation uses multiple sub-frames to converge over time on a higher quality final image.
Do you use Accumulation today?
If so, how well does it meet your needs?
If not, is it because:
It lacks features you need?
It’s not performant enough?
It’s too hard to use?
You didn’t know about it?
What features does Accumulation lack or need to do better?
Your feedback will help drive our roadmap…so please be vocal and as blunt as necessary to let us know how we can improve.
P.S. If we get high quality feedback using this approach, we’ll likely ask on this forum more regularly.
Love that you are asking the question so I will reply (even though low quality reply).
I don’t use Accumulation today - I don’t use HDRP at this stage because I wanted to master the basics first and I had problems with some tools and Unity asset store purchases trying to use HDRP. So I have stuck to the default render pipeline for now.
I did not know about Accumulation.
For me (off topic) it is show stopper bugs I would address before adding new features. Example (based on the world revolving around ME! ME! ME!), I would love to use Animation Rigging and Sequences together. E.g. do a live recording, but then override it so hand reaches out to grip door handle (hard to line up in a live recording). But they throw exceptions. (Bug has been registered.) So the feature is completely unusable.
Or another one is Recorder with MP4 generates very low quality video files if the scene gets too complicated. You cannot tell Unity to “create good quality video no matter what”. Fancy tools to make the creation better are not useful if I cannot generate a good quality video file when done (for Film making).
Once the basics work reliably, then I am interested in fancy features (e.g. richer depth of field blur etc). (But I appreciate you may have different internal teams working in parallel, so Accumulation might be the roadmap for a different team. I just have no experience with that. Reading docs it looks fairly advanced, so no interest to me until I am convinced the basics are rock solid and my skills improve.)
Thanks for the reply.
There are 2 features that are very close: dxr that needs directX and nvidia, and accumulation without any special hardware requirements, that allows for very slow but pretty rendering. Both are supported for HDRP.
I did not, although i knew about for a while and i only use HDRP for the most part,
I’d say (3), not exactly though, looking at the doc its pretty easy to use, you did all the work already.
But it doesn’t come with HDRP, as in, I don’t just click a few buttons in editor and its on, or add a component that comes with HDRP. Although if/when I need it bad enough, I’ll use it, it’s not a deal breaker, but I probably would’ve already used it if it came with HDRP. I realize my laziness here but this is my honest guess as to why I haven’t used it so far
–
Can’t give feedback on it feature/quality wise as I haven’t used it so far.
I see. From what I understand from the article, it combines physically based motion blur + path tracing and other post processing effects to create high quality renders. Last I checked, I can’t access path tracing (from the volume settings) because I don’t have an NVidia card. For physically based motion blur, i’m already using Deckard Render from the asset store. It also cleans up other post processing effects like SSR and is fast, but doesn’t have path tracing . What I want from Unity is a path tracing solution that supports AMD graphics cards. Considering making the switch to Blender because they have solutions for AMD.
Btw, out of curiosity, I tried out Accumulation with HDRP in the Recorder, set it at 128 samples, and it was very very slow, and the results weren’t that good. Could still see significant aliasing and motion blur was just okay. Deckard Render from the asset store is blazing fast and the results are very high quality. So, maybe you guys could work on the speed of the Accumulation feature
Still interesting feedback to know if people used it or not.
You don’t need an NVIDIA card to use accumulation. It combines path tracing when it’s enabled but it’s not a requirement.
If you have some result you could share feel free to do so. It shouldn’t be this bad with 128 samples. There might be some issue here (Maybe open an other thread).
It’s a feature that’s intended for quality more than real time. Motion blur might be more adapted in that case.
I am using recorder actually for an HDRP project that mean to be a video, but I didn’t try the accumulation method because i didn’t know it exist (until now), but I will give a try asap for sure!
It could have a gorgeous effect on very organic scenes.
We tried accumulation/recorder to render timeline sequences with Path Tracing but found it not too performant compared to manually accumulating samples. We had to tweak a bit of accumulation source to enable manual accumulation while managing the timeline manually but found our solution to be ~4x faster than using Recorder/Accumulation. On our tests, a frame that took 41 sec using the method above took 8.5s manually managing everything. We had to implement the motion blur based on the accumulation technique by manually advancing the timeline at sub -rame speed, but were able to set custom shutter speeds, and on top, we integrated Open Image Denosie to get clean frames, which is also not possible with recorder ATM.,
So to the question:
Lacks Denoising
It’s not performant, at least in our use case.
We still don’t understand where the raw speed of the HDRP is being throttled, but it would be awesome to allow users to manually accumulate samples or get to the root cause. Our project is still unreleased but if UT is interested, I can share some videos and source files.
Unfortunately, it’s the property of the gaming company we work for, plus it was very custom-made for our use case, but I’m super puzzled as to why the Recorder/Accumulation path is much slower than it should (at least for us).
It is the same case with me, it’s very slow with the recorder. + I noticed that the samples don’t accumulate “intelligently” in darkly lit areas. Unity could use some kinda AI powered sampler or something to make it production worthy
Thanks so much for the answer.
Are you using the public HDRP accumulation API or are ticking a timeline manually at a crazy small Dt.
I presume you get the higher performance since you need less samples because you de-noise the resulting images ?
If so, what’s the typical number of samples you need ?
I tried the first approach but it was somehow slow, so I added a flag to avoid the auto clean calls and are moving the timeline ourselves. Our tests are based on the same number of samples (360).
One thing we found, which might be related (we have not tested recently, was that path tracing was actually capped when vSync was active to 60 samples per second. I understand that vsync should cap FPS, but seems to be capping Path tracing sampling since a frame is in effect a sample. Not sure if this was s=affecting our tests with Recorder, back then, but the main reason we walked away from recorder/accumulation was the fact we needed to create clients that could actually process the frames for us in runtime, and Recorder is Editor bound.
Hi. Wanted to share that after querying the premium support team, we found the culprit why accumulation was slower than the manual override we added to the HDRPsource ourselves. Apparently, there is a property called Time.captureDeltaTime that is set to 0 by default which makes the physics system go nuts generating significant load on the CPU.
I set the variable using the reciprocal property Time.captureFramerate and now the accumulation API runs as fast as my manual method, although it limits a bit on what we do, as it can only move forward by calling the renderPipeline.PrepareNewSubFrame(); method. Having a manual accumulation option allows devs to decide what to draw on every sample, for example in our use case, move manually across a timeline. This freedom allows us, for example, to calculate the motion blur as nth of a second and even use blur lengths over that of a frame if we wanted by setting for example 1/12s on a 24fps clip.
For anyone wanting to give this route a go, to achieve manual accumulation we simply added a flag in the PathTracing.cs file as follows:
public bool ManualAccumulation = false;
TextureHandle RenderPathTracing(RenderGraph renderGraph, HDCamera hdCamera, TextureHandle colorBuffer)
{
...
if (!m_SubFrameManager.isRecording && !ManualAccumulation)
{
// Check if things have changed and if we need to restart the accumulation
camData = CheckDirtiness(hdCamera, camID, camData);
// If we are recording, the max iteration is set/overridden by the subframe manager, otherwise we read it from the path tracing volume
m_SubFrameManager.subFrameCount = (uint)m_PathTracingSettings.maximumSamples.value;
}
...
}
To use it, we simply set renderPipeline.ManualAccumulation = true; to start accumulation and renderPipeline.ManualAccumulation = false; when we are done with the frame, which essentially is triggered by renderPipeline.IsFrameCompleted.
It would be really wonderful to have such flag available within the official version versions.
We use captureDeltaTime extensively in the recorder. This ensures that the engine is ticked in “frames” rather than forcing it to run at unpredictable framerates. The fact that setting it to 0 makes physics go haywire is odd, especially since that is the default unity way of running.
