custom motion vector rendering in 2018

Unity Engine
,
1

hi guys

was wondering is there any way i can custom render motion vector in my own pass?
i was thinking about stay away the camera pre-depth pass, and use a color buffer and depth buffer for normal rendering, then use the depth buffer to carry on with the custom motion vector pass.

i tried shader replacement for this custom motion vector pass, its not working, the essential per object data like “_PreviousM” is not sent, when doing shader replacement pass.
has anyone implement custom motion vector like using shader replacement before? in 2018

another way of doing it, i guess it could be using command buffer. but i m not sure if unity has something that i can control to make this custom motion vector pass work. if anyone has tried this way before in legacy pipeline, plz do enlighten me a bit

the standard way of getting motion vector rendered in legacy pipeline is fine, putting aside i need to update all shaders to conform to that, its that you have to make depth + motion vector mode to the camera, which will trigger the infamous pre-depth pass

2

have you checked keijiros repos, if anything related to this?

https://github.com/keijiro/KvantSprayMV

theres probably more also.

3

so i set up a bit with command buffer way, the essential data (of course) is not presented, so my only hope is to replicate what unity send when doing MotionVector pass, in a material property block(maybe?) , essentially, send those _PreviousM and the like my self ??

7964574--1021032--upload_2022-3-15_18-19-54.png
7964574--1021032--upload_2022-3-15_18-19-54.png2309×801 124 KB

if that is the only way, fine , i will have to keep track all the renderers then, but what are these 2? especially what is “_MotionVectorDepthBias” ? where do get this param from? Renderer ?
7964574--1021035--upload_2022-3-15_18-24-43.png

4

thx 4 reply, didnt see your reply when i make my last reply post yesterday, i will check out those repos later.

actually upon carrying on the code yesterday, i pass the data through propertyblock, and made non-skin renderer work (the RG output are the same just like the standard unity motion vector output).

but not really working with skin mesh renderer, must be something else needed to be sent for the skin renderer

the code is something like this :

        _cam.RemoveCommandBuffer(CameraEvent.AfterForwardOpaque, _cb1);
        _cb1.Clear();
        _cb1.SetGlobalMatrix("_NonJitteredVP", no_jitter_vp);
        _cb1.SetGlobalMatrix("_PreviousVP", vp_matt_prev);
        _cb1.SetGlobalTexture("_TbCamDepthBuffer", _DepthBuffer);
        _cb1.Blit(null, _MotionBuffer, _motVecMatt, 0);
        _cb1.SetRenderTarget(_MotionBuffer, _DepthBuffer);
        //_cb1.ClearRenderTarget(false, false, Color.black);
        var count = _MotVecObjs.Length;
        for (var i = 0; i < count; i++)
        {
            var rend = _MotVecObjs[i];
            if (rend == null) continue;
            var matts = rend.sharedMaterials;
            if (matts == null) continue;

            var isSkin = rend is SkinnedMeshRenderer;
            var matt_count = matts.Length;
            for (var j = 0; j < matt_count; j++)
            {
                var eaMatt = matts[j];
                if (eaMatt == null) continue;

                var usingPassName = isSkin ? "TbMotionVectors" : "TbMotionVectors_Camera";

                //TbMotionVectors
                var passIndex = eaMatt.FindPass("TbMotionVectors");
                if (passIndex < 0) {
                    UnityEngine.Debug.LogWarning("Cannot Find Pass Name : TbMotionVectors");
                    continue;
                }
                //eaMatt.SetShaderPassEnabled("TbMotionVectors", false);

                var trans = rend.transform;

                PerObjectData_TbMotVec perObjData;
                if (_perObjData_keeper.ContainsKey(rend))
                    perObjData = _perObjData_keeper[rend];
                else {
                    perObjData = PerObjectData_TbMotVec.Default();
                    //_perObjData_keeper.Add(rend, perObjData);
                }

                MaterialPropertyBlock mb = new MaterialPropertyBlock();
                mb.SetFloat("_ForceNoMotion", rend.motionVectorGenerationMode == MotionVectorGenerationMode.ForceNoMotion ? 1.0f : 0.0f);
                mb.SetFloat("_HasLastPositionData", perObjData._HasLastPositionData ? 1.0f : 0.0f);
                mb.SetFloat("_MotionVectorDepthBias", perObjData._MotionVectorDepthBias);
                mb.SetMatrix("_PreviousM", perObjData._PreviousM);
                rend.SetPropertyBlock(mb);

                //eaMatt.SetFloat("_ForceNoMotion", rend.motionVectorGenerationMode == MotionVectorGenerationMode.ForceNoMotion ? 1.0f : 0.0f);
                //eaMatt.SetFloat("_HasLastPositionData", perObjData._HasLastPositionData ? 1.0f : 0.0f);
                //eaMatt.SetFloat("_MotionVectorDepthBias", perObjData._MotionVectorDepthBias);
                //eaMatt.SetMatrix("_PreviousM", perObjData._PreviousM);

                _cb1.DrawRenderer(rend, eaMatt, 0, passIndex);
                //eaMatt.SetShaderPassEnabled("TbMotionVectors", false);

                perObjData._PreviousM = trans.localToWorldMatrix;
                perObjData._HasLastPositionData = false;
                _perObjData_keeper[rend] = perObjData;
            }
        }

question remains :
what are [_HasLastPositionData] , [_MotionVectorDepthBias] ?
7967157--1021665--upload_2022-3-16_12-9-39.png

and [oldPos] ?
7967157--1021668--upload_2022-3-16_12-10-20.png

5

hmm, i figured the above process could never get the skin mesh working, so i came up with a work around.
since the infamous camera pre-depth pass was all that i dont want to have, i adjust the process like the following:
1.ensure all the shader in the scene does not have “ShadowCaster” pass (near all of my shaders meets this requirement already)
2. set main camera depth mode → depth | motionVec
3. set custom color and depth buffer to the main camera
4. set global texture : depth buffer as “_CameraDepthTexture” ,
5. let the unity do its motion vector pass
6. sample mot vec tex

well, its almost 100 % working
here is the comparison:(normalized)

7967508--1021755--MotVec_Compare_0001.png
7967508--1021755--MotVec_Compare_0001.png1629×613 71.3 KB

raw motion vector compare

7967508--1021758--MotVec_Compare_0002.png
7967508--1021758--MotVec_Compare_0002.png581×662 11.2 KB

looking at the frame debuger, the data looks correct, how come when in the work around process, the skin renderer still produces result different from the unity standard output?

any thoughts, guys ?

6

frame debug compare

7967559--1021785--MotVec_Compare_0003.png
7967559--1021785--MotVec_Compare_0003.png2279×828 114 KB

7

man, its becoz of this

7967649--1021815--upload_2022-3-16_18-2-34.png
7967649--1021815--upload_2022-3-16_18-2-34.png764×21 1.69 KB

somehow, my habit made me changed it from 4 to 0 when i copied it from the internal shader

now the work around is 100% like the standard pipeline

8

Hi! Did you try to do it before RenderForwardOpaque with additional depth pass?