Hi there!
Problem
I am trying to make an effect where players paint on paintable objects by placing a painter object in between their camera view and the surface. In the screenshot below, the sphere (painter object) should paint a green circle on the quad surface behind it (the paintable object).
First Steps
I have tried to implement this in the High Definition Rendering Pipeline with a custom pass. Currently, I am first painting all paintable objects in the red channel of a custom render texture. Then I paint all painters in the blue channel on top. That gives me a texture where I can read all overlaps, and thus know where, in screen space, to paint the green. The result can be seen in the image below:
Texture Painting
The next step, which is what I’m having trouble with, is to translate this screen space into pixels on the object’s texture (the quad, in the above case). My idea was to loop through all paintable objects and run a custom shader on them, which would use its own texture as render target, and then read data from the overlap texture to know where to draw. I tried lots of different transformations but could never get anything to work. Then I tried just setting the output color to red in the fragment shader, and I realized that this will never work. The red does not even cover the entire quad. Rather, the part in the quad that is red is the same (barring perspective I guess) relative to how the quad itself covers the screen. It can be seen in the “game view” image above.
So I think what is happening is that the fragment shader will ask for the pixel color of the pixels from the camera view that is covered by the quad, and nothing else. What I really would need is sort of the other way around; Decide where to write on the output texture depending on the input.
In the case of the quad, I think I could maybe make some transformation in the vertex shader to fix this (not sure exactly what, though), but I don’t think it would work in the general case. Most objects won’t have such nicely mapped UVs, so then I don’t think translating vertexes is enough.
Compute Shader
I started off with trying to make a compute shader instead. Since I am writing information about which objects are overlapping in the overlap texture (the red color differs depending on array index), I could run through all the pixels in the overlap texture, then write to the correct texture from there. It should also be faster since it makes better use of already calculated data. In theory, at least. In practice, I would need to pass in all textures at once, which may or may not be a problem. But I also need the map between object space and texture coordinates (the UV map?), which I have no idea how to pass in. So I feel kind of stuck here as well.
Questions
I am not exactly sure how to proceed from here (and not entirely sure what questions to ask either), but I would appreciate some help. 
- Would it be possible to do this without compute shaders?
- How to map screen space to texture pixels, in either solution?
- Is it feasible to load dozens or hundreds of textures (not sure what resolution, but I don’t want the paint to look blurry) into a compute shader at a time?
- How could I get the UV mapping of objects into the compute shader?
- Is there some other better solution to all of this?
Thanks in advance!
Code
For reference, below is the code for the custom pass currently.
using System.Collections.Generic;
using System;
using UnityEngine.SceneManagement;
namespace UnityEngine.Rendering.HighDefinition
{
[Serializable]
public class PaintPass : CustomPass
{
static readonly int OBJECT_ID = Shader.PropertyToID("_ObjectId");
//TODO: Make nicer.
const int PAINTABLE_LAYER = 10;
const int PAINTER_LAYER = 11;
Material _PaintableMaterial = null;
Material _PainterMaterial = null;
Material _TexPaintMaterial = null;
RTHandle _IdTexture;
RTHandle _OverlapTexture;
List<Renderer> _Renderers = new List<Renderer>();
protected override void Setup(ScriptableRenderContext renderContext, CommandBuffer cmd)
{
base.Setup(renderContext, cmd);
AssignObjectIDs();
_PaintableMaterial = new Material(Shader.Find("Shader Graphs/Mark"));
_PainterMaterial = new Material(Shader.Find("Shader Graphs/Painters"));
_TexPaintMaterial = new Material(Shader.Find("Shader Graphs/TexPaint"));
_IdTexture = RTHandles.Alloc(Vector2.one, colorFormat: Experimental.Rendering.GraphicsFormat.R8G8B8A8_SRGB);
_OverlapTexture = RTHandles.Alloc(Vector2.one, colorFormat: Experimental.Rendering.GraphicsFormat.R8G8B8A8_SRGB);
}
protected override void Cleanup()
{
RTHandles.Release(_IdTexture);
RTHandles.Release(_OverlapTexture);
}
protected override void Execute(CustomPassContext ctx)
{
LayerMask mask = new LayerMask();
ShaderTagId[] shaderTags = new ShaderTagId[]
{
HDShaderPassNames.s_ForwardName, // HD Lit shader
HDShaderPassNames.s_ForwardOnlyName, // HD Unlit shader
HDShaderPassNames.s_SRPDefaultUnlitName, // Cross SRP Unlit shader
HDShaderPassNames.s_EmptyName, // Add an empty slot for the override material
};
mask.value = 1 << PAINTABLE_LAYER;
PerObjectData renderConfig = HDUtils.GetRendererConfiguration(ctx.hdCamera.frameSettings.IsEnabled(FrameSettingsField.ProbeVolume), ctx.hdCamera.frameSettings.IsEnabled(FrameSettingsField.Shadowmask));
RenderStateBlock paintableDepthState = new RenderStateBlock(RenderStateMask.Depth)
{
depthState = new DepthState(false, CompareFunction.Equal),
};
RendererUtils.RendererListDesc renderDesc = new RendererUtils.RendererListDesc(shaderTags, ctx.cullingResults, ctx.hdCamera.camera)
{
rendererConfiguration = renderConfig,
renderQueueRange = GetRenderQueueRange(RenderQueueType.All),
sortingCriteria = SortingCriteria.CommonOpaque,
excludeObjectMotionVectors = false,
overrideMaterial = _PaintableMaterial,
overrideMaterialPassIndex = _PaintableMaterial.FindPass("ForwardOnly"),
stateBlock = paintableDepthState,
layerMask = mask,
};
RendererList rendererList = ctx.renderContext.CreateRendererList(renderDesc);
CoreUtils.SetRenderTarget(ctx.cmd, _IdTexture, ClearFlag.Color);
CoreUtils.DrawRendererList(ctx.renderContext, ctx.cmd, rendererList);
RenderStateBlock painterDepthState = new RenderStateBlock(RenderStateMask.Depth)
{
depthState = new DepthState(false, CompareFunction.LessEqual),
};
renderDesc.overrideMaterial = _PainterMaterial;
mask.value = 1 << PAINTER_LAYER;
renderDesc.layerMask = mask;
renderDesc.stateBlock = painterDepthState;
_PainterMaterial.SetTexture("_IdTexture", _IdTexture);
rendererList = ctx.renderContext.CreateRendererList(renderDesc);
CoreUtils.SetRenderTarget(ctx.cmd, _OverlapTexture, ClearFlag.Color);
CoreUtils.DrawRendererList(ctx.renderContext, ctx.cmd, rendererList);
_TexPaintMaterial.SetTexture("_OverlapTexture", _OverlapTexture);
_TexPaintMaterial.SetColor("_PaintColor", Color.green);
foreach (Renderer renderer in _Renderers)
{
if (!HasPaintPassMaterial(renderer) || !renderer.enabled || !renderer.gameObject.activeSelf)
continue;
Texture targetTexture = FindTexPaintTexture(renderer);
CoreUtils.SetRenderTarget(ctx.cmd, targetTexture, ClearFlag.Color);
ctx.cmd.DrawRenderer(renderer, _TexPaintMaterial, 0, 6);
}
}
Texture FindTexPaintTexture(Renderer renderer)
{
return renderer.sharedMaterials[renderer.sharedMaterials.Length - 1].GetTexture("_MainTex");
}
public void AssignObjectIDs()
{
int sceneCount = SceneManager.sceneCount;
for (var i = 0; i < sceneCount; i++)
{
Scene scene = SceneManager.GetSceneAt(i);
if (!scene.IsValid() || !scene.isLoaded) continue;
GameObject[] rootGameObjects = scene.GetRootGameObjects();
foreach (GameObject rootGameObject in rootGameObjects)
{
foreach (Renderer renderer in rootGameObject.GetComponentsInChildren<Renderer>())
{
int layer = renderer.gameObject.layer;
if (layer == PAINTER_LAYER || layer == PAINTABLE_LAYER)
_Renderers.Add(renderer);
}
}
}
int renderListCount = _Renderers.Count;
for (int i = 0; i < renderListCount; i++)
{
Renderer renderer = _Renderers[i];
MaterialPropertyBlock propertyBlock = new MaterialPropertyBlock();
propertyBlock.SetInteger(OBJECT_ID, i);
renderer.SetPropertyBlock(propertyBlock);
if (renderer.gameObject.layer != PAINTABLE_LAYER || HasPaintPassMaterial(renderer) || !Application.isPlaying)
continue;
Texture rendererMainTex = renderer.material.mainTexture;
int width = rendererMainTex != null ? rendererMainTex.width : 1024;
int height = rendererMainTex != null ? rendererMainTex.height : 1024;
RTHandle paintTex = RTHandles.Alloc(width, height,
colorFormat: Experimental.Rendering.GraphicsFormat.R8G8B8A8_SRGB);
Material paintMaterial = new Material(Shader.Find("Shader Graphs/PaintedSurface"));
paintMaterial.SetTexture("_MainTex", paintTex);
paintMaterial.name = "PaintPass";
List<Material> materials = new List<Material>();
renderer.GetSharedMaterials(materials);
materials.Add(paintMaterial);
renderer.SetSharedMaterials(materials);
}
}
bool HasPaintPassMaterial(Renderer renderer)
{
foreach (Material material in renderer.sharedMaterials)
{
if (material.name.StartsWith("PaintPass"))
return true;
}
return false;
}
}
}