Question: How to apply a shader for particular part of image only (like in lens studio)?

Unity Engine
,
1

Hello.

Let’s assume a I have a camera stream (or set of images). This stream is passed to ONNX neural network.
As a sample:

This is input image:

9542080--1347634--upload_2023-12-21_17-29-52.jpeg
9542080--1347634--upload_2023-12-21_17-29-52.jpeg980×1039 125 KB

Output image of neural network:

9542080--1347637--upload_2023-12-21_17-31-21.jpeg
9542080--1347637--upload_2023-12-21_17-31-21.jpeg980×1039 20.4 KB

So, with detection there is no problem.

But how can I apply shader for a particular part of image? My understanding shader is applied to material in Unity, I have to create a dynamic meshes with materials which is overlayed the original image or pass original image to shader and handle positions there.

But I can’t find any sample of such shader. Has anybody seen such shader or probably someone has an idea how lens studio does it?

2

This is not really an AR question. I recommend that you check out the Shaders forum: Unity Engine - Unity Discussions