Image Based Reflection

Unity Engine
1

Hi,

I’m working on an Image Based Reflection system based on the work here :

http://forum.unity3d.com/threads/218139-Billboard-Reflection

Which derive from those paper aswell :

http://udn.epicgames.com/Three/rsrc/Three/DirectX11Rendering/MartinM_GDC11_DX11_presentation.pdf

Currently everything is working fine except the fact that I can’t use more than one object to reflect ( and a screenPos problem I have ).

As I can’t pass any array from a script to my shader I really don’t know how I could sent more data to the shader to use more than one object.

I thought about using it as a screen space effect but then I’m stuck to the same problem, how can I pass more than one object postion to the shader.

Any help would be much appreciated.

This is the script I use to pass my data :

using UnityEngine;
using System.Collections;

[ExecuteInEditMode]
public class ImageReflection : MonoBehaviour 
{
	
	//IMAGE REFLECTION
	public float culling;
	public GameObject vertice;
	public Texture2D imageReflectionJitter;
	public Texture imageReflection;
	public MovieTexture[] movieTexture;
	public bool hideDuringPlay = true;
	
  	protected Vector3 m_right = new Vector3(1, 0, 0);
 	protected Vector3 m_up = new Vector3(0, 1, 0);
	
	private Vector3 quadLLPos;
	private Vector3 quadX;
	private Vector3 quadY;
	private Vector2 quadScale;
	
  	protected Material m_imageReflectionMaterial;

	void Start()
	{
		this.gameObject.renderer.enabled = true;

		gameObject.renderer.sharedMaterial = new Material(Shader.Find("Infinite Realities/IR ImageReflectionProxy"));
	}


	void goVariable()
	{
		
		Shader.SetGlobalVector("_QuadLLPos",quadLLPos);
		Shader.SetGlobalVector("_QuadX",quadX);
		Shader.SetGlobalVector("_QuadY",quadY);
		Shader.SetGlobalVector("_QuadScale",quadScale);
		Shader.SetGlobalTexture("_ImageReflectionJitter",imageReflectionJitter);
		Shader.SetGlobalTexture("_ImageReflection",imageReflection);
		Shader.SetGlobalFloat("_Culling",culling);

	}

	void Update () 
	{
		
		foreach (MovieTexture mov in movieTexture) 
		{
			mov.loop = true;
			mov.Play();
		}
		
		if(hideDuringPlay  Application.isPlaying)
		{
			this.gameObject.renderer.enabled = false;
		}
		
		quadLLPos = vertice.transform.position;
		
		quadX = this.transform.localToWorldMatrix.MultiplyVector(m_right);
    	quadX.Normalize();
		
		quadY = this.transform.localToWorldMatrix.MultiplyVector(m_up);
    	quadY.Normalize();
		
		quadScale = this.transform.localScale;
		goVariable();
	}
}

Thanks.

2

I’m kind of dealing with a similar problem (tring to get an Ambient Occlusion effect).

Either make multiple Variables (Object1pos, Object2pos …)
It’s tedious, but in the end, low end shaders have to unroll any kind of for loop anyway, which means a fixed number of objects anyway.
Just define some “empty” state, so the shader can “ignore” it.

The other thing you can try is doing it with a texture. Write the postion etc data to a texture and sample the texture. (with SetPixel and Apply()). Problem here is, you’ll have to encode Float data into RGB. :frowning:

3

I see :).

Do you think I could use matrix instead to pack them all ?

Like my script make array of all data then send those in some matrix that I can send to shader and the shader take the value when the matrix is filled ?

The texture trick is also interresting but I don’t have any idea how to implement that currently.

4

I haven’t tried it with Matrix, should be possible aswell.
I would probably just go with several variables. I don’t think there’s a speed gain using Matrix instead of several vectors and it would probably just make the less readable.

With a Texture: (pseudo snippet code) JavaScript

var AOareas = new Texture2D(16, 16);   //create a Texture

...
// in Start()
		AOareas.anisoLevel = 1;
		AOareas.filterMode = FilterMode.Point;

// inside Update()
		for(var x = 0; x < AOareas.width; x++)
			for(var y = 0; y < AOareas.height; y++)
				{
				AOareas.SetPixel(x, y, UnitToColor24Sign(YOURFLOATPOSITION.x));	// needs to be etween 0 and 1!
				}
		AOareas.Apply();

// helper function
function UnitToColor24Sign(unit : float) : Color {
		    var col : Vector4;
		    var sign = Mathf.Sign(unit);
		    col = unit * sign * Vector3(1, 255, 65025);
		    col.y = col.y % 1;
		    col.z = col.z % 1;
		    col.x -= col.y * 0.00390625;
		    col.y -= col.z * 0.00390625;
		    col.x = Mathf.Clamp01(col.x);
		    col.y = Mathf.Clamp01(col.y);
		    col.z = Mathf.Clamp01(col.z);
		    col.w = sign;
		    return col;
		}

Then inside the Shader you can Loop over the texture and retrieve the values with:

        inline float ColorToUnit24Sign(in fixed4 col) {
        		float f = dot(col.xyz, float3(1.0, 0.00392156862, 0.0000153787));
        		if(col.w == 0) f *= -1; 	// sign is in w
			    return f;	
			}

This might only work with Pro, not sure.

5

Why don’t you use the depth buffer as a crude scene representation to reflect the whole scene without having to pass in every single object?

6

danybittel : Thanks I will try this out asap.

Dolkar : What do you mean ? Like screen space raytraced reflection ?

If screen space raytraced reflection I already do it, the reason why I want image reflection is that it can compensate the SSRR fail.

7

Yeah that’s what I meant… I think I misunderstood what you are trying to achieve…
So, you’re mixing together screen space reflections, image based reflections and billboard reflections? …or are you trying to implement parallax-corrected IBR, which have little to do with billboard reflections?

Anyways, don’t use textures for that purpose, if you care about performance. I haven’t tried it myself, but you CAN use arrays in your shaders, even in SM3, as long as you access them with constant indices, which unrolled loops use. The only issue is with populating it with your script. In the compiled shader, you can see that the array is converted into single constants, like this:

float _Array[5];
// Compiles into:
float _Array0;
float _Array1;
float _Array2;
float _Array3;
float _Array4;

So, in theory, if you set these variables in your script, it should work :slight_smile: I suppose it’s worth a try.

1 Like
8

Yeah I’m trying to mix them and at the end I’ll have SSRR, ImageBasedReflection (to me billboard reflection is the same but with some BRDF approach) and cube map with parallax correction.

Ok thanks I’ll try this too!

9

In fact what I want to achieve with those reflective quad is what UDK have.

You place an ImageReflectionCapture object where you want to have some reflection and capture the image behind, then the shader reflect it (not the object used to capture) in a physical plausible way.