In-game video monitor without render-to-texture

Unity Engine
1

Ahoy there!

I’m having problems creating an in-game video monitor that renders the view from an in game camera. It’s the need to make a custom projection matrix to fit the output of a camera into that screen which has me beat. The screen of the in-game monitor is rendered with a kind of “mask” shader I guess you’d call it, which cuts a hole in the geometry and allows a camera set to a different depth to be rendered on that screen alone, behind the level geometry. That’s all working fine, but I still need to apply some sort of perspective transformation to the feed from the in-game video camera.

I’ve been messing around with the individual values of the video camera’s projection matrix, and can manually adjust the values so that the feed from the video camera distorts properly and conforms to the in-game video screen, but I can’t figure out how to have the projection matrix update automatically as the first-person camera moves around to view the screen from different angles. From this experimentation I see no reason why it wouldn’t be possible, it’s just that using the projection matrix to do it seems pretty difficult.

Sadly, I don’t own Unity PRO, and can’t use a render texture. I’ve looked for Unity code examples and have been googling Matrix math relating to camera projections, but haven’t found anything.

So… does anyone know if this has been done before, or if it really is possible?

2

you can not project a camera without R2T, cause you would always need to grab it in realtime and copy it over which will cost a lot performance for anything but small sizes (otherwise it won’t end on the model or anything alike. also you can’t define the output rect for cameras, its always screen axis aligned).

3

Helix, it is possible, but, as I’m sure you guessed, there is some serious maths involved.

4

what happens if you just multiply by the transformation matrix to get from main camera coords to screen coords.

5

Thanks for the quick responses, guys!

@ dreamora: I’m not quite sure what you mean by “grab it in realtime and copy it over”, but I possibly didn’t explain myself well enough to begin with. I wouldn’t need to change the camera rect at all, it stays at the default [0,0,1,1]. I’d be projecting the output from the camera to the screen by monkeying around with the projection matrix. You can distort, translate, rotate, scale and skew the output from the camera, just like using “free transform” command in photoshop, only alot less intuitive.

@ pakfront: Yep, this kind of maths terrifies me, hence my begging for help. :wink:

@ ivkoni: I tried your suggestion out today, but it gave fairly bizzare results. Admittedly, I’m not sure I’m doing it right. It seems that two values in the projection matrix of the in-game video camera that skew the perspective, [3,0] and [3,1] are unaffected by any combination of transform matrix multiplication I’ve tried so far.

I’ve made some progress though, I can center the video camera output on the ingame screen, and scale it. It looks fine if viewed from front-on, and the output is always centered on the screen no matter which angle it’s viewed from. I need to figure out how to distort the output properly now. Additionally, it seems the WorldToViewportPoint function gives weird results when the world point is too far off screen, resulting in wierd scaling when any of the monitor corners are off screen.

Anywho, here’s the code to center and scale the camera output to the in-game monitor.

 var tl : Vector3 = GetViewPoints(transform.position+topLeft);
var bl : Vector3 = GetViewPoints(transform.position+bottomLeft);
var tr : Vector3 = GetViewPoints(transform.position+topRight);
var br : Vector3 = GetViewPoints(transform.position+bottomRight);
		
var width : float = Mathf.Abs(((tl.x+bl.x)/2)-((tr.x+br.x)/2));
var height : float = Mathf.Abs(((tl.y+tr.y)/2)-((br.y+bl.y)/2));
		
var p : Matrix4x4 = videoCamera.camera.projectionMatrix;

p.m00 = windowWidth * width;//horizontal scale
p.m11 = windowHeight * height;//vertical scale
p.m02 = -2*(GetViewPoints(transform.position).x-0.5);//horizontal translation
p.m12 = -2*(GetViewPoints(transform.position).y-0.5);//vertical translation

videoCamera.camera.projectionMatrix = p;

It’s a start, but I’m still stuck on how to distort the projection matrix appropriately. Thoughts?

6

After about 3 days, I’m still stuck. It’s prolly a bit rude of me to bump this, so if I can’t get an answer I guess I’ll have to give up for now. So… is there anybody at all who’s skilled enough with tampering with projection matrices to solve this?