Accessing TextureMatrix in a surface vertex shader

Unity Engine
1

Hi all,

Been having some fun and games trying to generate uv’s in the vertex shader (of a surface shader) for use with ‘Texture Fetch’, as I couldn’t find any relevant examples online or in the builtin shaders.

I’ve finally got it working including the textures scale and offset values, but can only get it to work with UNITY_MATRIX_TEXTURE0. Try as I might I cannot get it to give me the TextureMatrix of the second texture in my material!

This is the simplest way to get TEXTURE0 is using the built in method TRANSFORM_UV(id);

half2 newTC = TRANSFORM_UV(0);
float nativeHeight = tex2Dlod( _HeightMap, half4 (newTC.xy, 0, 0)).r;

This works, but if I replace TRANSFORM_UV(0) with TRANSFORM_UV(1) it doesn’t, looks like the result back is (0,0)
As an aside, as far as I can tell this will only work with first uv co-ordinate set, as v.texcoord is hardcoded, where as you’d need v.texcoord1 for a second set of uv’s. Though thats not an issue in this case and the long version of the code below should fix that.

If I do it the long way

half4  newTC = half4(v.texcoord.x, v.texcoord.y, 0, 1);
newTC = mul( UNITY_MATRIX_TEXTURE1, newTC);
float nativeHeight = tex2Dlod( _HeightMap, half4 (newTC.xy, 0, 0)).r;

Again it works for UNITY_MATRIX_TEXTURE0, but not for UNITY_MATRIX_TEXTURE1, my vertices in the mesh remain flat and not offset via the heightmap as though the returned uv = (0,0) everytime.

I’ve tried adding a second set of uv’s (though don’t see why they should be needed) and adding uv_HeightMap to my input struct. I’ve double checked that both my textures have different scales and offsets and have even added the heightmap to my base map in the pixel shader to see that there they have different texture matrices and that the code works. e.g

half4 c = tex2D (_MainTex, IN.uv_MainTex);
half4 h = tex2D (_HeightMap, IN.uv_HeightMap);

So I guess I must be missing something?

Here is the full vertex shader, don’t forget this is in a surface shader, so trying to use _HeightMap_ST wont work as it gets redefined by unity when it compiles the shader.

void VertexFetchFunct (inout appdata_full v)
{		
	// Get UV - Simple version use _MainTex uv, no scale or offset matrix applied
	// float nativeHeight = tex2Dlod( _HeightMap, half4(v.texcoord.xy, 0, 0)).r ;
				
	// Get UV - TextureMatrix version - Caution: only works with v.texcoord not v.texcoord1
	half2 newTC = TRANSFORM_UV(1);
	float nativeHeight = tex2Dlod( _HeightMap, half4 (newTC.xy, 0, 0)).r;
				
				
	//	half4  newTC = half4(v.texcoord.x, v.texcoord.y, 0, 1);
	//	newTC = mul( UNITY_MATRIX_TEXTURE1, newTC);
	//	float nativeHeight = tex2Dlod( _HeightMap, half4 (newTC.xy, 0, 0)).r;			
		
	// Original vertex.y is either 1 or 0 to denote front or back position
	v.vertex.y = v.vertex.y*(nativeHeight*_HeightMod);			 
}

Neither the uncomment nor the commented version of the code will give me the first set of uv’s * TextureMatrix for second texture in my material.

Edit:
Something I thought a bit strange when debugging this is that I found

v2f_img vert_img( appdata_img v )

which is meant to be used for image effect shaders according to the comment in UnityCG.cginc only applys textureMatrix scaling to uv’s and not the offset. Since I copied the

MultiplyUV( UNITY_MATRIX_TEXTURE0, v.texcoord );

originally for my tests it took me ages to work out why offset was not being included.

Not sure if that has any relevance to image effects? Would you expect to be able to translate a texture?
if so is this a bug in UnityCG.cginc?

2

Still playing around with this, but still no luck.

I was able to get the following code to compile

half2 newTC = TRANSFORM_TEX(v.texcoord, _HeightMap);		
float nativeHeight = tex2Dlod( _HeightMap, half4 (newTC.xy, 0, 0)).r;

but still doesn’t give me the uv scaled offset.

The code only works as long as I don’t try

half4 h = tex2D (_HeightMap, IN.uv_HeightMap);

as then I get the _HeightMap_ST redefine error.

Edit:
Just had a thought I can obviously work around this ( I think) by simply swapping my texture order round, so the heightmap is in the first texture unit and the base image in the second. Since the pixel shader can get the correct uv for the second texture unit and i can get the correct uv for the first texture unit in the vertex shader. Bit of pain as it means the base texture swaps position compared to every other shader though.

Still i’d like to know what i’m doing wrong and why I can’t get the right uv for the second texture unit in the vertex shader.

edit 2: Follow up
Still no joy getting this to work, but also noticed that shadows completely fail if I use texture matrix to scale/offset the heightmap texture using vertex fetch. The vertices are repositioned, but he shadow acts as though the texture was using no scale or offset.

3

it’s 5 am and i can’t think straight about the rest of your post, but thought you might want to know if you use the #pragma addshadow command in a surface shader you should get your shadows back, as in #pragma surface surf Lambert vertex:vert addshadow alpha

4

Errrrr … I shouldn’t comment on this since I can only make guesses about surface shaders, but what exactly do you need the texture matrix for? Why do you assume that Unity sets more than the 0th texture matrix?

Shouldn’t it be sufficient to use _HeightMap_ST for scaling and offsetting? In GLSL I would write something like:

gl_FragColor = texture2D(_HeightMap, _HeightMap_ST.xy * textureCoordinates.xy + _HeightMap_ST.zw));

5

Been a while since I posted this so had to re-read it, but the problem isn’t that I don’t have shadows, but that when using the texture matrix (or rather offset and scale ) to manipulate the vertices in a vertex program via vertexFetch (i.e changing the vertices based on data from a texture) the shadow no longer confirms. Instead it stays at the default position, i.e. as if offset and scale were default values.

6

Well I’d expect a texture matrix for each texture in order to be able to control the scaling and offsets per texture.

I can’t use _HeightMap_ST because if i define that in the shader the compile complains about redefine since Unity defines that when it parses the code. If I don’t define it then the compiler fails, presumably because its not be defined yet by Unity.

Don’t forget the problem this post describes is that I can’t get the second texture matrix (offset scale) in the vertex program, not the fragment program.

7

Hmmm, are textures numbered? How do you know which texture matrix would correspond to which texture? I can see how you would match texture matrices with texture coordinate attributes but I’m not sure how to match texture matrices with textures.

Does Unity support texture fetches in vertex programs by now?

8

Interesting question, I don’t think there is explicit numbering, but i’d expect some mechanism to be in place to link/bind a texture with its respective texture matrix. Now you mention it, this is an area i’ve not really thought about and just assumed that texture matrix is automatically bound or accessible in a shader with its respective texture.

However conversely it makes no sense to give every texture in a material a texture matrix in the editor if a material/shader can only ever use the first? If it isn’t per texture then it should be per material and thus a member of a material rather than texture.

As I said I posted this some time ago and have had to move on from R&D since I have a client project to finish so my recollection is a bit hazy now. I think from my tests the main problem was accessing the offset/scale (what I refer to as a texture matrix i guess from opengl days) in the vertex program and only getting access to the first texture values. I don’t remember having any issues in the past getting the matrix data for multiple textures in a fragment program or if I ever needed to.

My gut feeling was bar a mistake on my end, that perhaps it was an oversight in Unity ShaderLab.

Yep got added around Unity 3.2 I think, but it does need ShaderModel 3 declared. Great fun to play with though.

9

AFter having to re-read my original post I realise that its not particualry well written to describe the problem, so I’ll have another go.

I’m using vertexFetch to manipulate vertices in the vertex program of a shader. I have a material with two textures, the first is a diffuse image, the second is essentially a heightmap, used to modify the height of each vertex.

This works, the problem comes when I wanted to offset or scale the uv by second textures matrix(i.e offset or scale properties). I can use the diffuse textures offset or scale to manipulate the displayed results (e.g. scrolling the texture), but trying to do the same with the heightmap offset/scale fails, specifically in the vertex program.

I was hoping it was a silly error on my part, but the lack of replies or rather ‘answers’ suggests perhaps not. I think the best thing now would be when i get some free time I make a simple example shader that others can try and see first hand the issue.

10

hi,
try this:
half2 _scale={UNITY_MATRIX_TEXTURE1[0].x,UNITY_MATRIX_TEXTURE1[1].y}; for tiling
half2 _offset={UNITY_MATRIX_TEXTURE1[0].w,UNITY_MATRIX_TEXTURE1[1].z}; for offset
!! the Numbering of UNITY_MATRIX_TEXTURE follows the order of declaration of the sampler2D in surfaceFunction !!

and then:
half3 _displace= tex2D (_DisplaceMap, v.texcoord.xy * _scale + _offset).rgb;
v.normal += _displace * _DisplaceAmount/4;
v.vertex.xyz += v.normal*_displace * _DisplaceAmount;

i must compile with #pragma only_renderers d3d9 #pragma target 3.0

hope it helps

11

Thanks. I don’t have the time to try it out at the moment, but hopefully I’ll get an opportunity to have a look over the next few days.

12

is this an attempt to extrude a mesh based on incoming kinect rgb/depth data, and then to align the rgb image in some way?

13

Yes and No :wink:
I’d been wanting to mess around with vertex fetch for a while and whilst I did try it with the kinect depth information the results weren’t as good or as ‘tweakable’ as simply building a dedicated mesh and extruding the vertices each frame.

The actual shader discussed here is the basis of doing this
https://www.youtube.com/watch?v=HhAhCngoPHA

14

i like this. I’d be interesting in hearing more about your approach