Webcam based heightmap performance issues

Performance & Profiling
1

Hi all,

I made a array of cubes which derrive their scale and color based on webcam input. It works, but performance is pretty bad right now, 4fps on my i7 Macbook pro when I use 160x120 cubes… What would be the best way to increase performance? I wouldn’t mind so much if the scaling and color of the cubes are updated less frequent (for example 1 time per second), but I don’t understand how to achieve this. Also changing the amount of cubes to 80x60 would be an option, but since the minimum resolution of the webcam is 160x120 I’m having some trouble with this too…
This is the curent code:

var webcamTexture : WebCamTexture;
var data : Color32[];

var amountX= 160; //this is a variable for the amount of cubes horizontally
var amountZ= 120; //this is a variable for the amount of cubes vertically
private var cubeArray= new GameObject[amountX*amountZ];
private var cube : GameObject;

function Start () {
// Start web cam feed
webcamTexture = WebCamTexture(160, 120, 10);
webcamTexture.Play();
data = new Color32[webcamTexture.width * webcamTexture.height];


var index=0;
for (var z = 0; z <amountZ; z++) { 
        for (var x = 0; x < amountX; x++) { 
            cubeArray[index] = GameObject.CreatePrimitive(PrimitiveType.Cube); 
            cubeArray[index].transform.position = Vector3 (x, 0, z+0.5); 
         index++;   
        }
    }
}

function Update () {

webcamTexture.GetPixels32 (data);

for (var i=0;i<amountX*amountZ; i++){
var temp: Color = data[i];
cubeArray[i].transform.renderer.material.color = data[i];
cubeArray[i].transform.localScale = Vector3(1,temp.grayscale*40,1);
	}
}

Hope somebody can help me get further!

2

Maybe I am overdoing things here but…

1) You could combine all your cubes into one mesh, give each cube an unique bone index (Unity - Scripting API: Mesh.boneWeights) and create 160x120 bones (when this is technically possible)

Now you can set the transformation by the bones and you dont have the performance overhead by managing 160x120 gameobjects and components.

2) You can create a custom vertex shader which reads directly from your webCamTexture and uses the pixel data to transform the vertexdata.

You could combine this with the first part of the 1) solution and instead of bones, you set some other per vertex value which you can use inside the vertex shader.

3

Hi Marrrk,

Interesting idea, curious how 160x120 would perform compared to 160x120 objects. I would like to get the initial approach to work also simply for learning purposes though. I guess 160x120 cubes might be overkill, 80x60 might be a better start. Problem is I can’t sync the 160x120 pixels the webcam is streaming to the 80x60 pixels I am generating. Any idea how I could solve this?
I also guess that if the transformations aren’t happening continuously but only 1 time a second would make a big difference, but I don’t understand how to pause the updates…

4

To perform the update you can use Coroutines (Unity - Scripting API: YieldInstruction).

To use a 160x12px texture when you only need 80x60, the simplest solution is to sample only half the size, or sample 2 pixel for one sample:

Pseudocode:

Color GetValue(int x, int y)
{
  return texture.GetData(x * 2, y * 2);
}
5

Hi Marrrk,

Thanks for the help! I tried coroutines before, but get this error: Update() can not be a coroutine
Probably I’m using it the wrong way (my programming skills are not great)…
This is what I do:

var webcamTexture : WebCamTexture;
var data : Color32[];



var amountX= 160; //this is a variable for the amount of cubes horizontally
var amountZ= 120; //this is a variable for the amount of cubes vertically
private var cubeArray= new GameObject[amountX*amountZ];
private var cube : GameObject;

function Start () {
// Start web cam feed
webcamTexture = WebCamTexture(160, 120, 10);
webcamTexture.Play();
data = new Color32[webcamTexture.width * webcamTexture.height];


var index=0;
for (var z = 0; z <amountZ; z++) { 
        for (var x = 0; x < amountX; x++) { 
            cubeArray[index] = GameObject.CreatePrimitive(PrimitiveType.Cube); 
            cubeArray[index].transform.position = Vector3 (x-40, 0, z-30); 
         index++;   
        }
    }
}

function Update () {

webcamTexture.GetPixels32 (data);

for (var i=0;i<amountX*amountZ; i++){
var temp: Color = data[i];
cubeArray[i].transform.renderer.material.color = data[i];
cubeArray[i].transform.localScale = Vector3(1,temp.grayscale*40,1);
yield PauseScript();
	}
}

function PauseScript () {
// suspend execution for 5 seconds
yield WaitForSeconds (5);
}

Thanks!

6

Regarding the coroutines, I think this will help you very much:

7

Hi Marrrk,

Thanks for the link, I also found it when I was trying the coroutines before. I still don’t get how I can implement the given example in my script though… Probably I should take a break :).