Draw on Cube/Plane/2D Surface using Setpixel?

[flav3]

I’m working on a project where I use a LeapMotion device, which tracks hands in 3D space. I intend for the hands to hold something like a paint brush, and when the user’s brush ‘collides’ with a 2D surface (but could be a 3D cube that is flat, or a plane), it would draw on the surface of the plane/cube/2D object. I’ve done research and found that i could achieve this through the SetPixel function, but couldn’t find any detailed youtube walkthroughs or example projects.

Could anyone guide me to how i would implement this please? I assume the code should go in the ‘OnTriggerEnter’ method or something similar.

Any help would be appreciated, thanks.

[Malleck666]

Unity - Scripting API: Texture2D.SetPixel

I assume you have consulted the docs?

It is doable through OnTriggerEnter as you will need the point of contact.

[KelsoMRK]

It might be easier to do it via raycasting Unity - Scripting API: Physics.Raycast
Get the texture coordinate of the hit returned by the raycast - Unity - Scripting API: RaycastHit.textureCoord
And then update the texture based on those coordinates (example in the above doc page)

[flav3]

Thank you very much, I will look into this and return with feedback on how I’m doing.

[flav3]

Okay, I have both scripts in the same script, and is attached to the end of my index finger (so where ever my index finger is pointing, it will draw a debug ray and print out what the raycast is hitting and its coordinates in the console).

I made sure convex box was not checked on the collider of the square plane.
I made sure that ‘read/write enabled’ checkbox to ticked of the blank png square image i made from paint.
I dragged the png onto the plane, and overrided the format to RGB 24 bit.

So it’s slightly working the way I want it so thank you! However, it’s very slow to draw, which is illustrated by a few dots appearing every 2 seconds, when i move my finger. What i would prefer is a smooth line like how you would do if drawing on paint with a mouse.

I assume that i would need to use the SetPixels instead of the SetPixel.

I’ve checked the documentation of SetPixels, but I dont understand it, the way i understand SetPixel.

Could someone explain to me how i should implement it? While i’m waiting, i will carry on examining the documentation of SetPixels

[StarManta]

@flav3

However, it’s very slow to draw, which is illustrated by a few dots appearing every 2 seconds, when i move my finger. What i would prefer is a smooth line like how you would do if drawing on paint with a mouse.

I assume that i would need to use the SetPixels instead of the SetPixel.

Yes, you should virtually always use SetPixels and not SetPixel; SetPixel is just torturously slow, and I can’t think of a scenario where you’d want to draw only a single pixel.

That said, I think you’ll be better served not actually using SetPixels. Consider what you want:

1. Speed
2. Lines from point A to point B
3. Probably anti-aliasing on those lines (1-pixel lines just don’t look good)
   SetPixels will get you NONE of these without a lot of work.

Here’s what I would do:

1. Create a render texture, assign it to your paintable thing. Resolution probably 512x512 or higher.
2. Create a new camera, and assign it to render to the render texture. Set it to orthographic, set its position to (0.5, 0.5), and its ortho size to 1. That way, it’ll be rendering anything from (0,0) to (1,0) in world space, to the texture
3. Create a linerenderer. Place it at (0,0) with its Z position in FRONT of your above camera. Create a new layer, and assign the LineRenderer to that layer. Make the camera render only that layer, and make your main game camera NOT render that layer.
4. To draw: Have a script on the LR’s object add each point being drawn (which you get from the texCoord just as before - you want (0,0) to (1,1) ) to a List; use SetPositions(thatList.ToArray() ) on that LineRenderer to apply it

This will be hardware accelerated, beautiful, and super efficient.

[flav3]

I assume the goal is to create a line renderer using a list of points, and that list will be updated by the coordinates where the raycast hits the texture. Task 4 looks very daunting but i’ll give it a go and reply with some feedback, thank you!

[flav3]

I’m on step 2, and i’m not sure if you mean to set the position of the transform to 0.5 and 0.5 for the x and y values, cause when i do it goes from this:

to this:

I’ll leave the transform of the camera as it is, for now, and carry on with the following instructions, but i’d ideally want that blackish square on the render texture to fill the whole plane object.

If it helps you to help me, i set the render texture to 512x512 and here are the console debugs:

Also, im assuming you want me to attach a linerenderer to an empty game object.

For some reason, the line renderer is putting some pink plane at the (0,0) location as shown in the followinig picture, and when i move the camera in scene view, it dynamically changes its z rotation, i’ll just leave it for now and carry on.

Currently trying to figure out how to add points to the vector 2 list…

Thank you for your help so far, and i hope you can get back to me as soon as possible please, thanks.

[hippocoder]

Regarding using Leap to paint on a surface I would avoid all setpixels usage, it’s not required. What I would do instead is create a render texture and use another camera to draw paint via line renderers, sprites, anything you want in 3D space. The main camera would not render these items, only the “texture camera”. So you want to keep your camera layers separate.

For now, remove leap from the equation: you have leap’s 3D position in the Unity scene so it follows your real problem is painting in a 3D scene, this will simplify your development.

You don’t need setpixel for this, so you can keep it all on GPU to begin with, using RenderTexture and another camera for that. This would be fast on any laptop or any device, and probably consume next to no processing.

You can’t solve this satisfactorily with Setpixel.

[flav3]

@StarManta

4. To draw: Have a script on the LR’s object add each point being drawn (which you get from the texCoord just as before - you want (0,0) to (1,1) ) to a List; use SetPositions(thatList.ToArray() ) on that LineRenderer to apply it

I don’t know how to add points gathered from the texCoord and insert it into the linerenderer. This is the original script:

and this is what i’ve tried:

but it throws this error:

What i’m trying to say is, i have run out of ideas on how to do task 4. Can someone please help me?

and thank you hippocoder, i definetly wont be using setpixel/s

[hippocoder]

[flav3]

@hippocoder

Oh sorry, I didn’t know I could do that. I will use that way from now on, and I will edit my post later to show as code as I’m currently away from my computer.

[methos5k]

I think in your screenshot, you haven’t made it so there is a second camera that only renders the linerenderer (and the main camera ignores it)

Let your list collect a few entries (if you want), but you have to initialize the list:
List myList = new List();
myList.Add( /* your thing here */);

note: You can create this list outside of the function, also. and after you apply it, you can do:
myList.Clear(); and simply re-use it.

[flav3]

@methos5k

I think in your screenshot, you haven’t made it so there is a second camera that only renders the linerenderer (and the main camera ignores it)

Let your list collect a few entries (if you want), but you have to initialize the list:
List myList = new List();
myList.Add( /* your thing here */);

note: You can create this list outside of the function, also. and after you apply it, you can do:
myList.Clear(); and simply re-use it.

My main camera is in ‘LMHeadMountedRig’ as a component, and I set the culling mask to everything except the ‘LineRendererLayer’ that I made. Then in the camera that’s in the hierarchy, I have the culling mask of that to only the ‘LineRendererLayer’. I ensured that I assigned the ‘LineRenderer’ to the ‘LineRendererLayer’.

And I saw that you edited your post, what you mentioned before you deleted it, I tried setting the viewport rect to (0.5,0.5) and it multiplied the camera to a quarter of the plane that the renderer texture was assigned to. I have some knowledge of viewport, as I used it when making openGL programs. Thank you though, I really appreciate your reply, I’ll try that list thing when I get home.

[methos5k]

Ya, I edited that because i wasn’t sure that was right… and i dunno did it help or ? lol
sorry I didn’t realize your rig was elsewhere that i couldn’t see. …
and yeah…good luck w/ the list lol

[flav3]

Texture2D tex = rend.material.mainTexture as Texture2D;
        Vector2 pixelUV = hit.textureCoord;
        pixelUV.x *= tex.width;
        pixelUV.y *= tex.height;


        List<Vector2> list = new List<Vector2>();
        list.Add ( new Vector2((int)pixelUV.x,(int)pixelUV.y));


 

        gameObject.GetComponent<LineRenderer> ().SetPositions (list.ToArray);

Note: There should be a ‘()’ at the end of ‘ToArray’, but the editor wont let me change it because it says my change is spam-like or innapropriate? :S So yeah, just pretend there are brackets there.

So the first 4 lines are the original code, and lines 7 and 8, are the ones i added using guidance from this thread.

StarManta said to then use that SetPositions on the LineRenderer, but its throwing the following errors in the console, which points me to line 13 in the code i posted above:

I managed to fix this by converting all the ‘Vector2’ to ‘Vector3’, and then it ran with no error, but still, no lines appeared on the render texture.

---

@StarManta

you want (0,0) to (1,1) ) to a List; use SetPositions(thatList.ToArray() ) on that LineRenderer to apply it

I don’t understand what you mean by (0,0) to (1,1) to a list, i know they are co-ordinates, but could anyone explain this to me please?

This must be a really easy task, i don’t know why this is so hard to accomplish :(. I’m desperate, can anyone help me?

[StarManta]

@flav3

I don’t understand what you mean by (0,0) to (1,1) to a list, i know they are co-ordinates, but could anyone explain this to me please?

So, since this comment thread I’ve turned this advice into an article with a little bit more detail on the setup of the camera and such.

Basically, what you have right now is just about right (and I should have used List for that list, which you figured out), that renders from (0,0) to (512,512) (or whatever your texture size is). The only difference between that and my recommendation is that I recommend using 0-to-1 instead because you’ll be more easily able to adjust resolutions in the future as desired. Plus, you can use the .textureCoords DIRECTLY without multiplying by anything, since those end up in the exact same (0,0) to (1,1) space.

You’ll have to use small values for stuff like your LineRenderer’s width (it will be as a fraction of the whole width of the texture), but that’s easy to adjust.

If you use that coordinate system, use the Camera Setup section from the linked article to set up the camera appropriately.

[flav3]

Thank you so much StarManta! i really appreciate it, and i dig how you do articles!

Okay its working in a way, for some reason its showing a square, not a circle, but that’s probably an easy fix, which i haven’t figured out yet, or if someone could just tell me to save time

This is whats currently happening:

It’s just tracking the location of where the raycast is hitting the plane, lines 30-31 show what ive tried but nothing happens in the game view:

Texture2D tex = rend.material.mainTexture as Texture2D;
        Vector2 pixelUV = hit.textureCoord;
        //pixelUV.x *= tex.width;
        //pixelUV.y *= tex.height;


        List<Vector3> list = new List<Vector3>();
        list.Add ( new Vector3((int)pixelUV.x,(int)pixelUV.y));


 

        //gameObject.GetComponent<LineRenderer> ().SetPositions (list.ToArray());


        //tex.SetPixel((int)pixelUV.x, (int)pixelUV.y, Color.black);
        //tex.Apply();

        if (lr == null) lr = GameObject.Find("LineRenderer").GetComponent<LineRenderer>();
        Vector3[] points = new Vector3[pointCount];

        for (int p=0; p<points.Length; p++) {
            float theta = (float)p * Mathf.PI * 2f / points.Length; // theta will be from 0 to 6.28, perfect for trig operations
            points[p] = new Vector3( center.x + Mathf.Cos(theta) * radius, center.y + Mathf.Sin(theta) * radius, 0f);
        }

        center.x = pixelUV.x;
        center.y = pixelUV.y;

        lr.numPositions = list.Count;
        lr.SetPositions (list.ToArray());


        lr.SetPositions(points);
        //lr.loop = true; //the loop apparently doesn't exist in lr so it throws an error, which is why i commented it out
        lr.useWorldSpace = true;
        lr.widthMultiplier = thickness;

i’ve also tried the following (edit: it doesn’t make sense now, just something i tried), but its throws an out of bounds error, where xyz is initialised as the number 0 as a global variable:

        lr.numPositions = list.Count;
        //lr.SetPositions (list.ToArray());

        foreach(Vector3 x in list) {
            lr.SetPosition (xyz, new Vector3 (pixelUV.x, pixelUV.y, 0f));
            xyz++;
        }

I’d like to create a line which follows the trail of the co-ordinates which the raycast hits.

Any ideas?

Edit: I know i shouldn’t user ‘gameobject.find’, but this won’t be a massive project, so i think its okay to use it for now, but i will definetly not use it when taking on bigger projects.

[StarManta]

I think you’ve incorporated too much of my sample code. Line 34, in particular, is just going to overwrite your line 31. In addition, if you’re not intending to draw a circle, then the for loop is unnecessary.

Also noticed the comment about lr.loop: are you on Unity 5.6? LineRenderer.loop definitely exists in 5.6. There were some updates to LineRenderer lately, and that may have been one of them. Importantly, though, the way LineRenderer used to handle corners was pretty hideous-looking, and you definitely want to be using the newer version just for these visual benefits.

[StarManta]

@flav3

Edit: I know i shouldn’t user ‘gameobject.find’, but this won’t be a massive project, so i think its okay to use it for now, but i will definetly not use it when taking on bigger projects.

When you’re ready to replace it, I’ve got an article for that too