I have a PNG image that I want to make a clickable button.
The problem is, I only want the area of the image to be clickable. Is there a way to define a custom shape for a button?
Thats a good question actually. The only way I can think of is to script small buttons to make that texture. I can't give a good solution to this.
EDIT: another might be to have that texture on a billboard/mesh (probably a mesh because billboards are rectangular), and have it positioned at a certain spot in front of the camera at all times. Then you can just raycast the mouse to see if it gets clicked on. Hopefully someone can provide a solution using GUI though, or this would be a big weak point in the GUI system.
I hacked together a quick demo that uses a raycast hit UV coordinate to do a texture lookup on your button texture. By checking if the alpha is > 0.5, you can differentiate a hit on the solid part vs the transparent part. The main script is here, but no guarantees that this is the most elegant way to accomplish this!
The unitypackage file is here: AlphaButtonTest.unitypackage
using UnityEngine;
using System.Collections;
public class RaycastAlphaCheck : MonoBehaviour {
public Texture2D buttonTexture;
// Use this for initialization
void Start () {
}
// Update is called once per frame
void Update () {
Camera.main.backgroundColor = Color.black;
if (Input.GetButton("Fire1"))
{
Ray ray;
RaycastHit hit;
ray = Camera.main.ScreenPointToRay(Input.mousePosition);
if (collider.Raycast(ray, out hit, 100.0f))
{
Color pixel = buttonTexture.GetPixel((int)(hit.textureCoord.x * buttonTexture.width + 0.5f), (int)(hit.textureCoord.y * buttonTexture.height + 0.5f));
if (pixel.a>0.5f) Camera.main.backgroundColor = new Color(0.2f, 0.6f, 0.2f);
Debug.DrawLine(Camera.main.transform.position, hit.point);
}
}
}
}
One possibility could be to define the button as a vector shape using a series of points, and use this script to see if a click is inside that shape.
You've already gotten some good answers, but I'll go ahead and mention another option, which is to write a custom point-containment function.
The point containment test for a 'wedge' as in your image is fairly trivial, as are containment tests for most convex 2-d objects. The point-in-poly function linked to earlier will work, but with a custom function you wouldn't have to approximate the curve. Also, the function linked to earlier is for a possibly non-convex polygon, which, while it will work, is more than is needed here (a simple point-in-convex-polygon test would suffice).
That said, the function linked to earlier will certainly work, and if the curve is subdivided to a sufficient degree, it's unlikely that the inaccuracy will be noticeable.
I have a solution, without writing a lot of code.Pixel testing can do that absolutly, but it seems consuming.
My solution is to use native Unity Polygon Collider covering the texture first, and then add some code to UICamera let it can check Collider2D.Now the UICamera can only check 3d collider by using Physics.Raycast(). By using Physics2D.Raycast(), your code can sence events as same as 3d colliders. What i have done is simply copied a same process of 3d colliders but change everything to 2d. By this way, you can easily get the feautures about 2d colliders, but little bit ugly, I think to make it beautiful is not my resposibility but tasharen.com’s 
here is the code I modified from NGUI 3.4.9
Download UICamera.cs
if you can read chinese, can visit this: this article