Drag a sprite by its 2D Polygon Collider component

I’ve got a draggable class that works great for dragging entire sprites and I’m trying to modify it to register dragging only within the defined 2D polygon collider component of the sprite object. I’m having a really hard time getting the drag to only register when I click within the boundaries of the polygon collider. Right now it’s still dragging wherever I click-drag on the sprite including outside the boundaries of my polygon collider. I could really use some guidance on this. I’ve also attached an image to illustrate what I’m trying to achieve. Thanks!

51880-draggable-illustration.png

void Start ()
    {
        collider = transform.GetComponent<PolygonCollider2D>();
    }

    public void OnDrag (PointerEventData eventData)
    {
        Vector3 mousePosition = new Vector3(eventData.position.x, eventData.position.y, 0);
        collider.transform.position = mousePosition;                                       

    if(transform.parent.gameObject == partsPanel)
            transform.SetParent(dragLayer.transform);                                      

        if(transform.parent.gameObject == buildBoard)
            transform.SetParent(dragLayer.transform);
    }

Does OnMouseOver work? Hmm, it might not because you can have the GUI element coexisting with the polygon collider. Anyway, try it.