Manually adjust button interaction collider

The Button component seems to register mouse events from anything touching its image or any child images. I want to adjust its “interaction area” manually and not have it constructed automatically. Is there a way to do this?

We use EmptyGraphic control that for that. Basically you use this component and use button script on same GO. This can go as child for GO in any size you want. Not sure about irregular shapes

public class EmptyGraphic : Image
{

    protected override void OnFillVBO(List<UIVertex> vbo)
    {
        vbo.Clear();
        var item = default(UIVertex);
        item.position = Vector3.one;
        item.uv0 = Vector3.one;
        vbo.Add(item);
        vbo.Add(item);
        vbo.Add(item);
        vbo.Add(item);
    }
}

though.