Is there a way for the button to be grouped with the panel so that once the button is pushed and the panel appears from the side, the button will also be pushed to the side?
You could manually move the button with the movement of the panel. You could make the button a child of the panel when the panel appears, so it follows the movement of the panel, and then make the button no longer a child of the panel before the panel disappears. You could make both the panel and the button a child of the same object, and move that object instead of moving the panel, which would also move the button. Probably plenty of other ways.