Highlighting UI Button when the button is pressed

Hello everyone,

I have a button in my canvas with the button component having transaction mode in sprite swap. The “Highlighted Sprite” has a sprite added in the inspector tab.

When the mouse is dragged on to the button, the highlight works correctly. However my question is when the button is pressed, if I try to repress the button second the time, when I hover the button the highlight sprite does not change anymore. If I press somewhere in canvas other than the button itself, the highlight sprite starts to change again. What I want to do is that the sprite changes when hovered even the button is pressed.

Is it the case with the highlight, do I do something wrong? Any suggestions on that is much aprreciated.

Thanks,