How to refresh sprite atlas in play mode

Normally after changing the texture asset for a sprite and then looking at the sprite in editor in play mode, the sprite is reimported automatically and the change shows up in game immediately. This is extremely useful for artists.

After adding the sprite to a sprite atlas, the changes no longer show up in play mode and require a restart. This probably has to do with how sprite atlas works but is there a way to refresh the sprites in atlas in play mode? I would do that only in editor as part of workflow.