This seems to happen more or less randomly, maybe with only new UI stuff. I had an UI solution that had 5 panels, each panel had a couple of buttons and an image (Sprite Renderer). When I pushed the whole thing (With the gitignore files advertized ~everywhere the default ones) and pulled a fresh copy, the image elements were still there, but the sprites did not populate them.
Where is this data stored or is this a stupid user error?