Why do I "Save Project"?

This seems a really stupid question, but I really don't understand what is "saved" when I use "File -> Save Project". As I understand it, a project is a collection of assets, each of which is a file in the file system (and versioned in the Asset Server or an external VCS). So what is "saved" when I "Save Project" and why isn't that (whatever it is) just an asset like anything else?

(I understand Save Scene - that's saving the Hierarchy as a .unity asset in the project)

Your assumptions are correct. "Save Scene" saves the current hierarchy into .unity scenefile.

"Save Project" saves all unsaved modifications to any asset in your projectfolder to disk. This also automatically happens when you close Unity.

We don't automatically write any asset modification to disk immediately. We mark the asset as dirty, so that next time we save all assets, it will get saved.