So I have done a build, and it plays OK in my browser. When I upload the files, do I upload the whole project folder (assets,library) or just the folder with the build files? And should the html file be at the top of the directory?
Yes, it's that easy. Just upload the build files. For a typical simple project, you just need two files, the .unity3d file, and the .html file.
When you build your webplayer project, these should be generated together, and they should stay together in the same folder wherever you choose to upload them.
The .unity3d file contains your entire built project, packaged up into a single file. The .html page is just the container web page which embeds the unity game.
You only need other files if you're including asset bundles, or other server side scripts (and by the time you're using those, you probably already know what you're doing in this regard!).