Definitely not an NGINX expert, so bare with me.
Here’s some ressources I found:
Changes to the WebGL loader and templates introduced in Unity 2020.1 page-2#post-6311457
Alternatively,
Above link is an older thread but if you adapt it to use the MINE types for the more recent Unity version :
It would probably work.
Good luck!