You can take a look at the progress window and it’ll tell you what it’s doing, check to see what is taking the longest.
I can recommend getting rid of any shaders that you aren’t using, including ones in the Oculus Integration content.
Also remember you can test with the Rift within Unity without having to push to the Quest to do testing.
I’m having the same problem on Quest using VR LWRP on Unity 2018.4.12f1. Simple scene with terrain (4 layers) and a few identical spheres takes 2+ hour to build. 90% of that time is in “Compiling shader variants”. For example, it has now been building “Lightweight Render Pipeline/Terrain/Lit: xxx of about 9072” for at least 10 minutes! I have an older HP Spectre but still.
NOTE: it is faster on 2nd build if I don’t change any lighting or materials, but if any project settings change, we’re back to a 2+ hour build.