How to speed up initial loading time?

From the common issues article regarding Gear-VR submission:

“App must begin accepting input, respond to head tracking, and display graphics within 4 seconds of app startup”

(Not Found | Oculus)

Since the custom Unity Plus loadscreen for VR seem to have severe problems with head tracking, we tried to implement our own (using LoadSceneAsync() and Application.backgroundLoadingPriority = ThreadPriority.Low). However, even though my loading scene is only a camera and a textured rect, it still takes 6 seconds before it shows up. What can we do to reduce initial loading time?

Bump - I am having the same issue, and since was the first result on Google, anyone with Insight would be a service to mankind.

The behaviour of Unitys built-in VR loadscreen (which can be used with a custom image) is totally fine regarding submission.