Is it feasible to play 4K video in Unity?

I’m starting work on a 360 VR video player, aimed at mobile devices (iOS/Android Cardboard + GearVR) and I’m evaluating whether it is possible to get video quality from 1080p to 4k in Unity or if I should instead just do the project natively (aka, OpenGL).

I’ve been trying to find information on the state of video in Unity but I haven’t arrived to a clear answer. I’m going to explain the info I’m running on as of now, hopping that someone can add to this or provide an answer or at least show that I am erring on the side of ignorance :slight_smile:

Assume a video in mp4, H264 at 4K resolution (equirectangular mapping):

OpenGL Approach:

If I were to do this in OpenGL, a (somewhat simplified) workflow would entail decoding each video frame, updating a GL_TEXTURE_2D with the pixel contents, and rendering to a sphere with glCullFace(GL_FRONT) as if it where a regular skybox.

With the above setup, the limiting factors would be:

  • Does the GPU in my device support 4K textures? what marketing calls 4k is usually 3840 horizontal, and, an iPhone6 for instance, returns 4096 for glGetIntegerv(GL_MAX_TEXTURE_SIZE,...). So I’d assume the iPhone6 can load textures as big as required for 4K.

  • Fill rate: i.e.: my iPhone6’s GPU might be capable of storing 4K Textures, but it might not be fast enough to fill up a 4K frame buffer at an acceptable frame rate. This unfortunately I think the only way to know is by testing. There is also the added overhead of the Cardboard rendering which means 2 passes per frame.

Unity Approach:

Now for a Unity approach, this is what I did:

  1. Empty scene with a sphere and cardboard camera at the origin.
  2. For the sphere, I created a material based on the Unlit shader with one texture and set the cull face to front.
  3. imported the video and assigned it as the material’s texture.

My observations from the above test are:

UPDATE: I now realize that my test was using Unity’s Move Texture, which is not available on mobile. So if I still pursuit this project in Unity I will anyway have to either do my own native Unity plugin and write the OpenGL code to map the textures to the geometry or purchase an asset like AVPRO

  1. When importing images, Unity also allows texture sizes of 4096 so 4K should be possible.
  2. Importing a video however, takes a ridiculous amount of time. I believe this is because Unity actually uses Ogg Theora and so it takes forever to transcode to Ogg. As a matter of fact, sometimes I was able to import and sometimes Unity would just stall.
  3. When importing a video, I was not given any mmap, filtering or size options as if I was importing an image. So I’m not really sure what Unity is doing to the video. All I get is a quality / compression slider. The end video is still 3840x1920, which was the original resolution.
  4. The rest of the experiment works ok, as in, yes, the video playbacks in the sphere. But the framerate is not great.

So at this point my conclusions are:

  1. Is it possible to do 4K in Unity? Kind of, In the sense that you can import 4K videos and reproduce them without any external assets.
  2. Is it really 4K quality? Probably not. mp4 is lossy and so is ogg, So I assume my video went a second round of lossy compression when it was transcoded to Ogg.
  3. Is it practical? Not sure. So far, video importing takes an impractical amount of time. Also, 4K in general might be too much for a mobile device rending in stereo, regardless of Unity or doing it straight in OpenGL.

As a side note and to complicate things even more, I was thinking of allowing both downloading and streaming video in the app. But from the above it seems that would be out of the question performance wise. Maybe I have to stick to 1080p or even 720p?

It’s possible to play 4k videos in Unity, but from personal experience without external assets, like you said, the framerate is not great and it could take hours to import.

So for myself, I use easy movie texture and it’s totally different, it’s fluid and in the StreamingAssets folders Unity doesn’t convert the videos to Ogg, so it’s really fast.

In case if anyone will be searching for a solution for desktop platforms in this topic (like I did), there is Demolition Media Hap plugin, which is capable of playing videos up to 8k@60fps!

Hi @saldavonschwartz, did you ever manage to get 4K video textures running at a decent frame rate in Unity?

I’m trying to create a project whereby the user watches a 360 video and then gets presented with two buttons to choose from which then selects the next scene. But it all rests on 4K textures running at a decent frame rate. Any help would be greatly appreciated. Thank you.

in my case with easy movie texture 4k video having 40 mb does not streming well. it just stop playing after playing some frame. if put this video in streaming asset folder it works well. but i need to play it from stream … plz help

With Unity 5.6, a new component called VideoPlayer has been introduced. Check it out. It plays 4k videos easily!
Currently no Android support though.

I need to make the same project and running into same question: “What quality should I steam video? Is it worth to use 4K video quality on Android and IOS devices? How can I make 4k been playable on mobile?” What was your solution and maybe you have some updates on this now?

I’m trying to create a project whereby the user watches a 360 video and then gets presented with two buttons to choose from which then selects the next scene. But it all rests on 4K textures running at a decent frame rate. Any help would be greatly appreciated. Thank you.

Even before Unity 5.6, there was always HTC Vive’s Media Decoder (up to 8K) Unity plugin on the asset store.