Since the introduction of the new Video Player component in Unity 5.6, we’ve had a lot of questions about how to integrate 360 video into a Unity project. Over the past months we have slowly been refining our suggested workflow and working on a high quality shader to make it happen. Today, we are pleased to announce that we are ready to share our work in the form of a beta shader ready for use in any Unity 5.6 or later project.
The concept is simple and straightforward. Take any supported video file (like an .mp4) containing either 360 or 180 equirectangular or cubemap content, import it as an asset, and play it through a Video Player component. The key is to target the Video Player to a Render Texture of the same dimensions as the video. Then, connect that texture to a Material set to use the new Skybox/PanoramicBeta shader and use that as your Scene Skybox Material.
Voila! You now have a Skybox being driven by your panoramic video! Turn on the Virtual Reality Support Player Setting and throw on a VR headset and you’ll immediately be surrounded by your video in full 360.
If you have 3D 360 content, you can take things a step further for the ultimate immersive experience by using the Skybox Panoramic shader’s 3D settings.
Hi @touloused I tried a 2k 360 video for Android device with Skybox Panoramic shader, but it didn’t run, but the same video with 720p and 1080p was working. can you suggest what could be the issue?
Sure @sushanta1991 , can we start by getting some more info from you?
Which Android device (manufacturer and model) are you working with specifically?
What exactly is the width and height of the video?
Does the video file play in general (ie. can you load it into a fresh VideoPlayer with RenderMode CameraFarPlane and see it on-screen)?
I got the player built out, but it was fixed with the camera facing in one position and wouldn’t let me look around the scene. I thought this was built in with native integration for cardboard…did I miss a step?
You can also support stereoscopic image if you flip the x and z radial coordinates for the right eye. (assuming the video is split vertically down the center)
Hey, this works great for me in the editor, but I get compiler errors when trying to build.
Is there something I’m missing?
The type or namespace name ‘AnimatedValues’ does not exist in the namespace ‘UnityEditor’
Also ‘ShaderGUI’, ‘AnimBool’ and ‘MaterialEditor’ could not be found
Sounds like you may have forgotten to place the SkyboxPanoramicBetaShaderGUI.cs file inside a sub-folder named “Editor” in your Assets. This would have the effect of trying to build Editor code into your game builds (which won’t work).
Hello!
I followed this tutorial to play 360 videos. I edited and built my project on a PC and it works well on PC and Android. However when I copy my project to a Mac, console shows “Metal: Fragment shader missing texture binding at index 0 (_Tex / Skybox/PanoramicBeta)” . When I run the game, the sky is green with some distortion. It would be great if anyone could tell me how to fix this >.< Thank you!
Dear touloused, thanks for sharing the awesome work! Publishing a 360 video with Unity has become super simple indeed!
One issue that I am having is that the top and bottom of the 360 video is “hollow” as you can see from the attached images. Is there a way to “tighten” the hollow gap on the top and bottom of the videos to make it perfectly 360?
I’m not sure why you are seeing the Metal fragment shader warning. You can likely silence it by disabling the “Metal API Validation” checkbox in the Player Settings if you like.
As for the green/distorted video, I’ve seen this on OSX and iOS when the hardware video decoder has trouble decoding the video in realtime. You can validate if this is the problem by either slowing down playback speed (say to 0.25x or .5x) in the VideoPlayer component inspector. You can also take the 360 part completely out of the equation and just try playing back the video to the Camera Back Plane in the VideoPlayer. If that’s the case, it most likely means your video is too high a resolution and/or too high a bitrate for your graphics hardware to keep up. You can try re-encoding your video to be more lightweight using an external tool, or, by using Unity’s transcoding options in the Video asset import settings.
Is it possible that your RenderTexture dimensions don’t match the video dimensions?
It’s hard for me to see clearly in your screenshots, but it looks like your RenderTexture itself may include some black letterboxing at the top and bottom (which would be due to differing resolutions) which would explain what you are seeing.
very nice ! have to test this! it should be much more simple but is it possible to use stereo panorama images not videos? images don´t suffer from video compression and limited resolution. i did not find a way to do even this out of the box right? just simple stereo backdrop images for vr.
i know how to use images as skybox and maybe i can setup cameras to render differnt skyboxes but what would unity consider for lighting (GI Ambient Reflection etc!?) i am kind of lost here
Good intuition. Yes, you can use the new PanoramicBeta shader with a simple static texture. It is not necessary for it to be fed by a video file as described in the instructions. This would allow you to use a 3D 360 or 180 equirectangular still image loaded as a regular Texture asset and will properly render it for each eye.
Hi, what’s the recommended aspect ratio for stereoscopic 360 video? Ive been working with 2160×2160 mp4 files and I’ve had to transcode them every time for Unity to recognize. Is that a file type thing or a size thing? Also what’s the largest aspect ratio that will work?
nice hopefully i get some time to test this! i really wonder what the shader is doing is it just overwriting the clear flags per camera and what is the skybox for lighthing and reflection doing if i merge the panorama with actually 3d content in front of it. good work anyway and a really must ahve for vr!
