Alpha channel in 360 video pano?

Unity Engine
1

I’ve been using the new panoramic video features of 2017.3 and finding that they simplify a lot of my old stereo / 360 workflows. One problem though: has anyone found a way to get support for alpha channels working properly?

This was demonstrated at a Unite presentation this summer:

The new Video Player supports .Webm alpha channels, but it doesn’t seem to extend when applied to this new Material. Hope there’s a fix, it would be really powerful to be able to layer these projections.

1 Like
2

Hi marcusestes! The panoramic shader currently produces an opaque result. It will use the incoming alpha for affecting the color level if the HDR controls say so, but this is not what you’re asking.

Skybox shaders, by definition, produce an opaque output so I’m not sure how to fix this at the shader level.

However, in 2017.3, we have introduced a number of new playables that allow you to play video content through a graph where you can control how multiple streams are mixed. The output of such a graph is a RenderTexture, that you can then feed to the material which is using the panoramic shader.

The following example shows how to use the texture mixer and video clip playables to perform a simple mix of 2 images. You can instead use the MaterialEffectPlayable to take full control of the compositing operation if you prefer:

using UnityEngine;
using UnityEngine.Playables;
using UnityEngine.Experimental.Video;
using UnityEngine.Experimental.Playables;

public class Compositor : MonoBehaviour {

    public UnityEngine.Video.VideoClip clipA;
    public UnityEngine.Video.VideoClip clipB;
    public RenderTexture output;

    private PlayableGraph graph;

    void Start () {
        graph = PlayableGraph.Create ();
        var playOut = TexturePlayableOutput.Create (graph, "out", output);
        var inA = VideoClipPlayable.Create (graph, clipA, true);
        var inB = VideoClipPlayable.Create (graph, clipB, true);
        var mix = TextureMixerPlayable.Create (graph);
        mix.ConnectInput (0, inA, 0);
        mix.SetInputWeight (0, 1.0f);
        mix.ConnectInput (1, inB, 0);
        mix.SetInputWeight (1, 1.0f);
        playOut.SetSourcePlayable (mix);
        graph.Play ();
    }

    void Disable()
    {
        graph.Destroy ();
    }
}

So, not sure if this answers your question in the way you expected, but maybe you can workaround the opaque skybox limitation with this.

But I’ll definitely have a look at the possibility of having the alpha be preserved in the panoramic shader, in case we can provide a more direct solution for the workflow you describe.

Hope this helps,

Dominique
A/V developer at Unity.

5 Likes
3

Quick update on this for anyone following this thread: Dominique’s code snippet does a good job at blending two video sources. Thanks for thinking that through for us, it was a good intro to the playables system. But note that what this does is evenly reduce opacity from two sources in order to composite them. Unfortunately this doesn’t help pull off a traditional keying effect with green screen footage, or video sources with an embedded alpha channel.

It’s possible that delving more deeply into MaterialEffectPlayable would allow for this, but the new playables system is still too opaque for me to go down that research route. I’ve decided instead to take advantage of the new 360 video features in Premiere to create a composited video output with an equirectangular projection and then just play the pre-composited footage back using the new pano shader. It’s a more limited approach, but it’s probably the best option until they decide whether to preserve alpha channels in the pano shader.

For what it’s worth, I think it’d be a very useful feature. And if we have to export a separate alpha channel to help with masking, that’d be more than acceptable. Thanks for your help Dominique, you rule:)

1 Like