Video Player custom playable

Unity Engine
1

I’m trying to write a custom playable that allows me to scrub the timeline to scrub the video inside the video player. I have this.

using System;
using UnityEngine;
using UnityEngine.Playables;
using UnityEngine.Video;


[Serializable]
public class TheaterPlayableBehavior : PlayableBehaviour
{
    private GameObject m_Theater;

    private VideoPlayer m_VideoPlayer;

    public GameObject theater
    {
        set { m_Theater = value; }
    }

    public override void OnGraphStart(Playable playable)
    {
        m_VideoPlayer = m_Theater.GetComponentInChildren<VideoPlayer>();
        m_VideoPlayer.Prepare();
    }

    public override void PrepareFrame(Playable playable, FrameData info)
    {
        m_VideoPlayer.time = playable.GetTime();

    }

}

[Serializable]
public class TheaterClip : PlayableAsset
{
    public ExposedReference<GameObject> theater;

    public override Playable CreatePlayable(PlayableGraph graph, GameObject go)
    {
        ScriptPlayable<TheaterPlayableBehavior> playable = ScriptPlayable<TheaterPlayableBehavior>.Create(graph);

        playable.GetBehaviour().theater = theater.Resolve(graph.GetResolver());
        return playable;
    }
}

For some reason the playback is really choppy. Is there a way to sync the video player playback to the timeline playback?

3 Likes
2

2018.3.1 still has issues with timeline and custom playables, mainly some events not firing properly, and official video player playable from “default playables” pack (https://assetstore.unity.com/packages/essentials/default-playables-95266) doesnt work as well.
I managed to modify the code above to make it work. The scrubbing/seeking is not instant, but otherwise things work fine.

using UnityEngine;
using UnityEngine.Playables;
using UnityEngine.Video;

[System.Serializable]
public class TheaterPlayableBehavior : PlayableBehaviour
{
    [SerializeField]
    private GameObject m_Theater;
    [SerializeField]
    private VideoPlayer m_VideoPlayer;

    FrameData.EvaluationType lastEval;

    public GameObject theater
    {
        set { m_Theater = value; }
    }

    public override void OnGraphStart(Playable playable)
    {
        m_VideoPlayer = m_Theater.GetComponentInChildren<VideoPlayer>();
        m_VideoPlayer.Prepare();
    }

    public override void PrepareFrame(Playable playable, FrameData info)
    {
        if (info.evaluationType == FrameData.EvaluationType.Evaluate)
        {
            m_VideoPlayer.Prepare();
            m_VideoPlayer.time = playable.GetTime();
            m_VideoPlayer.Pause();
        }
        if (lastEval==FrameData.EvaluationType.Evaluate && info.evaluationType==FrameData.EvaluationType.Playback)
        {
            m_VideoPlayer.Play();
        }
        if (info.evaluationType == FrameData.EvaluationType.Playback)
        {
            m_VideoPlayer.time = playable.GetTime();
        }
        lastEval = info.evaluationType;
    }

    public override void OnBehaviourPause(Playable playable, FrameData info)
    {
        base.OnBehaviourPause(playable, info);
        if (m_VideoPlayer)
        {
            m_VideoPlayer.Prepare();
            m_VideoPlayer.time = playable.GetTime();
            m_VideoPlayer.Pause();
        }
        lastEval = FrameData.EvaluationType.Evaluate;
    }

}
1 Like
3

First off, thanks for the post, that is definitely an improvement on the video player.

Second, can you be more specific about what is not firing properly?

4

Sure thing. Basically OnBehaviourPlay doesnt fire if you seek to anywhere on the timeline and then press play. It only fires if you follow these steps:

  1. start playback
  2. pause by pressing play button
  3. press the play button again to resume.

Any other way it wont fire at all.

Also frameData.seekOccured always returns true, even when you are not seeking - i.e. when you are simply playing timeline, and even in the game mode. Thats why i had to resort to using evaluationType and comparing it against last known evaluationType manually.

5

Ah thanks - yes, OnBehaviourPlay is a misleading name. In timeline, it’s more of active state for timeline. The bug is the OnBehaviourPause getting called when the graph is paused, but the playable is still active - we added an
effectivePlayState to the frame data so you can check for that case in OnBehaviourPause. Not the ideal fix, but one that won’t break every custom playable already out there. OnBehaviourPause gets called when one of three cases happens - the graph is paused, the playable is paused (i.e. disabled) or a parent node in the graph is ‘paused’. The effectivePlaystate will be Playing if the playable and it’s parents are active.

frameData.seekOccured always returns true because timeline is aggressively setting the local time on all (clip) playables each frame, causing them to be ‘seeked’. The evaluation type is the correct way to check, especially since 2018.2, where the graphs actually play in the editor (2018.1 and prior it was just faked via scrubbing).

1 Like
6

somethinks working ?

7

I’m actually curious too if it’s in the pipeline to update the “default playable” example, or something integrated to add Video tracks to the timeline?

8

It’s in the pipeline yes, but not scheduled for release in the near future.

1 Like
9 
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.Timeline;
using UnityEngine.Video;

public class ITimeControlVideoPlayer : MonoBehaviour, ITimeControl
{
    [SerializeField] VideoPlayer videoPlayer;

    void Awake ()
    {
        videoPlayer.Prepare ();
    }

    public void OnControlTimeStart ()
    {
        videoPlayer.Play ();
    }

    public void OnControlTimeStop ()
    {
        videoPlayer.Pause ();
    }

    public void SetTime (double time)
    {
        videoPlayer.time = time;
    }
}

This works in 2019.1

10

I was adding this class to a videoplayer and setting up 3 UI controls. I am able to hook up OnControlTimeStart and OnControlTimeStop to the button events which work but SetTime does not show up in the list?

Update: Cannot seem t use doubles from the event UI. Floats show up.

11

@Goyoman Thank you for your work. How is this script intended to be used? Do I have to modify the Default Playables video script from Unity to actually call these functions or do I just drop it on a GameObject and assign my VideoPlayer into the field?

12

Hi @customphase I’ve tried using these modification of yours with the code above. But every time I scrub the timeline the reference to m_videoPlayer gets lost and is set to null. It works in OnGraphStart as it should, but then in PrepareFrame suddenly m_videoPlayer is null again and I get an error message. Any idea on why that is, and how to fix it? Thanks! :slight_smile: