Gui movie texture lagging

Unity Engine
1

I’m trying to get a movie to play on a Gui texture fullscreen for a school project. I got it to work with the following script but the video is lagging a lot. I’ve seen several posts from people with the same problem but nothing works for me, I’ve tried rendering the video without sound to no avail.

The video format is 1280x720 Quicktime .Mov at 24 fps.

Here’s the script:

public MovieTexture movie;
   
    void OnGUI()
    {
        useGUILayout = false;
        GUI.DrawTexture(new Rect(0, 0, Screen.width, Screen.height), movie);
        movie.Play();
    }

Any ideas on why it lags and what to do to make it run smoothly?

2

I’ve not used MovieTexture myself because I don’t have Pro, but I’ve read that a lot of people are unhappy with its performance. To get around it, I’ve written a little image sequence movie texture script. If you can convert your movie to an image sequence (if you happen to use After Effects, it can export to image sequences), this might be of use to you.

I originally wrote the script so it loads the images asyncronously, but Resources.LoadAsync takes too long to load the images, no matter the buffer size. I found that the regular, blocking load call is very fast, so I reckon that the frame buffer could be dropped in the script; I’ve not gotten around to testing this yet. :> It could also be made less resource intensive by not changing .material directly, but in my project I’ve not experienced any performance impact the way it’s currently set up.

Image sequences are bit unpractical to work with of course, considering the large amount of files and small amount of compression involved. I intended to use EmguCV to read video files and play them back frame by frame (I already needed EmguCV for other stuff in the project), but I couldn’t get the ffmpeg library working. Today I happened across a different C# ffmpeg library, so I might take another shot at doing video playback in Unity with that.

using UnityEngine;
using System.Collections;
using System.IO;
using System;
using System.Linq;

public class VideoTextureImageSequence : MonoBehaviour
{
    public string _imageSequencePath;
    public float _fps = 24;

    private string _frameName;
    private int _amountOfNumbersInFrameName;

    private const int FRAME_BUFFER_SIZE = 50;
    private Texture2D[] _frameBuffer = new Texture2D[FRAME_BUFFER_SIZE];

    /// <summary>
    /// The index in the current frame buffer that's being displayed.
    /// </summary>
    private int _frameBufferIndex;
    /// <summary>
    /// The index of the frame in the image sequence.
    /// </summary>
    private int _imageSequenceIndex;
    private int _amountOfFramesInSequence;

    private bool _isPlaying;

    // Use this for initialization
    void Start()
    {
        FindFrameNameAndExtension();

        //Init buffer
        for (int i = 0; i < FRAME_BUFFER_SIZE; i++)
            BufferFrame(i);
    }

    void OnGUI()
    {
        _isPlaying = GUI.Toggle(new Rect(10, 10, 200, 20), _isPlaying, "Play");
    }

    /// <summary>
    /// Look up the naming convention of the frames in the image sequence folder.
    /// </summary>
    private void FindFrameNameAndExtension()
    {
        DirectoryInfo info = new DirectoryInfo(@"Assets\Resources\" + _imageSequencePath);
        FileInfo[] files = info.GetFiles().Where(f => f.Extension != ".meta").ToArray();
        //Get the name of the first frame, without its frame indexer.
        _frameName = files[0].Name.Substring(0, files[0].Name.LastIndexOf('_'));

        string nameWithoutExtension = Path.GetFileNameWithoutExtension(files[0].Name);
        _amountOfNumbersInFrameName = nameWithoutExtension.Substring(nameWithoutExtension.LastIndexOf('_') + 1).Length;

        _amountOfFramesInSequence = files.Length;
    }

    private void BufferFrame(int bufferIndex)
    {
        string path = string.Format("{0}/{1}_{2}", _imageSequencePath, _frameName, _imageSequenceIndex.ToString("D" + _amountOfNumbersInFrameName));
        //Load directly instead of async - loadasync is too slow to keep up with the frame rate, and the blocking load does not impact the FPS.
        _frameBuffer[bufferIndex] = Resources.Load(path) as Texture2D;
        _imageSequenceIndex++;
        _imageSequenceIndex %= _amountOfFramesInSequence;
    }

    // Update is called once per frame
    void Update()
    {
        if (_isPlaying)
            StartCoroutine(ShowNextFrame());
    }

    private IEnumerator ShowNextFrame()
    {
        GetComponent<Renderer>().material.mainTexture = _frameBuffer[_frameBufferIndex];
        //Unload the previous frame, and load the next one in the slot.
        UnloadTexureAndBufferNextFrame((_frameBufferIndex - 1 + FRAME_BUFFER_SIZE) % FRAME_BUFFER_SIZE);
    
        _frameBufferIndex++;
        if (_frameBufferIndex == FRAME_BUFFER_SIZE)
            _frameBufferIndex = 0;

        //Play at the desired FPS.
        yield return new WaitForSeconds(1f / _fps);
    }

    /// <summary>
    /// Frees up memory and loads the next texture into the given buffer slot.
    /// </summary>
    private void UnloadTexureAndBufferNextFrame(int frameBufferIndex)
    {
        Resources.UnloadAsset(_frameBuffer[frameBufferIndex]);
        _frameBuffer[frameBufferIndex] = null;
        BufferFrame(frameBufferIndex);
    }
}
1 Like
3

Thanks for the script, Ill definitely try it out. I am a bit worried about the size of my image sequence being as its 1645 frames :slight_smile: It’s strange though, a lot of games have cut scenes that are rendered videos, there must be a way to make unity play a video smoothly. I’m thinking about creating a whole new level just for the cut scene vid, maybe that’ll make it lag less. Anyhoo, thanks a lot for your feedback and idea to use an image sequence instead of a rendered video. :wink:

4

Could it be that the video is 24 fps but your game is (ideally) running at a much higher framerate than that?
You’re also calling Play in OnGUI which means it’s going to start playing multiple times per frame.

Re: Pro - just upgrade to 5. No features are locked.

5

Oh, right you are! I could’ve sworn that in 5 MovieTexture was still paylocked, but it looks like I was quite mistaken.

6

It could be both.
I think the OnGUI might be the problem if its called multiple times per frame. I’m using GUI because it makes my video play fullscreen. Is there another way to play my video fullscreen that doesn’t require OnGui and that isn’t called multiple times per frame?

7

You can draw the texture in OnGUI - just don’t call Play() from there.

8

Update: I tried putting the video on a plane with this simple start script to test the lagging:

void Start () {

        MovieTexture movie = renderer.material.mainTexture as MovieTexture;
        movie.Play ();

   
    }

And it’s still lagging a lot, so perhaps it is the frame rate. I just wonder why it’s not more talked about in the community and internet in general.

9

Oh okay, Ill try it :slight_smile:

10

Try setting this to 24 - Unity - Scripting API: Application.targetFrameRate

11

I tried, it didn’t change anything in the lag department… Ill have to look into trying to find some tutorials about cut scenes in general and see if I can’t find a solution. It feels so close though… Darn!

12

I figured out it’s lagging because the video was rendered with compression. If rendered as a PNG sequence and turned into a sprite theres no lag. If rendered as a .mov with animation codec the file size is 2.3GB and for some reason unity reports an error when I try to import it into my assets folder, but I’ve tried with smaller uncompressed videos (.mov) that play without lag in unity. So the next step would be to figure out how to get a sprite to play fullscreen…I can ad a separate render of the sound from the video and it would work that way.

13

I had the same lagging problem and fixed it by setting QualitySettings.vSyncCount to Don’t Sync.

e.g.

public MovieTexture myMovie;
int vsyncprevious;

void Start ()
{
   vsyncprevious = QualitySettings.vSyncCount;
   QualitySettings.vSyncCount = 0;
   myMovie.Play();
}

void Update()
{
   if (!myMovie.isPlaying())
   {
     QualitySettings.vSyncCount = vsyncprevious;
   }
}