Movie Cutscene Nightmare

Questions & Answers
1

I am trying to get a full-screen opening scene movie to play in Unity when the game loads.

Using the traditional technique I am able to make that happen, i.e dumping the .mpeg file as an asset and using it as a MovieTexture. However the performance is ridiculous. In VLC this movie plays smoothly and beautifully. In Unity it is choppy and the first few frames are missed.

As an alternative, I tried the LibVLC. With a few simple Interops and 6 lines of code I was able to make the movie run in a simple C# application in a window the I created. It was dead simple, the performance was awesome, it run smoothly.

However doing that in Unity seems to be a nightmare. I tried the simple technique of just creating an empty scene with the camera told not to clear the buffer, and then telling libvlc to use the top level window (acquired using User32.dll interop). The result is that the movie is playing because I can hear the audio, but the video does not display in the Unity window.

What is the recommended way to run game cutscenes in Unity with good performance?

Edit: Just to give some context or in case the code might give someone an idea for the solution, here is my dead simple LibVLC-based mpeg player:

public class EmbeddedVlc
{
	public void Play(string file)
	{
		var libVlc = libvlc_new(0, null);

		var media = libvlc_media_new_path(libVlc, "test.mp4");
		var player = libvlc_media_player_new_from_media(media);

		libvlc_media_player_set_hwnd(player, FindWindow(null, null));
		libvlc_media_player_play(player);
	}

	[DllImport("user32.dll", EntryPoint = "FindWindow")]
	public static extern IntPtr FindWindow(string className, string windowName);
	
	[DllImport("libvlc")]
	private static extern IntPtr libvlc_new(
		int argc,
		[MarshalAs(UnmanagedType.LPArray, ArraySubType = UnmanagedType.LPStr)]
		string[] argv);

	[DllImport("libvlc")]
	private static extern IntPtr libvlc_media_new_path(
		IntPtr p_instance,
		[MarshalAs(UnmanagedType.LPStr)] string path);

	[DllImport("libvlc")]
	private static extern IntPtr libvlc_media_player_new_from_media(IntPtr media);
	
	[DllImport("libvlc")]
	private static extern void libvlc_media_player_set_hwnd(
		IntPtr player, IntPtr hwnd);
	
	[DllImport("libvlc")]
	private static extern void libvlc_media_player_play(IntPtr player);
}
2

Compress it with Theora with acceptable quality, scale it down.
For expample, if you have a 1024x768 app, compress it to 800x600 with 80% of quality (-v 8 while using ffmpeg2theora), and then stretch it to fill the screen.

Your technique is unadequate because of capturing the render context for the empty window in your demo app is the one thing, while trying to recapture context in the window, that already has one, is really the other. It is up to OS to resolve this conflict, and IMHO, second context initialization via libVLC will fail.

The other reasonable way is that may be some day in some parallel universe they will give us the possibility to make low-level render extensions.

3

How did you get the libvlc.dll to work with Unity?

I’ve got Unity pro and am able to copy the libvlc.dll into the Assets\Plugins directory (libvcl.dll obtained from the C:\Program Files (x86)\VideoLAN\VLC). However, when I try to create a new C#/Mono script to attach to a gameObject it doesn’t recognize the DLL.

Do I have to rebuild the libvcl.dll to work with Unity? Or, should the P/Invoke stuff work properly with the DLL?

Thanks.
-Tony