How to use, external tools inside of Unity? (the ofVideoPlayer from openframeworks)

Unity Engine
1

Hi, I’m looking for some free and efficient video playback methods for Unity. I know about the MovieTexture, but it’s not free and a lot of people are complaining about performance. Same for ffmpeg. I heard good news about openframeworks but I have no idea how to use it inside Unity. Do I need to create a plugin for Unity, that exploits ofVideoPlayer? or…?

http://www.openframeworks.cc/documentation/video/ofVideoPlayer.html

Thanks for any tips.

2

If you’re using free Unity license, you just can’t do it, you need Pro features for either MovieTexture or for writing a native code plugin.

Now that been said, here’s what you can do in your free version:

  • depending on your target platform you can create video playback window outside unity for full screen cutscenes. Video start and end will likely be glitchy. There are some iOS plugins in asset store that seem to do it.
  • for short and lo-res clips, you can lay the frames on one or many textures and play them back by altering UV offset and size. Some asset store packages do that too. Personally I’ve tried to extend this approach by adding some on the fly decompression in pixel shader, but that wasn’t worth it beck then.
  • an extra hacky solution would be to use a separate process thread, not related to Unity, running in elevated privileges that accesses the OpenGL render context. Knowing the texture id (GetNativeTextureID), it can feed video stream in there frame by frame. Fraps does something like that, it reads framebuffer. Obviously, you can’t do such hack in a shippable product.

If you upgrade to Pro, all possibilities open up.

3
4

Thanks for your input guys, yes @fire7side I have seen that script (last answers you link, that says to convert the video to picture sequence, and animate them) today and gave it a try. The problem with it is that it loads all the images at once, and it’s using the Resources folder, so all the images will be loaded to ram once the game starts (correct me if I’m wrong) - It took a good amount of time to load! - So like, for a low quality crappy 3 min video, VirtualDub gave me more than 4K images, the folder was like 200Migs, so 200Migs of ram, for one video once the game starts. That’s bad.

However, currently I’m loading the images as I need them, so, no overhead when you first start the game. but as the video progresses, ram usage increases (cause more images are loaded).

This has good performance, and good quality (depending on the quality of the pictures, generated by converting the video to a picture sequence)
obviously the higher quality the higher the size, the LONGER it takes for Unity to import the images (it took a good amount of about 10 mins, to import those 4K images)

I guess, this is the best we could do for free. Feel free to suggest other methods.

5

I’ve been playing around with video playback a bit and here’s one solution for free Unity. Currently it’s decoding raw MJPEG bytestream on the fly, but the full bytestream must be preloaded, either at the start of the scene or later when you request it. Next I’ll try to add support for some video container, probably not AVI, but something that is more easy to decode in Unity and is widely supported by other AV tools.

http://shuugames.com/previews/mjpeg_seeu/

It’s a 110Mb web player :evil:

Actually in the long run, I have a custom video codec in mind that could compress and decompress videos either at runtime or in Editor. Hopefully it’d be quality wise somewhere between MJPEG and MPEG2. And the best part is that it runs fully in managed code and in custom shaders, so it’s within free unity limits.

I don’t want to share these scripts in public yet, but if you’re interested in trying it out, just PM me.