Specifically, if we can create a Texture or AudioClip at runtime from external files, why can’t we do the same for a VideoClip? I really need this!
I’m in a tough place here with Dominus Galaxia. The majority of assets are loaded externally and moddable. I’ve been using a relatively expensive asset called OrenVideo up until now. I only have the Windows licence and have no desire to expand that–DG has Linux/OSX builds, and some day I might try a mobile port, but OrenVideo is $200 per platform. For now, Linux/OSX just get static images in lieu of video. but I don’t regret the original purchase seeing is it predated the (now not quite as new) video system.
Even without considering the cost for additional licences, the situation with OrenVideo is far from ideal. It hasn’t been updated in years, every now and then a black frame is inserted between videos, and it uses a proprietary format that isn’t super friendly or accessible.
With the current state of the Unity Video system, I can get things working great when I import my files into the Assets folder as VideoClips, but then it’s impossible to modify anything outside of the editor. This kills any serious race mod, and is also more difficult for artists to work with.
On the other hand, streaming external files is too slow when switching between videos and this results in noticeable hitching.
I’ve been waiting with baited breath for years, periodically checking the state of the video system, for it to finally catch up with my needs. It’s almost there now, but the inability to create VideoClips at runtime is an essential omission.
We hear you loud and clear. This has been a regular request for a long while, and has been on our list of wanted improvements since day one but haven’t been able to get there. It’d be nice if you could explain exactly how you intend to create new VideoClips during runtime. Would you be recording them frame by frame from gameplay? Or just procedurally generating the images in background, adding them as fast as possible? Or would you be downloading already encoded files (e.g. mp4) - or creating them with a plugin - that you just want to turn into VideoClips? In this latter case, you may have seen you can already point the VideoPlayer to local files so you don’t always have to create actual VideoClips; the performance will be the same.
On this note, I’m curious to hear details about what you mean here:
You always have the option of asynchronously preparing the next video that will be played to minimize the delay when you actually start playback. However, gapless playback when sequencing videos is unfortunately something we cannot do right now, but you should be quite close.
As you may have noticed - only in the editor for now - the MediaEncoder API is what we have to create movie files from code. This is what the VideoClip importer uses when transcoding movie files, and it’s also what the Unity recorder is using when encoding mp4/webm files.
So: looking forward to hear more about your needs so we can see how to address this!
Right, sorry. To be more specific, a VideoClip with Transcode enabled gives acceptable performance. At least for the videos that I’ve been testing, I see gapless transitions between clips. Without Transcode there’s noticeable gaps in playback when transitioning VideoClips, or even when playing the same clip with loop enabled. Streaming performs the same as a VideoClip without Transcode, which results in gaps. Sorry again for the confusion.
(Quick aside–based on this behaviour I’m guessing Loop reloads the video entirely, which it probably shouldn’t. Also, it doesn’t look like Loop can be set programmatically, which it probably should.)
Correct me if I’m reading things wrong, but you can’t play one video while preparing the next one, right? In my scenario the initial delay to start the first video is well hidden by a screen fade. It’s just the transitions between videos that hitch. If there were a way to prepare one video while playing another, I think that would work just as well or better for me than being able to programmatically create VideoClips. (Ideally of course, we would have both options!)
Quick update: I tried setting up two video players, so that while one plays the current clip the other prepares the next, alternating between players to work through all clips. Unfortunately the results were the exact same: Noticeable gaps between clips, seemingly equivalent to before. Once again, enabling the Transcode flag greatly improved performance.
Each video player rendered to a separate RenderTexture. I was able to confirm that the second video player finished preparing itself while the first rendered. The first video player enabled the second via callback. So… it looks like there may be some performance issue with non-Transcoded videos that isn’t fixed by preparing the video in advance?
Hi. I will try to give you an update. There are multiple points I want to clarify. From what I understand, what you want is a way to transition between clips smoothly. And to achieve that, you either want to expand a video clip at runtime or have a way to fix the gap between clips.
TL;DR We still don’t provide a way to create clips at runtime outside the editor. We are still planning on doing it and this is a popular request. But in your case, I think you should be able to transition between clips without gaps. I am not sure exactly what you mean by gaps. What I understand, is there seem to be missing video frames or audio samples making the transition between clips feel like there is a jump in-between. I tried making a script for this workflow and would like to know if this works. And if not, can you record the issue, so I can better help you?
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.Video;
public class GaplessTransitionController : MonoBehaviour
{
public VideoPlayer StartPlayer;
public VideoPlayer WaitingPlayer;
private long currentFrame = -1;
void Start()
{
WaitingPlayer.Play();
WaitingPlayer.Pause();
WaitingPlayer.frame = 0;
StartPlayer.Play();
StartPlayer.loopPointReached += StartPlayer_loopPointReached;
}
private void StartPlayer_loopPointReached(VideoPlayer source)
{
Debug.Log("StartPlayer_loopPointReached");
StartPlayer.Pause();
WaitingPlayer.targetTexture = StartPlayer.targetTexture;
StartPlayer.targetTexture = null;
WaitingPlayer.loopPointReached += StartPlayer_loopPointReached;
StartPlayer.loopPointReached -= StartPlayer_loopPointReached;
WaitingPlayer.Play();
StartPlayer.frame = 0;
var temp = StartPlayer;
StartPlayer = WaitingPlayer;
WaitingPlayer = temp;
}
private void Update()
{
if(WaitingPlayer.isPlaying)
{
Debug.Log("WaitingPlayer.isPlaying");
}
if(StartPlayer.frame != currentFrame)
{
currentFrame = StartPlayer.frame;
Debug.Log(currentFrame);
}
}
}
And this is what it looks like on my side. I get in the Debug.Log all the frames in my clips.
You can hear a gap in the audio but the audio in the file doesn’t loop well. So not sure if it is an issue on your side.
All transcoded videos using H264 are encoded in Baseline. If precision is really important, you need to encode your video in Baseline.
Again, all this is a workaround. We know it should not be this difficult to implement a gapless transition, and we are working on improving this. We have a lot of platform-dependent code that we don’t have access to, making simple features take years to complete. To fix that, we plan to build our own tools while trying to give as much user value as possible. On the same subject, we now have a roadmap that you can vote on and propose new features, and you can see what we are doing and planning. https://unity.com/roadmap/unity-platform/audio-video.
I think I have a question, connected to this topic.
I have several videos that play when corresponding AR markers are detected by phone camera.
To speed up this process I download videos to phone’s memory (they are relatively small), but still there is a 2-3 second lag between recognition of marker and starting of playback. As far as I understand, this time is taken by Prepare function. Excluding it from workflow doesn’t help, since I see white screen for approximately same time before playback.
I don’t know what video would be shown next, so I can’t prepare certain video before showing.
So, my question is - is there is a way to prepare downloaded videos somehow to speed up playback start?
Maybe move them to some particular folder, encode every video stored on server in specific manner or something else.