h.265 video codec not quite supported

Unity Engine
, ,
1

I need the unity video player to play a 2880 x 1440 resolution video encoded as a h.265 mp4. It plays fine on both mac and windows in vlc. In Unity on windows the playback is so choppy its unusable. I on mac I just get an error. Unity claims to support this codec but has anyone had any success with it in a project?
Thanks. - John

This is the error I get on mac.

Error ‘Cannot Decode’
Error details:
Error Domain=AVFoundationErrorDomain Code=-11833 “Cannot Decode” UserInfo={NSUnderlyingError=0x7fbedb45e6c0 {Error Domain=NSOSStatusErrorDomain Code=-12906 “(null)”}, NSLocalizedFailureReason=The decoder required for this media cannot be found., AVErrorMediaTypeKey=vide, NSLocalizedDescription=Cannot Decode}

Codec: hev1
Format:
{
BitsPerComponent = 8;
CVFieldCount = 1;
CVImageBufferChromaLocationBottomField = Left;
CVImageBufferChromaLocationTopField = Left;
CVPixelAspectRatio = {
HorizontalSpacing = 1;
VerticalSpacing = 1;
};
Depth = 24;
FormatName = “‘hev1’”;
FullRangeVideo = 0;
RevisionLevel = 0;

…

2

Hi!

Like all the codecs used by the VideoPlayer (except vp8), H.265 is supported on platforms where the native OS supports it, using the libraries what’s already present in there. This means some variability is unfortunately bound to happen.

On AVFoundation (OSX), only the ‘hvc1’ variant of H.265 files are supported, and here I see the file you are using is of the “other” variant: ‘hev1’. The difference is not only the string itself but some details in the file layout.

If you don’t have the original source to re-create a new video with the right variant/layout, then you can transcode it with an external tool. With ffmpeg for example, this would go along the lines of

ffmpeg -i input-hev1.mp4 -c:v copy -tag:v hvc1 -c:a copy output-hvc1.mp4

and this will just re-create the packaging so it won’t lead to loss of quality.

The same goes for Windows in terms of variability: the fact it plays at all means you have installed the HEVC video extension, but the slow performance means something is not keeping up.

For example, Microsoft’s guarantee for hardware acceleration of H.265 is up to 1920x1088. Past this, it will still work but if the hardware cannot do it, then it reverts to software decoding. See the note about Maximum Resolution in here: H.265 / HEVC Video Decoder - Win32 apps | Microsoft Learn

If you’re using H.265 in a context where the playback system has too many unknowns for all of this be reliable, then you should probably revert to using H.264 since the resolution you are using does fit within this codec’s constraints (though the size difference will clearly be a disadvantage).

Hope this helps; let us know if this is enough to get you going. If there’s anything that still isn’t making sense after validating some of the above, we can keep looking together or ask you to file a bug so we can reproduce internally.

Dominique Leroux
A/V developer at Unity

3

Thank you,
I appreciate you taking the time on this detailed explanation. So it looks like from what you are describing, apple only supports a certain format and windows will support it, if you download the codec separately (which i did) but only up to a certain resolution. And Unity essentially just falls back to whatever the OS supports.

My question then is, what is the recommendation for developers working with high resolution video. My current project would ideally be 5760 x 2880 . Everything I have read is points to H265 as the codec designed and recommended for high resolution videos. Unity on their main page brags/promotes developing 360 video with their game engine. So it seems like Unity wants to be the go to tool for this. However, from a practical standpoint it looks like h265 as its currently implemented is not ready for prime time. I don’t think the average user on windows can be expected to go download a codec. End users expect things to just work out of the box and I think that’s fair. So it seems like developers are currently blocked. Does Unity have a recommended path forward?

My second question is, Right now VLC works swimmingly out of the box. No need to download the codec on windows, no need to change the tag and format on iOS. No drop in framerate at high resolutions. (At least on my machines.) From what I can tell, this is because they are not relying on the windows or iOS implementations, or it would have the same issues as Unity faces. Is there any plans in the future for Unity to move to their own implementation H264 much like VLC does, and thus allow developers to work with high resolution video?

4

In the last sentence, I meant h265. The forum wont let me edit it to make the correction.

5

Hi!

Sorry for the long pause, been away for a few weeks. The most common format right now is H.264, which means living with a max of 4k resolution. And make sure you distinguish between what the spec allows, and what the platform has chosen to support. For instance, on Windows 8+ (so this includes Windows 10), H.264 is supported up to 4096 × 2304 pixels. But on Windows 7 the max is 1920x1088, as described by Microsoft over here: H.264 Video Decoder - Win32 apps | Microsoft Learn

And as you are finding out, H.265 - although we do support it when present - is a tough requirement to put on users. The reason for this complexity around H.265 seems to be licensing. Video codec technology is quite protected with patents/licences, so that when it’s included in a product (OS, game, …) it’s because someone has accepted to foot the bill in one way or another.

One way out of this situation is to use the vp8 codec. This is royalty free and works up to 16384 x 16384. But because Unity uses a software implementation, the performance may not match that of hardware-accelerated implementations we tap into for H.264/H.265. Android is an exception here, where many devices have a hardware implementation of vp8. Successors of vp8 (namely: vp9 and av1) are also something we have an eye on but haven’t yet had time to dig into. Note that I’m not recommended you go throw a 16384x16384 movie at Unity right away: there are issues that need to be fixed (I tried…). But you’ll be able to import 8K content (7680 × 4320) without trouble and get a feel whether the performance level is up to expectations.

This brings us to VLC: the reason it can sidestep all this complexity is by not honouring software patents/licenses. So VLC likely has its own implementation of all the codec it supports, making things quite simple for users, but at the cost a legal vulnerability that is not appropriate for Unity. Note that I’m not a lawyer; this is just my programmer-level understanding of the ecosystem.

Not sure if any of this is useful, but I hope it’s at least somewhat interesting!

Dominique

2 Likes
6

Thankyou. This is useful. I am hoping to see AV1, vp9, and possibly h266 implemented in the near future. If there is any news in this front please let me know. Thanks.