Unity Render Streaming Status Update (Q4 2022)

Unity Engine
, ,
1

Contributors in 2022
First, We posted a look back at last year 2021 and our plans for 2022 in December 2021 here . In this year as well as last year, we have received various feedback from many developers. We would like to thank all the developers in the below list who have made pull requests to our products in 2022. Thank you developers!

A look back at 2022
Looking back at 2022, the notable work of our team includes as follows.

  • Hardware encoder performance improvements
  • Add NvCodec hardware decoder
  • UX improvements for streaming components
  • Integration of WebRTC with the Unity profiler

Many other modifications have been made to the two packages WebRTC and Unity Render Streaming. Most notably, the Unity Render Streaming 3.1.0-exp.4 version has been upgraded to allow real-time control of streaming in the Unity Editor component. Please see this post for more information about this version change.

A case study
At Unity Sync which is an online event hosted by Unity Japan in October 2022, a case study of Unity Render Streaming was presented by the third party developer. The video is in Japanese, but once you watch it you will get many ideas from it.

1110×633 901 KB
自社システムからUnity Render Streamingに組み替えた | Unity Learning Materials

Plan for 2023
For the WebRTC package, we are beginning preparations to release it as an official package. We have already updated the major version of the package and released it as 3.0.0-pre.1. During the pre-release period, we will focus primarily on improving API deficiencies, bug fixes, and performance improvements. These issues can be viewed on GitHub Issues, and new ones can be added.

We also plan to address the following issues that could not be started in 2022.

Hardware Acceleration Support
We will continue to provide support for hardware codecs. First, there is room for performance improvements in existing video codec implementations. In particular, the CPU load on video decoders for mobile platforms can be significantly improved.

In addition, support for codes of NvCodec (AV1, VP8, VP9), as well as Intel OneVPL, AMD Advanced Media Framework.

Metadata Support
The Encoded Transforms API can be used to add metadata to streaming video (example). This can be used to synchronize video and input. This feature is increasingly requested by developers.

Platform Support
We have received reports from users that Unity Render Streaming does not work on some Linux distributions, possibly because QA has not been performed on Red Hat-Based Linux Distributions. We are planning to support Rocky Linux since Cent OS is already out of support.

Next, I would like to discuss our plans for the Unity Render Streaming package. Currently, this package has some missing features and stability issues, so it will continue to be developed as an experimental package in 2023.

We plan to support the following issues.

Configuration Window and Wizard
The current version of Unity Render Streaming is available by setting up several dedicated components in a scene created by developers, but this process is a bit complicated and inconvenient. Therefore, the next version will provide the ability to configure scenes automatically. Developers can make streaming applications with minimal configuration in the settings window.
Additionally, we will provide a wizard window which will be provided to suggest the best project settings for using Unity Render Streaming.

Command Line Configuration Support
It is inconvenient for users to have to rebuild the application or prepare a UI for changing settings. Therefore, we provide the ability to set configuration items such as signaling server URLs from the command line at application runtime.

Integration with SFU
SFU is a mechanism to deliver streaming to more users, reducing the processing load on the sender side due to the number of viewers (reference).

Providing a way for Unity Render Streaming and SFU to work together to address more use cases.

Sample for Docker Container
A user asks about an efficient and scalable way to use the Unity runtime for streaming applications.

We support the creation of a Docker container to address this issue.

Extensibility of signaling logic
Provide APIs to facilitate user customization of signaling.

Create package for web client modules
We plan to make a package as reusable modules for creating web clients.

Frequently Asked Questions
Many users have asked the following questions.

  • Release date of the officially supported version
  • VR/AR support

Release date of the officially supported version
The WebRTC package is now stable and is being prepared for official release by the end of 2023. The timing for official packaging of Unity Render Streaming has not yet been determined.

VR/AR Support
We have received many bugs with specific VR/AR headsets.

Issues on certain Android SoCs may take longer to be addressed.

In Closing
If you have any questions about our products, please feel free to reply to this thread or create a new thread in the Forum. There are already many threads discussing various topics, so if you search in the forum, you may possibly find what you are looking for.

So I look forward to seeing you all here again next year. Have a great holiday!

2 Likes