WebRTC, Render Streaming, SocketIO integration for WebGL Build [Help mee :'( ]

Unity Engine
, , , ,
1

Heya, everyone. I have a conundrum I am trying to resolve with my team regarding Unity WebRTC, Render Streaming and WebGL with the connections happening with SocketIO.

We are building a project that features “cloud gaming” basically a server which will handle game logic and a unity instance handling HDRP graphics with Real Time Reflections will be posted on a Google Cloud Server with a GPU.

Our backend server will work with node.js + express, prisma, PostgreSQL and SocketIO

The Players will connect to 11 player lobby clones of this setup via WebGL, and are able to play normally, the graphics are streamed to the WebGL build and the Logic is handled by the server (since I want the WebGL build to be as light as possible, only relying on connection speed).

The problem I am encountering is that Neither UnityRenderStreaming, nor WebRTC support WebGL and I was not able to build nor would my tests work.

What I have understood and developed is a jslib that runs everything in javascript and is being referenced by C# in the controller script (I have done that for WebRTC only up till now). What I am stuck on is how to render the stream of the unity instance on the cloud to the player as I can get connections but no video playback.

Now why would you want to stream it to the player via WebGL you may ask and not install the game to their machine/android device in order to save this issue?

Sadly the game must be made on WebGL platform as the leader of the team wants it to be available on any supported browser, and to have no control over the executables, apks, etc.

I would love to discuss this issue with you guys in order to resolve it, since it is an interesting concept and I would like to see it made (Streaming RTX Graphics on WebGL with minimal hardware impact to player sounds fun as heck)

Thanks from beforehand for reading through this, and for any help whatsoever

1 Like
2

Unity used to have a service for this …

What you want to do, if you have the time & budget to implement this yourself (hint: it’s not going to be done and finished within 4 weeks) is to not use Unity as the frontend solution.

Instead, on the client side a thin WebRTC renderer and some input handling runs which passes this information to your server which then plugs this into Unity (don’t ask me how) which then renders and the output is captured by WebRTC and sent back. Add WebSockets to handle your networking needs.

I know this works because these guys have done it for us five years ago, albeit without audio at the time, and it was “sort of” multiuser in that any privileged viewer could also move the mouse and click on buttons.

If you use Unity WebGL as the frontend you’ll be locked into a single-threaded, black-box rendering framework which, as you noticed, doesn’t work with many common web technologies out of the box.

Does that mean you have one server rendering for each player? If so, you could still do the networking within Unity because every player would be remote-controlling their own instance.

Did you run a financial analysis regarding the costs of running a live service at that scale? I mainly mean all the cloud server instances and the traffic they generate. This could easily turn out to be prohibitive in terms of monthly running costs.

1 Like
4

Heya, first of all thank you for all this amazing info, it is helping me out a ton and the things you are saying make total sense.

Thankfully the WebGL client will only handle basic interactability and except from the video stream little byte transfers of data between server + database and itself.

For the 11 players what i mean is that we will have a server instance for one group of players and then if another group wants to create a game it will dynamically make another instance for them, since this is to be used for specific environments the cost of gcloud running with a T4 nvidia gpu is right up our alley.

The only thing players will be able to control is to view statistics and place items on the board.

Hence my solution was based on the game master (player 11) connecting via websockets/socketio to our game server then via webrtc communicating with the 10 players kind of like a host for each “clone” instance.

Thanks again for all the info, i hope my analysis is ok hehe. WebGL is a bit hard in regards to compatibility sadly, but this project continues to be interesting.

If i indeed do find a solution i will be sure to provide a git for any future reference hehe.

1 Like