Record 90 minute game on devices webcam and broadcast over rtmp

News & General Discussion
1

Hi,

For my project I wanted to live stream my android/ios webcam (front camera) to rtmp. But not just short clips, i want to record a whole game of football (90 minutes) to youtube live at the highest quality possible from the device e.g. android or ios mobile.

I have looked into ffmpeg and used a downloaded demo, this demo achieved this but it is broadcasting at a low resolution, not the top quality the mobile device camera is capable of. Plus there is interruptions with internet connection drops and missed video for the viewers. So increasing the quality wont help me as the device is struggling to keep up at a low resolution.

I also thought about recording the video to the devices storage and at the same time streaming this over rtmp as it is recording. My theory is to record the devices front camera as you would natively but using this as the source for rtmp e.g. mp4, instead of capturing the devices screen using ffmpeg. Essentially the user will then just use Unity app to see the camera for video shooting purposes.

Has anyone have any ideas on the best way to record 90 minute games with good quality playback over rtmp? I just want to see if this is possible or not.

e.g. app already doing this but have no idea how they have approached this, but wanted to re-create this concept in Unity.

thanks

Nick

2

I assume the use case is: anyone with your app would be able to live-stream from his/her mobile and from right in the stadium to Youtube. Is that what you’re trying to do?

First thing that comes to my mind is: nobody will hold their phone up for 90 minutes. You need a stand for that. To get a stand in a stadium, even in a lower league, you’d have to get a (press) permit or something because security won’t let you in with a stand.

Then there’s all the legal implications, it is literally copyright infringement to take videos in any professional sport or practically any paid-admission scenario such as concerts. You can ignore that and leave it up to the users to take this risk, unless they’re supposed to share content on your Youtube channel. But even if they published to their own channel, uploaders may face the video taken down and if that happens repeatedly, they might get banned from YT altogether.

That said, it may still be feasible for amateur sports for instance.

Next up we have connectivity. With the user either in a crowded (possibly enclosed) stadium with thousands of other viewers on the same mobile cell block or sharing the same open wifi, the implication is that each device will get only a small portion of the potential bandwidth.

So even if you had a solution that was able to stream at very high resolution from a phone either over the air (LTE) or using Wifi, that wouldn’t be the experience the end-users might get.

Unless, again, we talk about amateur sports. In that case only a few dozen or perhaps a few hundred viewers will share the same mobile cell block - however, often these sports venues are at more remote locations, possibly meaning less bandwidth to begin with.

With all of that said, you can expect the streaming quality to be limited to begin with. You may be able to get 720p quality streaming on your local wifi but hardly more than that. For instance, the Oculus Quest has such an option, streaming the view to a device on the same wifi. And the last time I tried this, it was okay to watch with frequent drop-outs and glitches but nowhere near watching a Youtube video.

Streaming from mobile without being tasked to do 3d rendering at the same time I would expect to be better quality but it’ll be about the same as what you get from the usual webcam streaming in an online meeting, and in such scenarios, users joining from a mobile device generally have noticably less video/audio quality.

In summary and TL;DR; => it is possible but you may need to lower your expectations about the achievable quality of the video stream regardless of which capture, compression and transport tech you use.

1 Like
3

Yes it would be a live stream directly from their mobile to YouTube and yes it would be on a stand.

I have the rights to do this for my local club at amateur level so probably around 1200 capacity. But before i commit to this project i wanted to explore my options beforehand.

Thanks for all the info, i never thought about the Wi-Fi implications. I just wondered how pixlive was achieving HD footage from a ios webcam. I dont mind researching but only if its worth looking into.

Thanks

4

Glad to hear you have none of the issues I was worrying about! :slight_smile:
As to what pixlive is using … just a hunch, but if pixlive use a common library or protocol something like that could be analyzed with a packet sniffer, I think Wireshark is one of these. Maybe with a little research in that direction you may be able to find out what a specific “marker” in all of that traffic could indicate or even identify a certain tech.

5

I guess my main question is “why use Unity for this and not just OBS?”

1 Like
6

Hi, well mainly as the client doesn’t want to use OBS, I suggested it, but as all i know is Unity I thought I would look into this and explore what is possible.

7

Ok thanks for this, I will take a look into this and see what i can find out. I know Unity is powerful and just using it for a webcam recorder essentially is basic but I see this as a good project and the scope beyond this basic test to apply to other projects makes it worth doing the research.

8

The thing is that this just really isn’t something that Unity as a tool is geared towards, similarly to how you wouldn’t use it to build a website or a spreadsheet app. This really does fall into the domain of streaming software, which is going to have things like proper support for everything you’re looking for built in.

9

Yeah, rather than working around Unity to make this happen, I’d spend a few days getting familiar with a more appropriate toolset and then make it in there.

Though I’d also take a look at using an existing tool instead of writing one.

“Basic” isn’t the issue. It’s designed for a fundamentally different use case. Unity is a game engine, so it’s designed from the ground up for real-time interactive applications. That’s what it’s main loop, event structure, API, core rendering and other fundamental components are designed around. You can make them do other stuff, but why do that when there are plenty of tools designed to suit your use case to begin with?

I’d look into camera streaming libraries for iOS and/or Android (whichever you’re targeting) and see what’s available, then go from there with a native app, or looking into Xamarin or whatever.

10

Why not PixLive itself? Looks pretty good. And why did they say not OBS? I’ve done the exact same thing at a local sporting event but with a webcam, a laptop and OBS. Worked great!

11

yeah I understand your points, but as other have successfully developed this inside of unity I wanted to know personally how it works essentially. Pixlive would be ok but my clients wants to add features (overlays etc) which Pixlive does not support (but can raise a feature request of course). I do get both sides of the argument but after researching I found a few SDK libraries which do this well in Unity e.g. agora.io/unity

I would use Pixlive but as I am researching this topic i personally just wanted to know how an app like this is going about recording and broadcasting. When I find SDK’s I find that user reviews tend to mention that they have good results in Unity Android/IOS app. Then alot of articles/forum posts point to FFMPEG which is why I ended up testing locally on Windows and got the same results (quality etc) inside Unity. Maybe my ffmpeg setup is not optimised (as im not an expert).

Pixlive mentioned on an Android app can achieve HD quality from rtps.

Well looks like im back to researching other approaches. Thanks

12

i’d guess ffmpeg is the best solution, if want to make your own streaming app… (without 3rd party plugins)

it has lots of options/settings to adjust quality/size/compression etc. so could look into those.
or maybe mobile cannot process that much data on time…? Did you test with new ipad devices, they are quite powerful.

And have you compared quality with existing streaming apps, like that obs/streamlabs?

Alternatives:

  • unity has renderstreaming plugin, easy to use, but probably would need server side solution to re-stream into youtube (if unity renderstreaming doesnt support rtmp)

PlanB:

  • Use small laptop + 4k webcam, stream from that (using existing tools, or make your own, easier to do on win/mac)
13

I have not tested the new ipad devices, ill give this a go when i can get my hands on one. Yes I agree that ffmpeg could be the best solution, I mean i bet some of the 3rd party plugins use this. Its going to mean i have to do some learning on the ffmpeg config side.

Ill check this out, I never thought about a server side solution. thanks

No i have not managed to test them all yet, from the ones i have so far I can hardly tell the difference between then personally. I know youtube caps the framerate at 60fps but i know devices will struggle to even get close to this anyway, never mind over rtmp.

14

Does the camera on those devices even capture at that rate?

The Unity approach can work. It’s just a square peg, round hole kind of solution.

15

Apple has had support for 1080p60 since the iPhone 6 and 4K60 since the iPhone 8. Android is much more varied as you would expect from a platform having a wide range of hardware configurations but there are phones that it’s available on like the Pixel 5.

16

Just to add to what everyone said. I would be wary of ffmpeg for any commercial use. It is a truly fantastic tool but their codecs have supposedly patent issues. It is why most apps don’t distribute it and ask the user to install it themselves and it is why we don’t already have it in Unity. FFmpeg License and Legal Considerations

Live streaming is a frequently requested feature, more often on the digital twin side but sometimes for gaming. I know in the digital twin, they use the WebRTC package, but I don’t necessarily think it fits what you want to achieve. As everyone said, iOS and Android natively support live streaming, but I never checked the API for broadcasting. Most mobile phones have encoders, so it is technically possible. WebRTC | WebRTC | 2.4.0-exp.11

2 Likes
17

Ok thanks for this, I was getting conflicting information around various forums about licencing, but I guess my project is in the research stage so not 100% decided upon what to use. I think this information alone may put me off completely from using it.

Not read much about WebRTC but I will look into this also to see if it could work but not hopeful based on the points you made. I think the more I research on this that there is no clean way for streaming to rtps inside Unity for gaming or for my use case. But I have a few more things to look into thanks to everyone on here. I am shocked that Unity doesn’t have an option by now. I guess the lack of support by Unity makes me believe i may have to go down the 3rd party plugins route.