Peer to Peer Video/audio for iOS app development

[kevm123]

Hi,

Query

• I am searching for a solution to implement peer to peer video calling in unity for an iOS app which I am developing. I hope to be transmitting data from the front and maybe rear facing cameras as well as the player prefab or mesh data to the other device in real-time.

Factors to consider:

• I have looked into Agora but I feel that what I need can potentially be achieved with a simpler solution without the need for third party costs (also I’m unsure if Agora can support simultaneous video feed from both front and rear facing cameras)
• I had a look into WebRTC but I’m unsure whether it can achieve my technical requirements

Thanks for any assistance

[MarkMcCorkle]

I’d actually start with recommending that you do use something like Agora or another platform + SDK that does the heavy lifting of video capture, call negotiation, packet transfer, reconnections, stream re-coding, encryption, etc, etc, etc. There is a TON of work to do just to make a video (and audio) feed go from one device to another one, particularly when you can’t control the devices specs, the network, and the display size on both sides.

If it was me, I’d use a third party sdk for the MvP, find out if I can get enough potential users really excited and using the app enough that I start hitting that 10,000 min/mth free cap, and then I’d take the time to figure out how I want to keep going with a business model that can afford to pay the third party fees. No matter what sdk+platform I’d recommend, it is going to be way cheaper that the number of hours you’d have to spend with an experienced Unity C# developer to build that same tech.

Oh, and as far as webRTC is concerned, it does handle a lot of the video and audio frame stuff, but it doesn’t (or at least it didn’t last I checked) handle all of the user to user routing, graceful degrading, and graceful reconnections. You will have to build a lot of code in Unity and on your server side yourself, and there are going to be a huge number of edge cases that are going to hurt your user experience and you are going to spend way too much time being frustrated with trying to debug that during your MvP, which is why I suggest seriously just going with Agora.

[kevm123]

@BetaMark

I’d actually start with recommending that you do use something like Agora or another platform + SDK that does the heavy lifting of video capture, call negotiation, packet transfer, reconnections, stream re-coding, encryption, etc, etc, etc. There is a TON of work to do just to make a video (and audio) feed go from one device to another one, particularly when you can’t control the devices specs, the network, and the display size on both sides.

If it was me, I’d use a third party sdk for the MvP, find out if I can get enough potential users really excited and using the app enough that I start hitting that 10,000 min/mth free cap, and then I’d take the time to figure out how I want to keep going with a business model that can afford to pay the third party fees. No matter what sdk+platform I’d recommend, it is going to be way cheaper that the number of hours you’d have to spend with an experienced Unity C# developer to build that same tech.

Oh, and as far as webRTC is concerned, it does handle a lot of the video and audio frame stuff, but it doesn’t (or at least it didn’t last I checked) handle all of the user to user routing, graceful degrading, and graceful reconnections. You will have to build a lot of code in Unity and on your server side yourself, and there are going to be a huge number of edge cases that are going to hurt your user experience and you are going to spend way too much time being frustrated with trying to debug that during your MvP, which is why I suggest seriously just going with Agora.

Thanks for the thorough response! really appreciate it

[joel-agora_io]

Hey @kevm123 , how have things been progressing for you? Were you able to find a solution that you were satisfied with?
If you are still having trouble I’d love to hear more about your use case and potentially connect you with an engineer here at Agora. Cheers!

[joel-agora_io]

@BetaMark

I’d actually start with recommending that you do use something like Agora or another platform + SDK that does the heavy lifting of video capture, call negotiation, packet transfer, reconnections, stream re-coding, encryption, etc, etc, etc. There is a TON of work to do just to make a video (and audio) feed go from one device to another one, particularly when you can’t control the devices specs, the network, and the display size on both sides.

If it was me, I’d use a third party sdk for the MvP, find out if I can get enough potential users really excited and using the app enough that I start hitting that 10,000 min/mth free cap, and then I’d take the time to figure out how I want to keep going with a business model that can afford to pay the third party fees. No matter what sdk+platform I’d recommend, it is going to be way cheaper that the number of hours you’d have to spend with an experienced Unity C# developer to build that same tech.

Oh, and as far as webRTC is concerned, it does handle a lot of the video and audio frame stuff, but it doesn’t (or at least it didn’t last I checked) handle all of the user to user routing, graceful degrading, and graceful reconnections. You will have to build a lot of code in Unity and on your server side yourself, and there are going to be a huge number of edge cases that are going to hurt your user experience and you are going to spend way too much time being frustrated with trying to debug that during your MvP, which is why I suggest seriously just going with Agora.

Thanks for the shout! I’d love to hear more about what you’ve been working on, and how your experience with Agora has been so far.
Let me know if I can help or support you in any way.
Cheers.