AudioStreamNetCode | network audio based on NetCode for GameObjects

Asset provides client/server audio comms over network using Unity Transport as part of NetCode for GameObjects
[ Networking & Netcode Software Solution | Unity
About Netcode for GameObjects | Netcode for GameObjects | 1.0.2 ]

It’s currently based on latest release of NetCode and uses Unity Transport only - | this means it doesn’t support Relay and is thus OOTB suited only for LAN based solutions. |
/ NetCode supports 3rd party transports so it’s possible to substitute for it.

It can stream any AudioSource or AudioListener to all connected clients and clients are allowed to stream back into server mix with their own audio.
(Audio is encoded using OPUS codec - this also means max. 2 channels per source)

Network throughput for single audio source is up to ~30 kB/sec depending on settings.

Please see demos here:
Windows x64 | macOS | Android/ChromeOS

Please see latest documentation here
and for more information and demo builds Asset Store page

Asset is summitted, hopefully in a month or so it will be available.
I set 50% discount for two weeks after launch.

I will be taking reasonable requests after the first release.

~~

1.0 o92o22, Xcities

  • initial release
  • client/server audio mixer


live (-:

Hi, Can I use this as voice communication solution for netcode for gameobjects based unity apps?

Yes, that’s its purpose
It will work as is on local network (LAN)

Internet streaming via Unity Relay is not supported - Unity explicitly prohibits transfer of audio data via their services - since they have their own Vivox solution (which is technically based on the same OPUS codec btw)

  • it’s possible to replace relay (internet transport) with other 3rd party service, but this is not implemented yet

Hope this answers the question! feel free to ask if something’s not clear still

i buyed the asset, it is very good , but i want capture microphone in the clients and send it to the other clients. the mixer in the server is no good for this . Is this posible?

Hi @MigolasUnity thanks !
I think there are currently two pieces missing:
– transmission of client Unity microphone capture (in demo)
Unity mic can be scripted by user and otherwise should be the same as capturing AudioSource in scene as it is now
But adding this to a demo scene makes sense

and
– when a client is transmitting its own mic input/audio that shouldn’t be transmitted back to it from server mix
Is that what you think the problem is ?
I’ll add a setting for this and moreover this behaviour should be even the default I think

If you want, send me your invoice # to PM, I’ll make sure to let you know about update as soon as it is ready / please do note that it will take some time though /

Thanks once more!

Just submitted an update, it could be online on the store shortly:

==================================================================
1.1 1o2o22

  • demo scene: + audio visualization textures on main listener
    ==================================================================
    1.2 o82o23
  • Unity NetCode ~1.5.x, requires min. Unity 2021.3 LTS
    (this should also fix macOS/M1+ unability to bind server at times)
  • clients transmission (when clients transmit back to server/host):
  • fixed/updated server’s client mixer - each client receives its own specific audio w/o its own audio included
  • added possibility for each client to transmit Unity Microphone
  • demo scene:
  • added Unity Microphone transmission option to clients

this should now allow complete ‘chat room’ like scenes to be built, hope it can useful ~!

super cool! does it work with iOS too?

Hi, yes, everywhere where NetCode runs / except WebGL currently /

-** December '23 SALE **-

  • For two weeks - until about ~28th - the asset is 40% OFF

Don’t forget to download demos and enjoy the holidays - \o|

Hi. Thx for your asset. Can you say what latency should I expect If all ist setup correctly, around 20ms?

Hi thanks ! This currently doesn’t use better (2D only) latency I realized - it uses full spatialization only so there’s always additional overhead -
Will get to adding 2D only client after the new year - with an option in demo scene; - but can’t promise any ETA
(as for precise ms I never measured it but it should be in ballpark over localhost interface then)
Hope this answers the question !

managed to prepare an update, but it’s not submitted yet ( demo builds are not yet updated, too)
there was a bug applying volume for clients, and the new “2D mode” will help with latency
it’s still little bit more than OSC used in AudioStream, and this should be around 1oo’s of ms rather than 20ms in most cases most likely

there is one server CPU optimization which will be done later still pending,
and if you were asking about latency mainly because of client’s microphone: this will definitely help, but for better latency than default Unity Microphone, there’s no other way than to use something like AudioStreamInput2D from AudioStream asset
[it should be easier and probably much less work to just use both assets if/when needed compared to submitting a new store package which would integrate these two currently, too, but we’ll see]

thanks && happy new year everyone.

===============================================================================
1.2.1 122o23

Host/Server:

  • added new toggle to AudioStreamNetCodeServer ‘use2DRecevingAudioSources’:

  • when On all receving AudioSources (clients and clients’ audio on server) are in 2D mode which improves latency over network - see Docs for more

  • fixed a bug with ‘All clients mix audio volume/monitoring volume’ where volume/monitoring volume was not applied correctly for clients’ audio mix

  • demo scene:

  • added a toggle ‘Use lower latency mode (2D only receivers)’ for the above which is available before launching host/server

  • this applies only to receivers - original signal can still be spatialized - run at least two separate client instances demos to see how this behaves

update sent for review…

  • please note that demos download links will be updated on the store page only once this is live
    ~~

. it’s working with latest NetCode (currently 2.7.0)
. this requires Unity 6 at least, so i bumped version of the asset to 2.0:

2.0:

  • Unity 6.0 LTS as minimal version together with NetCode 2.7.0 (current latest)
  • Il2CppSetOptionAttribute attributes used for IL2CPP builds are enabled by default - necessary setup steps on how to use them properly were added to initial import README doc
  • fixed small bug w/NetworkManager.Singleton and public server’s Encoder in demo