Hello, I just started using Vivox in Unity today, and so far it has been working flawlessly. I have two questions however.
I’ve seen that it is possible to apply filters to a participant’s received audio using taps. However, is it possible to apply filters to the microphone audio before it is sent to the server? I want to be able to change the pitch of my voice, specifically. My application does not have any kind of state syncing outside of what Vivox provides, so I cannot tell other participants to edit the audio they receive from me (unless I use hacky ways like sending text messages, which I wouldn’t like to do). I’ve seen the capture source tap, and while the documentation is not really clear (it only mentions the existence of it), it seems to give read-only access to the audio, but cannot modify it.
Does the spatializer used in 3D position channels have Doppler effect? The application I’m building involves players “moving” really fast, so being able to configure the doppler behaviour would be pretty useful.
I had to switch to non-positional voice channels because I couldn’t find a way to apply doppler nor pitch effects as decribed in the opening post. I finally used extra state syncing to tell other players info about my voice.
After doing all of this, I realised from another thread (Proper Use of Vivox Participant Tap (Audio Source)) that doppler effect is just not supported, as trying to use it causes the voice to have “gaps”.
Update 2:
Of course “pitch” is not supported by vivox, all it does is change the speed of the clip instead of doing proper pitch shifting. I guess I’ll have to use an audio mixer with the pitch shifter effect, and calculate the doppler effect myself.
We developed a “Capture Sink” audio tap that allows for manipulating capture audio before transmission but we didn’t release it due to it causing side effects across all of Unity audio. We are working on removing the limitation that caused the side effect.
The Vivox spatializer does not have Doppler effect or propagation delay. Without propagation delay, a pitch-shifting effect that doesn’t change the playback speed is required to avoid breaking causality when emulating Doppler effects as a listener and sound source approach each other. That is so because we are streaming real-time voice and not synthesizing or looping audio from disk that can potentially be sped up faster than real-time.
Good to hear that you can work around the absence of capture audio manipulation with the signalling for receiving clients to apply the effects at render. Sorry that it comes to that for now!
In case it is useful for anyone, here is the source code of my application. I’m calculating doppler manually, as well as using an audio mixer for pitch shifting: