I am developing a Vision Pro MR app using Unity.
After activating and using the microphone, if I press the Digital Crown button to return to the main menu and then go back to the app, all audio plays at 2 to 3 times the normal speed.
To compare, I built the sample app in the same environment and confirmed that there were no issues, but when I added only the microphone functionality, the same issue occurred. Is there a known solution to this bug?
Here is code about Microphone
private bool _isSpeaking;
public bool isSpeaking { get { return _isSpeaking; } }
private AudioClip _clipRecord;
private int _sampleWindow = 128;
private bool _isInit = false;
private bool _isActive = false;
public float _db;
public void Init()
{
_isActive = false;
if (_isInit == false)
{
if (_device == null) _device = Microphone.devices[0];
WaitForMicToStart().Forget();
}
}
private async UniTaskVoid WaitForMicToStart()
{
await UniTask.Delay(5000);
_clipRecord = Microphone.Start(_device, true, 999, 44100);
_isInit = true;
_isActive = true;
}
void StopMicrophone()
{
Microphone.End(_device);
_isInit = false;
_isActive = false;
}
float LevelMax(AudioClip clip)
{
float levelMax = 0;
float[] waveData = new float[_sampleWindow];
int micPosition = Microphone.GetPosition(null) - (_sampleWindow + 1);
if (micPosition < 0) return 0;
clip.GetData(waveData, micPosition);
// Getting a peak on the last 128 samples
for (int i = 0; i < _sampleWindow; i++)
{
float wavePeak = waveData[i] * waveData[i];
if (levelMax < wavePeak)
{
levelMax = wavePeak;
}
}
return levelMax;
}
float MicrophoneLevelMaxDecibels()
{
float db = 20 * Mathf.Log10(Mathf.Abs(_micLoudness) + 1e-6f);
return db;
}
public void OnUpdate()
{
if (true == _isActive)
{
_micLoudness = LevelMax(_clipRecord);
_micLoudnessinDecibels = MicrophoneLevelMaxDecibels();
_db = _micLoudnessinDecibels;
if (_micLoudnessinDecibels < 1 && _micLoudnessinDecibels > -40f)
{
_isSpeaking = true;
}
else
{
_isSpeaking = false;
}
}
}
public void Clear()
{
StopMicrophone();
}
}
I haven’t seen this issue before. It would be great if you could submit a bug report with the repro case you’ve listed here and let us know the incident number (IN-#####) so that we can investigate.
Thanks for the report! It turns out that this is related to an issue that we’ve had for a while where recording fails to start if you enable the microphone too soon after starting (which I can tell you’re aware of, since your sample code delays before starting). This is an issue specific to immersive spaces, and we believe it’s an issue on Apple’s side. We’ve reproduced it in a non-Unity project and submitted it to them as FB14903015.
The speedup is happening because we’re not shutting down the audio correctly in the cases where we fail to enable the microphone (which we also attempt to do when the app returns to the foreground). We should be able to fix that, but that won’t solve the microphone issue in general. Here’s the workaround I was able to get working: in addition to delaying Microphone.Start on startup, I had to call Microphone.End on losing focus and then do a delayed restart on gaining focus again.
The code looks like this:
using System.Collections;
using UnityEngine;
using Unity.PolySpatial;
public class MicrophonePlayback : MonoBehaviour
{
bool m_Focused;
public void OnWindowEvent(VolumeCamera volumeCamera, VolumeCamera.WindowState windowState)
{
if (m_Focused == windowState.IsFocused)
return;
if ((m_Focused = windowState.IsFocused))
{
StartCoroutine(StartRecordingAfterDelay());
}
else
{
StopAllCoroutines();
Microphone.End("");
}
}
IEnumerator StartRecordingAfterDelay()
{
yield return new WaitForSeconds(0.5f);
var audioSource = GetComponent<AudioSource>();
audioSource.clip = Microphone.Start("", true, 10, 44100);
audioSource.Play();
}
}
To get the OnWindowEvent callback, you need to have a volume camera (unbounded) in your scene and set its On Window Event to call that function on the behavior (MicrophonePlayback, in this case).
Thank you for your answer. But, this still has a problem.
Run the app in Vision Pro → Sound plays normally → Press the Digital Crown to return to the main menu → Select the app again to return → Background music and other sounds play 2 to 3 times faster.
This issue has not been resolved in my case.
Could you submit a complete repro project (ideally, using the workaround above)? Your previous submission only has a single source file, not a complete project that reproduces the issue. I just want to make sure that I can reproduce what you’re seeing exactly.
After that, I implemented the event handling method as follows:
public void OnWindowEvent(VolumeCamera.WindowState windowState)
{
if (focused == windowState.IsFocused)
return;
if (focused = windowState.IsFocused)
{
// code about start mic
}
else
{
// code about stop mic
}
}
I don’t know offhand. If you experience it in Metal mode, I’d suggest filing a bug report with a repro case and letting us know the incident number (IN-#####) so that we can look into it there, too.
I can’t speak to the audio speedup problem, but the mic start problem definitely persists with visionos xr plugin 1.3.1 on VisionOS 2.0.1, where immersive mode is now “Metal” mode. Using Unity 2022.3.47.
I’ll have to get another engineer to do the incident report, but the problem does persist and we’ve repro’d on 3 different AVPs.