Setting AVAudioSession category, results in Unity player freezing momentarily

I am working on a project that is using a native code plug-in written in Objective-C.

Before an audio file is played, the audio session category is set to mute/pause audio from external apps like Spotify or a podcast app. This works.

AVAudioSession *audioSession = [AVAudioSession sharedInstance];
[audioSession setCategory:AVAudioSessionCategoryPlayback error: nil];

When the audio source in Unity finishes playing, the session category is de-activated:

AVAudioSession *audioSession = [AVAudioSession sharedInstance];
bool success = [audioSession setActive:NO withOptions:AVAudioSessionSetActiveOptionNotifyOthersOnDeactivation error:&error];

And I also needed to call this Unity method from Objective-C after de-activating the session; audio sources don’t play without it.

UnitySetAudioSessionActive(1);

What is happening is there a noticeable freezing / pausing of the Unity player. Which creates a bad user experience. And the AV session code is getting an error:

[AVAudioSession setActive:withOptions:error:]: Deactivating an audio session that has running I/O. All I/O should be stopped or paused prior to deactivating the audio session.

Is there anyway to suspend and or de-activate the audio part of the player to avoid this glitch in Unity? I have tried adding a call to stop method & introducing a delay to see if that makes a difference with no joy.

 audioSource.Stop();

Have you figured this out? I am experiencing the same issue