WebGL and Microphone

Unity Engine
,
1

Hi,

I’m testing a working audio project under WebGL that works in editor but when I try and build I get:

error CS0103: The name `Microphone’ does not exist in the current context

I thought microphone was in WebGL?

EDIT: My mistake, it’s not!

http://blogs.unity3d.com/2014/04/29/on-the-future-of-web-publishing-in-unity/

2

Can anyone from Unity say if microphone access is likely to be added in the 5.x cycle? (It’s vague in that blog.)

3

Well, it is on our list but the priority is not high at all.

4

Is there an update to this in 2015. Finishing a project that requires students to email recordings to their teacher but cannot find a way to access the mic in version 5 when building to webgl.

5

If you need it in a hurry, and can invest some time and effort, you could make a jslib plugin to capture microphone input and present it to your C# code. It is not completely trivial but it’s also not too hard if you approach it the right way. The biggest step is finding out how to do it in Javascript in the first place. So long as you can get your Javascript to record the audio data into a buffer, it is pretty simple to present that buffer to C# code.

Here’s an example of getting live audio data and analyzing the data in Javascript, which surely involves buffering it in memory at some point: GitHub - cwilso/PitchDetect: Pitch detection in Web Audio using autocorrelation

6

There is no timeline associated with the mic support.
So if you really need it now go the way pointed out by gfoot.
Sorry.

7

@alanbrahmrowe : I had a play with Javascript microphone support yesterday and it was pretty easy to capture the audio stream. I could put a Unity wrapper around it fairly easily if you are still interested.

2 Likes
8

@gfoot : I too would be very interested in this.

9

Here is a microphone plugin then: http://www.gfootweb.webspace.virginmedia.com/MicrophoneWebGL.unitypackage

It is crude and basic. Look at TestMicrophoneWebGL.cs for a usage example.

You can also try out that example here: http://www.gfootweb.webspace.virginmedia.com/TestMicrophoneWebGL/

Note that the demo only works on Firefox due to limitations in Unity’s audio playback code. The microphone side of things works fine on both - at least, I see sensible-looking data even on Chrome, I just can’t get it into an audioclip to actually play it back again.

As shown in the example, you should call Init(), then call PollInit() until it stops saying “pending”. Hopefully it then says “ready”, in which case you can carry on; otherwise it will say an error of some sort. One reason for the polling is that the user will be prompted to allow access to the microphone, so we need to wait for that.

Once it is initialized you can call Start() to start recording and Stop() to stop recording. During recording, the plugin buffers up the microphone data in units of whatever buffer size you passed into Init(). You can query how many buffers it is holding using GetNumBuffers(), and you can pop the next buffer using GetBuffer().

These functions should work fine both during recording and after stopping, and you could in theory record indefinitely, so long as you consume the buffers. If there are no pending buffers then GetBuffer() also returns false, so you could just poll it from your Update() function and ignore GetNumBuffers.

2 Likes
10

Awesome, thank you so much!

11

First of all, thanks gfoot for writing microphone wrapper for Firefox.
I am very glad that now we can use microphone with WebGL.

Can you help me to figure out what kind of data it is and how to play it back again?

12

Nice work gfoot!

13

It’s just regular audio data - there are some options in the javascript to choose the format I think, when the microphone is initialized.

The problem appeared to be that Chrome’s Javascript interface for audio playback didn’t support streaming in data like this on the fly, at least not through the interface that Unity was using to access it. I don’t know whether anything has changed since then.

14

Thatnk you very much =)

15

Hi gfoot,
I noticed that when playing the recording in Unity it changes pitch of audio to half tone.
E → D#
D# ->D
D → C#
And so on.

Do you know what can cause this problem?

16
17

Thank you @gfoot for sharing.

I adapted the plugin to simply get microphone volume instead of creating a clip. I can share if anyone is interested. It was working great for me on Unity 5.1.1, except for occasional instability in Chrome.

Building from any version after 5.1.1 results in the Init never completing on any browser. Anyone have an idea what changed in the WebGL preview to affect this plugin?

18

I found this same issue and have posted a bug / request (it may not actually be a bug at all as I dont really understand the consequences and resons yet). However I do have some insights related to pointer and buffer management - it seems if you change the 2 pointer shifts as follows in these 2 functions, it will revert to working state (seems the pointers are now getting shifted unnecessarily twice now sometime between 5.1.1 and 5.1.3)

MicrophoneWebGL_PollInit: function(resultPtr, resultMaxLength)
{
var sliceLen = Math.max(0, resultMaxLength/2 - 1);
var str = microphoneWorker.initResult.slice( 0, sliceLen );
//stringToUTF16(str, HEAPU32[resultPtr>>2], resultMaxLength); - removed
stringToUTF16(str, resultPtr, resultMaxLength); // added
},

and

MicrophoneWebGL_GetBuffer: function(bufferPtr)
{
if (microphoneWorker.buffers.length == 0)
{
return false;
}
//HEAPF32.set(microphoneWorker.buffers.shift(), HEAPU32[bufferPtr>>2]>>2); - removed
HEAPF32.set( microphoneWorker.buffers.shift(), bufferPtr>>2 ); .// added
return true;
},

1 Like
19

Bump 2016. Bump 5.3.4. Microphone is still not exposed natively.

I don’t even see it on the roadmap.

Microphone support in WebGL is kind of important for in-game chat and of course voice-commands!

20

@theylovegames This thread shows a workaround to use the Microphone in WebGL today, so this should not prevent you from making anything. We cannot currently make Unity’s built-in Microphone class work in WebGL, because the WebGL audio backend can not be made not match to it’s usage, and I don’t want an API which works only somewhat like on other platforms, that would create too much confusion. Once we get FMOD to work on WebGL (which would require threads) this situation may change, but that is still going to take a while.

1 Like