Modulating audio files created by Prime31's Audio Recorder

I’m currently using Prime31’s Audio Recorder plugin for a project that I’m doing. I have figured out how to retrieve the audio file that is created by the audio recorder, but I can’t figure out how to pass this file into Unity as an Audio clip. I realize that the audio recording is being created on the iPhone itself and that Unity is not handling it, but isn’t there a way to modulate this sound file using Unity’s components? Any help would be greatly appreciated.

Bump, same question

I don’t believe Unity can see those files that the plug in writes. That’s why the plug in has its own playback method.

Hmm… want to record audio and them distort it… most like the Talking Tom/Hippo/Rex app

There must be a way to take this sound, its a little useless if you can’t acess the recorded audio

Dig into the plug in source. I bet you could alter the file in ObjectiveC.

This is the main problem hehe, i only know C#/C++ and not really familiarized to iStuff hardcode programing, in other words, i just let Unity do the work… :frowning:

Well unless you’re ready to do a little work, I don’t think it is possible. Have you asked Prime31 directly?

No, i hope he will answer this thread anytime,

This is not urgent, my boss first wanted a voice recon for some basic words like “sleep” or “walk”, there is a thread abut this, sadly with no replies, and now its just a talk and repeat, but the voice must be altered in some way…

I just implemented vowel recognition in an app so that a 3D model’s mouth roughly matches spoken audio. It’s not perfect but it’s better than just moving the mouth up and down based on audio volume.

If you want to do speech recognition on the iPhone, I would look at Redirecting...

Of course, that is an Objective C project so unless you make a plug in, it won’t be of much use in Unity.

Speech Recognition is a complex topic and I don’t know of any plug and play solutions.

@Paulo, the audio player used by the AudioRecorder and AudioPlayer Plugin is not low level enough to offer pitch support. I have a bug open with Unity about audio files that are loaded with the WWW class not working. Once they fix that, you will be able to record the files with the plugin then play them back with Unity’s FMOD.

That’s great news.

I think he wants more than just pitch support. It would be nice to be able to process the audio in real time or at least be able to read the file so you could read the data, process and write the result to a new file which you then could play back.

@kenlem - how were you able to implement the vowel recognition? We implemented a pretty complex solution to do this (using something like FacePoser to extract phonemes), then parsing through those XML files to generate all of the Animation files for different 3D character models. It was really complex and not very scalable, so I’d be curious if you had any ideas for a better way to do this, using OpenEars or anything like it.

My method is extremely simple but the results are fine for what I needed.

All I did was generate 5-6 wave files each with a vowel sound. These are named based on the vowel contained. I have script in the editor that then goes through those files, plays them and does an FFT. I then create a finger print for each file based on the ratios of F1, F2 and F3 and store those finger prints.

On playback of an audio file, I do an FFT as the audio plays, create a finger print of the currently playing audio and compare it to the 5 or 6 fingerprints I previously created. I chose the mouth shape based on the closest match. You couldn’t lip read from it but it sure nails the mouth positions to the sounds that would be obviously wrong.

It helps to create the finger prints from the same speaker as the playing audio. Once those finger prints are created, it can then playback any audio without any sort of hand editing.