Extract BitStream from file

[Dark-Protocol]

Hello everyone. Several months ago I posted a topic about reading external audio files during runtime. Helpful people from the forum helped me solve my problem partially but after all this time I’m still struggling over something. I needed to be able to import mp3 files in the game during runtime. I know this can’t be done with Unity only so people linked me to some external libraries for mp3 decoding and encoding. I coudn’t get the libraries to work. As soon as I imported the files, the console said that I’m missing Assembly references so I had to do a huge search to find propper DLL files and even after that, the libraries didn’t work and gave all sorts of errors. The libraries I’m talking about are mp3sharp and nAudio. Both give common errors and I cannot use them.

I’m currently using the WWW class to import ogg and wav files. I want to completely replace this method and not use the WWW class. I couldn’t find any topics about audio decoding. I guess nobody’s doing this or if someone has successfully managed to achieve something like this, he’s keeping it in secret I even wrote in sites not related to Unity about audio libraries that will enable me to decode an audio file but nobody has answered.

So does anyone know any library that will work with Unity and enable me to decode an mp3 file (preferably ogg and wav files too)?

Cheers!

[Dark-Protocol]

Alrightie. I found a new library - BASS from what I see it’s just what I need but of course it won’t run in Unity. It can’t find the BASS namespace and it throws a “missing assembly reference” error. This or similar things happen to all libraries I try. I’m so close to having external audio decoding in our game and yet so far away.

Can somebody be so kind and try to import it and use it in Unity ? I’m curious if somebody else, more experienced will manage to get it to work

[Dark-Protocol]

Ah, after a day of hard work trying to get this to work I gave up. It’s pointless. So far I’ve tried mp3sharp, nAudio and Bass and both give so many errors that I am unable to fix. I’ve lost so much time on trying to get a library to work. I’m starting to think this is completely impossible. I mean, I couldn’t find anyone else in UnityAnswers or in the forums who’s even trying to do such a thing. That’s pretty sad. I also searched for Unity audio-based projects that eventually have mp3 import but again, found nothing. Maybe after all mp3 decoding is impossible on Unity PC ?

I can’t give up. We’ve gone too far with our project and we have to finnish it. At least can somebody tell me who can I ask about this ? I am even willing to pay for this information.

[Dark-Protocol]

Alright. Still no luck on this!

I found this topic: http://answers.unity3d.com/questions/271537/loading-audio-files-at-runtime-for-playback-and-pr.html

Those guys got it to work. Nobody answered them but they still got it to work! So I guess it IS possible!
I have no idea what are they talking about though. What are those C# wrappers ?

[Dark-Protocol]

Still no luck and I haven’t stopped trying! Another problem with the built in WWW method is that I cannot read the PCM data from the file after I have loaded it! It will play, but I cannot know what will come next in the track and I also need that feature. From what I read this seems to be a bug in Unity.

I would very very very much appreciate it if someone sheds some light on the problem after all this time!

[Dark-Protocol]

Happy new year!

I know that I have a ton of posts now and I apologize for that but I’m still hoping that someone will be able to help me on this.
I even looked up the asset store, found several audio managers and things like that but nothing that will import audio during runtime.
If only someone could give me some direction what to read so I’ll eventually be able to import a library in the future…

Our game literally runs on music so if we don’t get this to work properly we cannot release the game

[AndyMartin458_1]

I also spent a lot of time trying to use each of those libraries (NAudio, BASS) because WWW never succeeded in doing what I wanted. With a lot of searching and tinkering I managed to get both of those to work within Unity. Although the real purpose was to have something work on Mac and Windows, I only managed to get them to work on Windows.

That being said, I’ve recently come across the full version of FMod that was wrapped for use with Unity by SqaureTangle. I’d highly suggest that you look into it. http://www.squaretangle.com/FMODUnity.html

[AndyMartin458_1]

You were on my mind today because I’ve been having success with fmod. Were you able to get results with it?

[Dark-Protocol]

Yeeey! Finaly someone replied. Sorry it took me so long to reply, but I had almost given up hope for this one.

So I got the Fmod library and played a little with it. I still can’t figure out how can I use it to import files during runtime though. I’m currently looking for documentation on it but I can’t find what I need. Maybe you know more about it ?

Thanks for yhe reply, this brings a lot of hope to me and made my day!

[AndyMartin458_1]

You can place the files in a folder, and then you can load and play them

result = system.createSound("Assets/MySounds/myclip.mp3", FMOD.MODE.HARDWARE, ref sound3);

If you go to http://fmod.org you can learn all you need to know. It’s going to take some work to get going, but there are sample C# projects that work similarly to how you will need to do it in Unity. Keep in mind though that you won’t want to use any Windows forms, but rather you’ll want to use Unity’s GUI. Good luck, and you can find more info here on the forums. http://www.fmod.org/forum/viewforum.php?f=7

[dave_mm0]

Hey,
Im trying to do the same thing as Dark Protocol. I got fmod to play my audio correctly, but I had already written code that makes use of AudioSource AudioClip. Whats the best way to transform a FMOD.System.Sound into an AudioClip? I cant seem to find any doc on FMOD.Sound.getRaw() (I downloaded the fmodex api), but I assume it returns the same format as readData(). So I end up with an IntPtr that I need to transform into a float array, so I can use audioClip.setData right? I want to iterate through the data and divide each value by MAX_INT so I can get a float array form the data. Im not too familiar with C#, but I dont have the length of the int array pointed by IntPTR, so how can I iterate through it? Im a bit lost at this point, so any help is very much appreciated!

IntPtr ptr = currentTrack.getRaw();
uint songLength = 0;
currentTrack.getLength(ref songLength,FMOD.TIMEUNIT.RAWBYTES);
float[ ] floatArray = new float[songLength];
for(int i = 0; i < songLength; i++){
floatArray = ???; /This is where I want to divide/

• }*

• audio.clip.SetData(floatArray, 0);*

[Dark-Protocol]

Thanks, Andy. I have been wandering around fmod’s site for a while and I cannot seem to find any documentation about it. Google doesn’t find anything either. How do you know about createSound() and where can I read about getRaw() ? I need to do the same as dave_mm0 does, but I don’t get where to start reading about this…

[dave_mm0]

Hey Dark, to access the fmod documentation, you need to download the fmodex API ( http://www.fmod.org/fmod-downloads.html ). Install it, and you will find the doc in the FMOD SoundSystem\FMOD Programmers API Windows\documentation folder (fmodex.chm). I got everything wrking, I ended up ditching the entire Unity3d sound engine and adopting fmod, since I already have experience with it. One thing though, when building your game, unity will not copy the fmodex.dll and fmod_event.dll to your games root folder, so you have to do that manually or it won’t initialize properly.

[Dark-Protocol]

Oh! Thank you so much! I’ll be looking at the documentation shortly after I download it. I need to do kinda the same thing as you do, so if I find out something, I’ll share it!

[Dark-Protocol]

So I’ve been struggling with this for a while now. After I read the docs I still can’t understand some things.

I need to make a function that will read an audio file and convert it to PCM so I can use it to insert it vie AudioClip.SetData.

So I use FMOD.System.createSound() as AndyMartin458 suggested:

function FileToPCM(filepath : String){
	if(File.Exists(filepath)){
		var fmodSys : FMOD.System;
		var fmodSound : FMOD.Sound;
		fmodSys.createSound(filepath, FMOD.MODE.HARDWARE, fmodSound);
		
	}
}

So I’m creating a new System object, creating a new Sound object and then I use createSound to get the data of the file into the Sound object, right ?

Well then what ? How can I access the PCM data of the file? I cannot seem to find any function from the documentation that will be able to do this. There’s getLength, getFormat, getName, but I can’t find something that will retreive the PCM data in an array.

Dave, how did you get your PCM data out of the sounds ?

[Dark-Protocol]

Okay, so I found out how to get the data out of a sound, however I ran into a problem.

The function will output “ERR_INVALID_PARAM” in the console. So that means there’s something wrong with createSound.

But what could it be ?

static function FileToPCM(filepath : String){
	if(File.Exists(filepath)){
		var fmodSys = new FMOD.System();
		var fmodSound = new FMOD.Sound();
		Debug.Log(filepath);
		var soundEx = new FMOD.CREATESOUNDEXINFO();
		var result = fmodSys.createSound(filepath, FMOD.MODE.HARDWARE, soundEx, fmodSound);
		var data : System.IntPtr = fmodSound.getRaw();
		Debug.Log(result);
	}
}

[dave_mm0]

Hey,
I noodled around a bit with the same approach you did, that is trying to convert int array from fmod to a float array that you can then assign to an AudioClip, but I quickly abandoned that trail.

Instead, I converted my old code that used AudioClips to use fmod directly. For example, I was using audioSource.frequency to perform the fft, instead I just use fmod.channel.getSpectrum, which does the same thing. A lot easier, and fmod lets you control the audio a lot more, at the cost of a bit of overhead code and initiation. I have already worked with native fmod, so not too much of a obstacle for me.

I suggest you forget about the built in unity audio engine, and use fmod. If you end up converting to PCM, let me know how you did it. I think the format is the same, so all you would have to do is convert the int[ ] from getRaw() (ranging from 0 to maxINT) to a float[ ] (ranging betweeen 0.0 and 1.0) and assign it to your AudioClip data… but I might be completely off track too.
Plus, if you use fmod directly, you won’t waste any precious processing converting the sound.

As for your code snippet, you don’t need a SOUDNEXINFO structure if your not using it:
FMOD.Sound currentTrack = null;
FMOD.RESULT result = system.createSound(path, FMOD.MODE.HARDWARE, ref currentTrack);

That works for me, so try it and let me know! I will try and monitor more closely this thread to help you out, I’ve spent the last week working on my beat extraction algos

[Dark-Protocol]

Thank you for sticking with me!
I will follow your advice and use Fmod for playing music in the game, I’ll get used to it eventually . Now for my function. I still can’t get createSound to work properly. As a matter of fact, my first attempt didn’t include the SOUNDEXINFO parameter, however because the function was giving me the same error (ERR_INVALID_PARAM), I decided that adding it might work, but it didn’t.

Soooo, I simplified it a little, removed the soundEx parameter:

static function CreateSound(filepath : String){
	if(File.Exists(filepath)){
		var fmodSys = new FMOD.System();
		var fmodSound = new FMOD.Sound();
		Debug.Log(filepath);
		var result : FMOD.RESULT = fmodSys.createSound(filepath, FMOD.MODE.HARDWARE, fmodSound);
		Debug.Log(result);
	}
}

…but it still outputs ERR_INVALID_PARAM. I thought it might be because I’m using Javascript to write the function, but I wrote it in C# and I got the same error.

Here’s the function in C#, it actually outputs an error, not a result, saying “error”:

 static public void Create(string filepath){
		if(File.Exists(filepath)){
			FMOD.System fmodSys = null;
			FMOD.Sound fmodSound = null;
			Debug.Log(filepath);
			FMOD.RESULT result = fmodSys.createSound(filepath, FMOD.MODE.HARDWARE,ref fmodSound);
			Debug.Log(result);
		}
	}

The console says: NullReferenceException: Object reference not set to an instance of an object and links to this line:

FMOD.RESULT result = fmodSys.createSound(filepath, FMOD.MODE.HARDWARE,ref fmodSound);

So yeah, I can’t even get to using createSound properly yet Can you see something wrong with my code ?

[dave_mm0]

The error indicates that you are using something that doesn’t exist - wasn’t initialized. This usually happens when you try unsing an object set to null.
If you look veeeerrrryyy closely to your code, you see that you assign fmodSys to null, then immediately try to call one of its functions. This causes your crash.

You need to create and initialize your fmodSys object! There are many examples of how to do this, but here’s my code to create and initialize the FMOD system (here called system, not fmodSys), in C#:

uint            version = 0;
	    FMOD.RESULT     result;
	
		//Create an FMOD System object            
	    result = FMOD.Factory.System_Create(ref system);
	    ERRCHECK(result);
		//Check FMOD Version
	    result = system.getVersion(ref version);
	    ERRCHECK(result);
	    if (version < FMOD.VERSION.number){
	        Debug.Log("Error!  You are using an old version of FMOD " + version.ToString("X") + ".  This program requires " + FMOD.VERSION.number.ToString("X") + ".");
	    }
		//Initialize the FMOD system object
	    result = system.init(32, FMOD.INITFLAG.NORMAL, (IntPtr)null);
	    ERRCHECK(result);
	    if (result == FMOD.RESULT.OK){
	        Debug.Log("FMOD init! " + result );
	    }

If you are having trouble re-writing to javascript, I can post the translation, but it’s a good exercise After this, you can then use fmodSys to create a sound! I don’t see anything wrong with your createSound() call.

As a side note, try to avoid static functions. You can initialize everything in the start() function and nest your Create() insidethe object. If you need to access the fmodSys from anywhere else, link to your parent object, or if you need to access it from pretty much everywhere, you can make a Singleton (wikipedia singleton).

[Dark-Protocol]

Nah, I can write in both C# and JS. It just happens that the Java-like syntax is more appealing to me and I write most parts of my code in JS.
So yeah, that helped and I finally got it working! Yey! And yes, I’ll define only one system object in the beginning, this function is just for testing purposes only.

The only problem now is that the OGG format is not supported. So I’m kind of infront of a dilemma should I stick to the AudioSource component or should I replace it completely with Fmod as you did. After all it’s better if the game can read mp3 files and not ogg files instead of ogg files and not mp3. And if I stick with fmod, I’m not sure if AudioSource.GetOutputData and AudioSource.GetSpecturm data have corresponding functions in fmod, but I’ll play with it a little and see. For now I’ve been looking at System::getWaveData and System::getSpectrum which I think do the exact same thing. If that’s the case, then I’ll just have to use several systems - one for the music only, one for effects… etc.

Anyway, if I decide to stick with the AudioSource and find a way to convert the IntPtr data to a float array, I’ll write you a PM to let you know how I did it.

Thank you! You and all the others who answered have been of huge help to me, my team and the project we are working on! I really appreciate all the help from you.

When the game gets to see the world, I’m totally buying you all a beer!