I'm working on a Rhythm-sidescroller. (Basically Mario, but you're encouraged to keep your actions in time with the music). The scene complexity is pretty low. Most of my objects are flat billboards or cubes. I have very good framerate (100+ in the editor, more in the executible). But periodically the game freezes momentarily, making it difficult for players to keep their actions in time with the rhythm. It also sometimes interrupts them in the middle of an action, and they need to go back and start over.
The one thing my level has that seemed potentially problematic is some blinking lights (they're visual cues to help with the rhythm). I tried removing them. It seemed to help a little but it didn't get rid of the lag completely, and given how simple the rest of the scene is I don't think it should be matter that much.
Does anyone know what sort of things cause intermittent lag spikes but not an overall drop in framerate?
Have you decided for sure that it's your game that's the problem? I know it may sound weird, but you should try creating a quick "dummy" game in a separate Unity project. Write some scripts from scratch, add some very simple movement, stick with primitives for the game objects, maybe throw in a few sound effects.
Only reason I say this is because I've had similar things happen over the years in all sorts of different games. Sometimes it was software running in the background, sometimes I just needed to restart my computer.
If it's definitely your game that's causing it, you might want to investigate all the things that are being loaded on-demand. Does the game skip when your music loops? Does it skip when a sound effect plays for the first time? When something is instantiated for the first time? To test, you could comment out your code that plays sound effects, for instance, and see if not playing sound effects clears up the problem. Or if not playing the music fixes it. Or not instantiating a certain thing. I've also heard that using GameObject.Find too much inside of Update() can cause some trouble. You're unfortunately going to have to go by process of elimination.
As far as Jessy's answer, I would agree if you were running some complex simulation and using Destroy() fifty times a frame (slight exaggeration), but I can't imagine you're destroying much of anything in a musical 2D platformer.
Oh well. Report back if you figure out anything solid and maybe one of us can help you further.