Hello Unity community.
I’m excited to present to you Sap - Spectral Audio Play. I have currently 12 years of Unity experience under my belt as a game developer, technical artist, but most of all, researching the domain of frequency analysis and audio-visualization within Unity. For the last few years I’ve been working solo on developing this project. I’ve build a complete custom audio analysis pipeline, and custom editor using IMGUI. In the video you can see the latest development updates, and the editor in action.
Spectral Audio Play - Feature Highlights
- Modular audio analysis pipeline — Custom chains using Source, Spectrum, Gate, and Signal
- Windows system audio capture — Analyze live system output without virtual cables
- Microphone & device input — Use any available input device for real-time analysis
- Low-latency live audio processing — Designed for responsive, performance-oriented workflows
- AudioSource & AudioClip support — Analyze Unity audio sources and clips directly
- High-resolution FFT analysis — Configurable FFT sizes for detailed frequency detection
- Perceptual frequency band generation — Linear, logarithmic, and MIDI-based band mapping
- Dynamic spectrum resizing — Change band counts and ranges at runtime
- Advanced normalization & scaling — Peak, average, and perceptual scaling strategies
- Phase-aware spectral data — Access amplitude and phase per frequency band
- Per-band gating system — Shape and filter spectral energy over time
- Temporal signal conditioning — Attack, decay, smoothing, and history-based processing
- Signal generation from spectrum data — Convert frequency energy into control signals or sound
- Experimental audio resynthesis — Hear spectrum data re-synthesized in real time
- Modular signal routing — Connect signals to any compatible visual or system component
- Preset-based workflows — Save and load complete setups, including structure and settings
- Editor-friendly modular management — Add, remove, clone, and enable modules non-destructively
- Designed for scalability — Supports both small experiments and large modular setups
- Real-time safe DSP design — Minimal allocations and audio-thread–safe processing
- Tier-based resolution scaling (12 / 21 / 42) — Different resolution envelopes without feature lockout
- Experimental high-precision analysis — Up to 32k FFT for live streams (hardware dependent)
Sap is estimated to release on April 12th, through Patreon, by monthly subscription. You can find more info on the project at Spectral Audio Play - Peer Play | Patreon. After atleast a year of further feature development and stability updates, I aspire to release it as a one-time purchase asset on the Asset Store.
Please let me know if you have any questions, suggestions, feedback!