Neuromorphic Speech Enhancement with Dual-Branch Spiking Neural Networks

Traditional deep learning models for speech enhancement often require millions of parameters and intensive floating-point operations, making them unsuitable for low-power edge devices. This paper investigates whether Spiking Neural Networks (SNNs)—which are inherently energy-efficient due to their binary, event-driven nature—can achieve competitive performance in speech enhancement by optimizing network architecture and spectral modeling.

The authors introduce GSU-DBNet, a dual-branch architecture that replaces standard recurrent units with Gated Spiking Units (GSUs). The model employs a dual-path separator that processes features along both frequency and time dimensions to capture global spectral correlations and causal temporal dependencies. To maximize performance, the decoder uses a dual-branch structure that simultaneously estimates magnitude and complex spectral masks, which are then fused to reconstruct the enhanced speech. The researchers also perform ablation studies to determine the optimal number of gates for the spiking units, testing the hypothesis that the binary output bottleneck of SNNs renders complex gating redundant.

GSU-DBNet achieves a PESQ score of 3.04 on the VoiceBank+DEMAND benchmark using only 394K parameters. This represents a significant improvement over existing SNN-based methods (e.g., +0.84 over DPSNN) and outperforms several representative ANN-based models while utilizing only 4.5%–10.6% of their parameter counts. The ablation studies demonstrate that the simplest single-gate GSU design is optimal; adding more gates increases parameter count without improving performance, confirming that the binary output bottleneck limits the utility of complex gating in spiking recurrent cells.

This work provides a blueprint for building high-performance, energy-efficient speech enhancement systems suitable for deployment on neuromorphic hardware. By demonstrating that architectural design—specifically dual-branch spectral modeling—can bridge the performance gap between SNNs and ANNs, the authors offer a path toward real-time, low-latency audio processing on resource-constrained edge devices.