Intrinsically Stable Spiking Neural Networks: Overcoming the Performance Barrier in the Absence of Batch Normalization

Deep spiking neural networks (SNNs) have historically struggled to scale to deep architectures without relying on batch normalization (BN). However, modern dynamic BN variants introduce runtime multiplications and statistical tracking that negate the energy-efficiency benefits of neuromorphic hardware. This paper investigates how to achieve high-performance deep SNNs without these costly runtime normalization layers.

The authors identify catastrophic firing-rate decay—where neuronal activity vanishes or saturates across layers—as the primary failure mode in normalization-free SNNs. To address this, they introduce the Intrinsically Stable SNN (IS-SNN). This architecture enforces signal homeostasis by using topology-aware weight standardization and modified residual connections. By analytically deriving scaling factors based on the network's topology and the intrinsic variance of spiking neurons, the authors can fold all normalization operations into static weights offline. This restores a simple, accumulation-oriented inference datapath that requires no runtime statistics tracking or multiplications.

IS-SNN demonstrates that normalization-free SNNs can scale to deep architectures like ResNet-152 and Transformer-based models while maintaining competitive accuracy. On ImageNet, IS-SNN achieves 68.05% accuracy, outperforming naive normalization-free baselines and rivaling computationally expensive dynamic BN techniques. Furthermore, hardware synthesis on an FPGA shows that IS-SNN reduces lookup table (LUT) resource consumption for neuron implementations by 96.4% compared to dynamic BN-enabled neurons, confirming its suitability for energy-efficient neuromorphic systems.

This work provides a practical framework for deploying deep SNNs on resource-constrained edge hardware. By decoupling training stability from inference-time normalization, IS-SNN bridges the gap between the high accuracy of modern deep learning and the hardware-efficiency requirements of neuromorphic computing. It demonstrates that biological principles like homeostatic regulation can be effectively translated into architectural design to solve fundamental engineering bottlenecks in spiking systems.