Rethinking Reservoir Pruning: A Dynamical Perspective for Echo State Networks

Echo State Networks (ESNs) are efficient for temporal prediction but often suffer from over-parameterization, leading to redundant neurons that increase computational costs without improving performance. Existing pruning methods typically rely on static metrics like weight magnitude or activation variance, which fail to capture how neurons influence the network's temporal state transitions. The authors propose Dynamical Mode Pruning (DMP), a data-dependent approach that ranks neurons by their contribution to the dominant dynamical modes of the reservoir.

To implement DMP, the authors compute the Jacobian of the reservoir state transitions along an observed training trajectory. These local Jacobians are aggregated into a trajectory-averaged Gramian matrix. By performing an eigendecomposition on this Gramian, the authors identify the principal directions of the reservoir's input-driven dynamics. Neurons are then ranked based on their participation in these dominant modes, and those with the lowest scores are removed. The readout layer is then retrained to adapt to the reduced reservoir.

Experiments on chaotic benchmarks (such as the Mackey-Glass system) and real-world time-series (electricity, temperature, wind, and solar data) demonstrate that DMP effectively identifies and removes redundant neurons. The results show that DMP consistently preserves or improves forecasting accuracy compared to unpruned networks and traditional static pruning baselines. This suggests that incorporating dynamical influence is a more robust criterion for reservoir refinement than relying solely on structural connectivity or individual activation levels.

This research provides a principled, data-driven framework for compressing ESNs, which is critical for deploying reservoir computing in resource-constrained environments. By shifting the focus from static properties to the actual input-driven behavior of the reservoir, DMP offers a more nuanced understanding of which neurons are truly essential for temporal processing. This work also highlights that larger reservoirs are not always superior, and that targeted pruning can reveal compact, efficient subnetworks that maintain high predictive performance.