Frequency Domain Reservoir Computing

Standard Echo State Networks (ESNs) are powerful tools for sequence modeling because they avoid the computational burden of backpropagation through time by keeping recurrent weights fixed. However, they suffer from an O(N^2) computational bottleneck during state updates, which limits their scalability compared to modern architectures like Transformers or State Space Models. The authors introduce Frequency Domain Reservoir Computing (FRESCO) to address this by moving the entire recurrent computation into the frequency domain, where dense matrix-vector products can be replaced by efficient element-wise Hadamard products.

FRESCO overcomes the traditional overheads of frequency-domain processing—specifically the costly Fast Fourier Transforms (FFT) usually required for input and output—through three primary innovations:

1. Dimensional Zero-Padding: Instead of performing a full N-point FFT on the input, FRESCO uses a 2D convolution formulation that allows inputs to be embedded using a smaller, more efficient O(Nx log Nx) transform.
2. Frequency-Domain Recurrence: By sampling weights directly in the frequency domain, the authors gain precise control over the reservoir's spectral radius at zero cost, bypassing the expensive O(N^3) eigendecomposition typically required to ensure the Echo State Property.
3. Packed Frequency-Domain Readout: The authors designed a memory-contiguous, redundancy-free readout that operates directly on the frequency-domain state. This allows the model to produce real-valued outputs without ever needing to perform an inverse FFT, maintaining mathematical equivalence to standard spatial-domain training.

FRESCO bridges the gap between the efficiency of traditional Reservoir Computing and the scalability of modern deep learning architectures. By reducing the per-step complexity to O(N), FRESCO enables the use of much larger reservoirs than previously feasible, allowing for better performance on long-horizon forecasting and complex sequential classification tasks while significantly lowering energy consumption and computational latency.