How Good Can Linear Models Be for Time-Series Forecasting?

Recent advancements in time-series forecasting have favored increasingly large, complex architectures like Transformers and foundation models, under the assumption that greater model capacity is necessary for higher accuracy. This paper challenges that paradigm, investigating whether the performance gap between simple linear models and deep architectures can be closed by optimizing preprocessing choices—such as lookback window length, normalization, and regularization—rather than scaling the model itself.

Using Ridge regression as a transparent, computationally efficient testbed, the authors conducted a systematic hyperparameter search across eight standard time-series benchmarks. They introduced a "grouped search" framework that allows for varying degrees of hyperparameter sharing across forecast horizons and individual series. By treating preprocessing as a first-class optimization problem, the authors evaluated how these choices impact forecasting accuracy and what the resulting optimal hyperparameters reveal about the underlying data structure.

• Preprocessing over Capacity: Carefully tuned Ridge regression matches or exceeds the performance of state-of-the-art Transformer, MLP, and CNN baselines on six of eight benchmarks, demonstrating that linear models are significantly more capable than previously assumed when properly configured.
• Non-monotonic Lookback: The relationship between optimal lookback length and forecast horizon is highly dataset-specific. Contrary to the common assumption that longer horizons require longer history, some datasets (e.g., Exchange and Traffic) actually prefer shorter context windows as the forecast horizon increases, likely due to non-stationarity where distant history introduces noise.
• Local Normalization: Normalizing over a learned trailing fraction of the context window consistently outperforms global normalization, suggesting that recent local statistics are more predictive than global dataset statistics.
• Diagnostic Utility: The optimized hyperparameters serve as a diagnostic tool, revealing structural properties of the data—such as scaling behavior and series heterogeneity—that are typically obscured within the learned parameters of deeper, black-box models.