Do We Really Need Transformers for Global Spatial Information Extraction in Traffic Forecasting?

Traffic forecasting models frequently employ complex spatial modules, such as Transformers or graph attention networks, to extract global spatial dependencies—the interactions between a node and all other nodes in the network. This paper investigates whether these high-degree-of-freedom adaptive mechanisms are strictly necessary, or if a simpler, low-degree-of-freedom global aggregation operator can achieve similar predictive performance.

The authors introduce a controlled ablation framework that isolates the spatial mixing module while keeping temporal encoders, input representations, and training protocols constant. They compare five spatial mixing strategies: no aggregation (No-SAB), identity mapping (I-Trans), local neighborhood aggregation (UD-Trans), uniform global mean broadcasting (U-Trans), and standard spatial attention (S-Trans). To understand the underlying mechanism, they decompose spatial attention into a row-uniform global background and a non-uniform residual, testing the predictive value of these components across six diverse traffic benchmarks.

The study demonstrates that U-Trans, which uses a simple row-uniform global mean, is a strong baseline that often outperforms or matches more complex models. Across six datasets, the difference in mean absolute error (MAE) between U-Trans and standard spatial attention is negligible (approximately 0.14%). Furthermore, U-Trans reduces the spatial mixing complexity from O(N²) to O(N), offering significant computational efficiency gains. The analysis reveals that the 'residual' component of standard attention—the part that deviates from a uniform background—provides only dataset-dependent marginal value, suggesting that complex attention is not universally superior to simpler global aggregation.

This research challenges the prevailing trend of using increasingly complex Transformer-based architectures for spatial modeling in traffic forecasting. By showing that a simple, efficient global mean operator is often sufficient, the authors provide a more scalable and interpretable alternative for real-world traffic systems, where computational efficiency is critical for large-scale networks.