Multi-Scale Convolution with Optimal Transport Attention Effect on Multivariate Time Series

Multivariate time series (MTS) forecasting models often struggle to balance the capture of complex cross-variate dependencies with the preservation of fine-grained local temporal patterns. While recent 'inverted embedding' approaches (like iTransformer) excel at modeling relationships between variables by treating each variate as a single token, they often discard critical local temporal dynamics. This paper investigates how to enhance these models to better capture multi-granularity structural patterns while simultaneously increasing robustness against noise and outliers.

The authors propose the Multi-Scale Convolution with Optimal Transport (MSC-OT) attention mechanism. This architecture introduces two primary enhancements to the standard attention module:

1. Multi-Scale Convolution Enhancement: The model applies parallel 2D convolutions (with kernel sizes 3x3 and 5x5) to the attention score matrix. This allows the model to extract local structural patterns from the variate-interaction space that would otherwise be lost in a globally compressed embedding.
2. Sinkhorn Optimal Transport Regularization: The attention computation is reformulated as an entropy-regularized optimal transport problem. By employing the Sinkhorn algorithm to enforce doubly-stochastic constraints, the model prevents any single noisy variate from dominating the attention distribution, effectively smoothing the output and improving robustness to outliers.

These components are combined with the base attention scores using an adaptive fusion strategy, which employs learnable weights to dynamically balance the contributions of the base, convolution-enhanced, and OT-regularized scores.

Experimental results across five standard benchmarks (including ETT, Electricity, Traffic, Solar-Energy, and Exchange-Rate) demonstrate that MSC-OT consistently outperforms existing state-of-the-art models. The authors report significant improvements in both short-term and long-term forecasting accuracy compared to models like Crossformer, TimesNet, and Fedformer. Ablation studies confirm that both the multi-scale convolution and the optimal transport components contribute synergistically to the model's overall predictive performance.