Weighted Score-Oriented Losses for Temporally Localized Event Prediction

In many operational time-series applications, such as anomaly and changepoint detection, the utility of an alarm is not determined by pointwise accuracy but by its temporal proximity to an event. Standard neural networks are typically trained using decomposable losses like cross-entropy, which treat every time point independently. This creates a score-loss mismatch, where the training objective ignores the temporal reward structure defined by downstream evaluation metrics. This paper investigates whether incorporating temporal weights into a score-oriented loss (SOL) framework can bridge this gap.

The authors propose a temporally weighted score-oriented loss (wSOL). Building on the existing SOL framework—which replaces discontinuous confusion-matrix counts with their expectations under a threshold distribution—the authors introduce temporal weights that discount false positives near events and reduce false-negative penalties when an event is preceded by an admissible alarm. The resulting objective is differentiable, allowing for optimization via back-propagation. The authors evaluate this approach on three benchmark datasets (NAB, SKAB, and Exathlon) using a Temporal Convolutional Network (TCN) architecture, comparing wSOL against cross-entropy and unweighted SOL.

The study finds that wSOL consistently improves performance on datasets where evaluation utility is localized in time, such as NAB and SKAB. By explicitly encoding the temporal reward structure into the training objective, the model learns to prioritize alarms that are operationally useful. However, the authors observe that on datasets like Exathlon, where labels span long disturbance ranges rather than specific event points, the benefits of temporal weighting are diminished, suggesting that the utility of the loss is highly dependent on the nature of the evaluation metric.