Audio for Sports Highlight Detection: A Comparative Empirical Study

Sports highlight detection is traditionally dominated by visual or visual-language models, often treating audio as a secondary or auxiliary signal. This paper investigates the untapped potential of audio by asking: how far can audio alone go in identifying highlights in long-form sports videos? The authors systematically evaluate whether audio contains sufficient semantic and affective cues to serve as a primary signal for highlight detection.

Using the SV-Highlights benchmark, which contains 320 long-form videos across eight sports, the authors construct controlled baselines. They compare visual-only, audio-only, and audio-visual fusion models. The audio-only models utilize pretrained Audio Spectrogram Transformer (AST) features processed through both simple MLP (clip-independent) and GRU (temporal) architectures. To understand the contribution of specific audio components, the authors use source separation (Demucs) to isolate vocal/commentary streams from background sounds (crowd, whistles, impacts) and analyze their individual and combined impacts on performance.

Surprisingly, the audio-only GRU baseline outperforms several existing audio-visual methods, demonstrating that audio is a powerful, standalone modality. Temporal modeling is found to be critical; GRU-based models significantly outperform clip-independent MLP models, suggesting that highlights are defined by the temporal progression of acoustic events (e.g., commentator build-up followed by crowd reaction). Source-separated analysis reveals that while vocal/commentary audio is generally more informative, background sounds provide essential complementary evidence. Simple audio-visual fusion yields the best performance, confirming that audio and vision provide distinct, non-redundant information for identifying exciting moments.

This study challenges the visual-centric paradigm in sports video understanding. By proving that audio is a primary signal, the authors provide a strong foundation for more efficient and effective highlight detection systems. The findings suggest that future research should prioritize sophisticated audio-visual fusion and temporal modeling to improve the reliability and coherence of automated sports summaries.