READ More than What You See: Reinforcement Learning for Accurate and Coherent Audio Description Generations

Audio Description (AD) requires generating concise, narratively coherent, and temporally constrained descriptions of visual content for blind and low-vision audiences. Existing methods typically rely on either training-free prompting of multimodal models—which often struggle with AD-specific style—or training-based models optimized via simple next-token prediction, which tends to favor generic, repetitive language. This paper investigates whether reinforcement learning (RL) can better optimize models for the sequence-level requirements of accurate and coherent AD generation.

The authors introduce READ, a reinforcement learning framework that treats AD generation as a sequence-level optimization problem. Unlike standard supervised fine-tuning, READ uses Group Relative Policy Optimization (GRPO) to train the model based on a multi-component reward function:

• Accuracy Reward: Measures faithfulness to human-authored reference descriptions using ROUGE scores.
• Length Reward: Encourages descriptions to fit within the specific time constraints of the video clip.
• Format Reward: Enforces a structured output format using specific tags to ensure consistency.
• Coherence Reward: A novel component that evaluates how well a generated description fits the surrounding narrative context, using a domain-adapted language model and anti-copy masking to prevent trivial repetition.

Experimental results on the MAD-Eval, CMD-AD, and TV-AD datasets demonstrate that READ consistently outperforms both training-free and training-based baselines. By moving beyond token-level supervision, the model achieves higher scores on both conventional captioning metrics (e.g., CIDEr, ROUGE-L) and AD-specific metrics (e.g., Action score, Recall). The results suggest that RL is a highly effective paradigm for aligning vision-language models with the specific narrative and temporal constraints of audio description.