Fangxu Yu, Tao Feng, Dehai Min, Zinan Lin, Weijia Xu, Michael Xu, Philip S. Yu, Ge Liu, Tianyi Zhou
4 min
Abstract
Audio reasoning is essential for machine understanding of the acoustic world. Reinforcement learning with verifiable rewards can elicit such reasoning, yet existing reward designs are complementary in their limitations: outcome-based rewards supervise only the final answer and let the model reach it without attending to the audio, whereas process-based rewards score the reasoning itself but rely on coarse, hand-crafted, and fixed criteria that neither adapt to each question nor stay grounded in the acoustic evidence. Moreover, questions differ in what they demand, with some hinging on perception and others on multi-step reasoning, and any static criterion weakens as the policy improves. Supervising the reasoning process with fine-grained, audio-grounded, and adaptive rewards is therefore crucial, yet challenging since such rewards are impractical to design by hand for every sample. To this end, we introduce AudioRubrics, a reinforcement learning framework that supervises audio reasoning with self-evolving, audio-grounded rubric rewards. AudioRubrics synthesizes per-sample rubrics from the raw waveform and, conditioned on the model's own rollouts, regenerates and reweights criteria per group, supplying a continuous learning signal that keeps targeting the current policy's weaknesses as static criteria saturate. Comprehensive evaluations across three audio reasoning benchmarks reveal that AudioRubrics substantially outperforms a wide range of open-source and training-based baselines. Furthermore, our analysis shows that the gains scale with the capability of the rubric generator and judge, and AudioRubrics converges to a stable reasoning length that avoids both degenerate collapse and unbounded growth. The improvement in audio perception further demonstrates the effectiveness of anchoring supervision in the acoustic evidence. Our project page is available at https://audiorubrics.github.io.
Sam: So the system is pushing the model toward reasoning that is both grounded in the audio and efficient. How does the scoring actually work under the hood?
Alex: The scoring uses a technique called Group Relative Policy Optimization — GRPO for short. Here's the key idea: instead of comparing each response to a single "correct" answer, the system generates a whole group of responses to the same question and then compares them to each other. Responses that do better than the group average get a positive signal; ones that do worse get a negative signal.
Sam: So the model is essentially competing against its own previous attempts, and learning from the spread?
Alex: Exactly. That relative comparison — the gap between a specific response and the group average — is what the researchers call the "advantage." The model learns to favor whatever reasoning strategies consistently produce that advantage. And because the rubrics driving those comparisons are generated from the raw audio, the whole feedback loop stays grounded in the sound itself.
Sam: So the three pieces — the evolving rubrics, the overthinking penalty, and the group-based scoring — all work together to keep the model honest.
Alex: That's a good summary. Each piece addresses a different failure mode: the evolving rubrics prevent saturation, the penalty prevents gaming through verbosity, and the group scoring ensures the model is always learning from meaningful comparisons rather than chasing a static target. The paper's argument is that combining all three is what keeps the training signal useful across the full arc of the model's development.
Sam: It's a more complete picture of what "learning to listen" actually requires.
Alex: And that's arguably the deeper point of the paper. Teaching a model to get the right answer is one thing. Teaching it to genuinely reason about what it heard — and to keep improving as it gets better — turns out to require a much more carefully designed feedback system. Thanks for listening to ResearchPod.