Mind the Cap: Output-Budget Regimes Change the Measured Multilingual Reasoning Gap

Researchers often observe that large language models (LLMs) perform better on reasoning tasks when prompts are translated into English compared to the original language. This paper challenges the assumption that this performance difference reflects an inherent reasoning deficit. Instead, the authors demonstrate that the 'multilingual reasoning gap' is heavily influenced by the output-token budget—a hidden experimental variable that is rarely controlled across languages.

The authors analyze the MGSM benchmark across German, Thai, and Swahili using Qwen3-8B and Llama-3.1-8B-Instruct. Because different languages have different tokenization efficiencies, a fixed token cap (e.g., 1024 tokens) restricts some languages more severely than others. By conducting a retrospective sweep of budgets and using length-normalized caps based on FLORES-200 token premiums, the researchers show that the measured gap is highly sensitive to these constraints. They further validate these findings using independently hard-capped decodes to ensure that the observed effects are not merely artifacts of shared prefix trajectories.

The study reveals that the measured gap can be manipulated significantly by adjusting the output budget. In some cases, tight caps can even reverse the relative performance ranking of different prompting strategies. The authors also identify that simply announcing a specific budget in the prompt can alter model behavior, even when the enforced cap remains constant. Furthermore, the researchers provide a mathematical identity based on answer-emission timing that accurately predicts the peak of the budget-binding regime, suggesting that the gap is driven by when a model emits its final answer relative to the truncation point.