LLM Evaluators are Biased across Languages

As LLMs are increasingly used as automated evaluators (LLM-as-a-Judge) and reward models for RLHF, researchers have assumed that these models provide language-neutral scores. This paper investigates whether multilingual evaluators actually provide consistent, language-invariant judgments when evaluating semantically identical content across different languages.

The authors conducted experiments using semantically identical instruction-response pairs across 23 languages, utilizing both prompted LLM-as-a-Judge models (e.g., LLaMA 3.1, Qwen 2.5) and trained reward models. They compared standard pairwise accuracy—the industry-standard validation metric—against pointwise score distributions. They further analyzed the relationship between these scores, model uncertainty (measured via negative log-likelihood), and language resource levels to determine the structural origins of the observed biases.

The researchers found that multilingual evaluators are not language-neutral. There is a strong, statistically significant correlation between a language's resource level and the scores it receives: lower-resource languages are consistently scored more generously than higher-resource languages. Crucially, this bias is invisible to traditional pairwise accuracy metrics, which remain high (>90%) even as absolute scores shift significantly. Under a global decision threshold, this leads to acceptance-rate disparities of up to 43 percentage points, meaning safety filters and reward mechanisms are effectively "softer" on lower-resource languages. While model uncertainty partially explains this effect, the bias is fundamentally a structural, language-level misalignment that cannot be fully mitigated by simple threshold adjustments.

This study reveals a major vulnerability in current AI safety and alignment pipelines. Because practitioners often rely on pairwise accuracy to validate evaluators, they may be unaware that their models are systematically failing to apply consistent standards across languages. This creates a "language-dependent reward hacking" risk, where models could potentially receive higher rewards by shifting languages, and poses a direct threat to the reliability of safety auditing in multilingual contexts.