Scoring Rules! Statistical and Strategic Alignment for Text Evaluation Metrics

Reference-based evaluation metrics are essential for assessing natural language generation (NLG) systems. Traditionally, these metrics are validated by their statistical correlation with human judgments. However, as these metrics are increasingly used as optimization objectives (e.g., in reinforcement learning from human feedback), they become targets for strategic gaming. A metric that correlates with humans in a static setting may fail when an agent learns to exploit its superficial features to inflate scores without improving the actual quality of the output.

The authors propose two distinct notions of alignment for evaluation metrics:

1. Statistical Alignment: The metric correlates with human quality ratings under normal, non-adversarial conditions.
2. Strategic Alignment: The metric resists perturbations that do not add task-relevant information. This includes sensitivity to degradation (penalizing the removal of key information) and manipulation robustness (resisting score inflation through style changes or filler content).

The authors introduce a unified framework for constructing mutual-information (MI) based metrics. By decomposing these metrics into four components—information measure, estimation method, text representation, and prediction mechanism—the researchers can systematically compare existing methods and explore new designs. This framework reveals that the best-performing metrics often operate at the statement level, providing a more reliable signal than token-level or whole-document approaches.

As LLMs are increasingly optimized against automated metrics, the risk of "reward hacking" grows. This paper provides a rigorous, standardized way to test whether a metric is truly measuring quality or simply rewarding a model's ability to manipulate the evaluation process. By demonstrating that high correlation does not guarantee robustness, the authors offer a roadmap for building more trustworthy evaluation systems that are resilient to adversarial optimization.