How Much is a Human Right Worth? ECtHR-NPD: A Benchmark for Predicting Non-Pecuniary Damage Awards

Predicting monetary remedies in legal settings is a complex task, especially when awards are determined on an equitable basis without explicit statutory formulas. This paper introduces ECtHR-NPD, a new benchmark designed to evaluate how well computational models can predict non-pecuniary damage (NPD) awards at the European Court of Human Rights (ECtHR). The dataset comprises 14,575 cases, providing a rigorous testbed for continuous monetary prediction. The authors evaluate a wide range of methods, including gradient-boosted trees, retrieval-based systems, fine-tuned encoder language models, prompted decoder language models, and knowledge-augmented agents.

The study finds that sophisticated language models and agentic approaches do not consistently outperform strong feature-based baselines like CatBoost. While models can sometimes capture the general ranking of awards, they struggle significantly with two critical aspects of the task: identifying cases that result in zero awards and accurately calibrating the magnitude of high-award cases. The results demonstrate that these models often fail to handle the heavy-tailed distribution of awards and the complexities introduced by multi-applicant or multi-violation cases. Furthermore, the performance of all models degrades sharply on the Challenging diagnostic subset, which includes complex cases like those heard by the Grand Chamber.

This research extends legal NLP evaluation beyond categorical classification and retrieval tasks into the domain of continuous monetary remedies. By exposing specific failure modes—such as zero-award recognition and magnitude calibration—the ECtHR-NPD benchmark provides a necessary tool for developing more robust and numerically aware legal AI. It highlights that current LLM architectures, which often treat numbers as token sequences, are not yet optimized for the specific demands of discretionary legal quantification.