BehaviorBench: Benchmarking Foundation Models for Behavioral Science Tasks

Understanding BehaviorBench

Behavioral science relies on understanding how individuals and populations make decisions. As foundation models are increasingly used to simulate human behavior, predict survey responses, and assist in research, there is a critical need to evaluate whether these models truly capture the nuances of human behavior. BehaviorBench is a new, comprehensive benchmark designed to systematically assess foundation models across four core capabilities: behavior prediction and simulation, strategic decision-making, subject-trait inference, and the application of behavioral science knowledge.

The Importance of Distributional Evaluation

Most existing benchmarks treat human subjects as independent data points, focusing solely on pointwise accuracy. BehaviorBench argues that this is insufficient for behavioral science, which often requires models to preserve the heterogeneity and distributional characteristics of a population. By evaluating models at both the individual level (per-subject accuracy) and the distributional level (alignment with population-wide behavior), BehaviorBench provides a more rigorous standard for behavioral validity. The researchers use the Wasserstein distance to measure how well a model's predicted distribution of behaviors matches the empirical distribution observed in real-world human data.

Findings and Model Development

In addition to the benchmark, the authors introduce Be.FM-1.5, a family of foundation models fine-tuned on behavioral data. The evaluation results highlight a clear trade-off: proprietary general-purpose models (like GPT-5.4 and Gemini 3.1 Pro) perform exceptionally well on individual-level prediction and knowledge-intensive tasks. However, these models often struggle to replicate the distributional patterns of human populations. In contrast, Be.FM-1.5 demonstrates that targeted adaptation—fine-tuning on diverse behavioral datasets—can close the gap, allowing models to achieve strong distributional alignment while remaining competitive on individual-level metrics.

This work establishes a new standard for evaluating AI in the social sciences. By highlighting the gap between individual prediction and distributional alignment, the authors provide a roadmap for developing AI systems that are not just accurate at predicting single outcomes, but are also faithful to the complex, diverse nature of human populations. This is essential for applications ranging from policy simulation and market research to personalized interventions.