Rushes: A Human Preference Dataset for Pluralistic Alignment

Research Question and Approach

Modern alignment techniques, such as Reinforcement Learning from Human Feedback (RLHF), are primarily designed to ensure model safety and helpfulness by converging toward a single, universally acceptable output. However, in creative and entertainment domains, the goal is often divergence—maximizing engagement for diverse individuals. The authors introduce Rushes, a large-scale dataset of 44,226 revealed-preference decisions from 8,167 unique users, to study how AI models can adapt to personalized, context-dependent narrative preferences.

Unlike static judgment datasets, Rushes captures sequential, longitudinal trajectories in interactive, multimodal branching narratives. Users interacted with AI-generated stories, making choices at decision points that influenced the narrative path. The authors use this data to benchmark the ability of models to predict these individual choices, comparing frontier LLMs against collaborative filtering baselines and popularity heuristics.

The Engagement Gap

The study identifies a significant "Engagement Gap." When tasked with predicting user choices, frontier models like GPT-5 fail to outperform a simple popularity baseline (which always selects the most historically chosen option). While collaborative filtering (SVD) achieves 37.7% accuracy, GPT-5 reaches only 34.2%, falling below the popularity baseline of 36.4%. This indicates that even highly capable models default to majority-preference behavior rather than adapting to the specific, evolving history of an individual user.

This research highlights a fundamental limitation in current alignment paradigms. By aggregating diverse human preferences into a single reward model, modern systems suppress minority viewpoints and fail to capture the subjectivity inherent in "fun" or "interesting" content. The Rushes dataset provides a necessary testbed for developing pluralistic alignment techniques that can move beyond population-level averages to support truly personalized, interactive generative systems.