ResearchPod Summary
How can researchers create flexible, general-purpose simulations of human individuals using large language models (LLMs)? Current approaches to predicting human behavior often rely on domain-specific training data, which is costly to collect and difficult to adapt. This paper explores whether grounding LLMs in rich, person-specific self-report data—such as semi-structured interviews or structured surveys—enables them to simulate human attitudes and behaviors across diverse tasks without task-specific training.
The researchers recruited a diverse national sample of 1,052 Americans and used an AI interviewer to conduct two-hour, semi-structured interviews based on the American Voices Project protocol. Participants also completed structured surveys (General Social Survey, Big Five personality inventory) and behavioral economic games. For each participant, the authors created generative agents grounded in (1) interview transcripts, (2) survey responses, or (3) a combination of both. These agents were then tested on held-out survey questions, personality inventories, economic games, and replication experiments. Performance was benchmarked against each participant's own two-week test-retest consistency to account for natural human response noise.
Agents grounded in participants' own self-reports (interviews and/or surveys) significantly outperformed traditional demographic-only or persona-based prompting methods. On the General Social Survey, these self-report-based agents achieved 82–86% of the accuracy of participants' own two-week test-retest consistency. The study finds that interviews and surveys provide complementary information; combining both methods yielded the highest accuracy, though the marginal gains suggest predictive benefits asymptote after sufficient data is provided. Furthermore, these grounded agents were better able to predict behavior in personality traits and social experiments while reducing accuracy disparities across racial and ideological groups compared to demographic-only models.
This work provides a practical path for social scientists to simulate specific individuals at scale, enabling pilot testing of experiments, exploration of effect heterogeneity, and refinement of survey measures. By shifting the burden from collecting massive structured datasets to collecting individual self-reports, researchers can build more flexible and representative simulations that are less prone to the biases common in simple demographic prompting.
AI-generated third-party summary by ResearchPod. Not official content or an endorsement by the paper authors or affiliated organizations.