Transcript: The Personality Trap: How LLMs Embed Bias When Generating Human-Like Personas

Alex: Welcome to another episode of ResearchPod.

Sam: Today we're discussing a paper called "The Personality Trap: How LLMs Embed Bias When Generating Human-Like Personas." It looks at whether large language models—those AI systems trained on huge amounts of internet text to chat and write like people—can create fake human profiles from personality test answers. The central puzzle is this: can these AIs make diverse stand-ins for real people, or do they trap us in narrow stereotypes?

Alex: So it's testing if AI can simulate varied groups just from quiz results... or if biases sneak in?

Sam: Yes, that's the core question. Researchers want synthetic personas—AI-made stand-ins for real humans—to help with psychology studies or polling, since gathering thousands of real responses is expensive and slow. But the models keep producing people who are mostly young, educated, from wealthy Western countries, white, straight, and leaning progressive or centrist. They call this a WEIRD bias—short for Western, Educated, Industrialized, Rich, Democratic—because the internet text these AIs learn from comes mostly from those groups.

Alex: Even when you feed in scores from a test like the EPQR-A—which measures traits like how outgoing or tough-minded someone is—the backgrounds that come out aren't diverse?

Sam: Exactly. The study used 826 sets of answers from the Eysenck Personality Questionnaire, a short quiz with yes/no questions about behavior and feelings. They fed those into five different LLMs with a prompt: "Pretend this is from a real person—now describe their full profile, like age, job, and beliefs, in a neat list." The personas matched the input traits well, acting outgoing if told to, for example. But demographics skewed WEIRD every time.

Alex: And when they pushed traits to extremes, like high Psychoticism—the tendency to be impulsive or tough—that's when stereotypes showed up more?

Sam: That's right. For max Psychoticism, non-binary identities jumped to around 89 percent in one top model, compared to 29 percent normally. LGBTQ+ labels hit over 94 percent there, while straight dropped near zero. The paper suggests the AIs link tough, impulsive traits to marginalized groups in their training data—like assuming extremes mean you're outside the norm. This pathologizes those groups.

Alex: So the promise of cheap fake populations... but with built-in traps. How did they confirm the traits really stuck despite those shifts?

Sam: They had the new personas retake the EPQR-A test. Scores lined up closely, with low error rates. They also matched well on the Big Five test for overlapping traits like extraversion. Internal checks showed reliable patterns too—the AI held onto the personality core despite skewed details.