This paper examines biases in large language models (LLMs) when generating synthetic populations from responses to personality questionnaires. Using five LLMs, we first assess the representativeness and potential biases in the sociodemographic attributes of the generated personas, as well as their alignment with the intended personality traits. While LLMs successfully reproduce known correlations between personality and sociodemographic variables, all models exhibit pronounced WEIRD (western, educated, industrialized, rich and democratic) biases, favoring young, educated, white, heterosexual, Western individuals with centrist or progressive political views and secular or Christian beliefs. In a second analysis, we manipulate input traits to maximize Neuroticism and Psychoticism scores. Notably, when Psychoticism is maximized, several models produce an overrepresentation of non-binary and LGBTQ+ identities, raising concerns about stereotyping and the potential pathologization of marginalized groups. Our findings highlight both the potential and the risks of using LLMs to generate psychologically grounded synthetic populations.
Alex: Welcome to another episode of ResearchPod.
Sam: Today we're discussing a paper called "The Personality Trap: How LLMs Embed Bias When Generating Human-Like Personas." It looks at whether large language models—those AI systems trained on huge amounts of internet text to chat and write like people—can create fake human profiles from personality test answers. The central puzzle is this: can these AIs make diverse stand-ins for real people, or do they trap us in narrow stereotypes?
Alex: So it's testing if AI can simulate varied groups just from quiz results... or if biases sneak in?
Sam: Yes, that's the core question. Researchers want synthetic personas—AI-made stand-ins for real humans—to help with psychology studies or polling, since gathering thousands of real responses is expensive and slow. But the models keep producing people who are mostly young, educated, from wealthy Western countries, white, straight, and leaning progressive or centrist. They call this a WEIRD bias—short for Western, Educated, Industrialized, Rich, Democratic—because the internet text these AIs learn from comes mostly from those groups.
Alex: Even when you feed in scores from a test like the EPQR-A—which measures traits like how outgoing or tough-minded someone is—the backgrounds that come out aren't diverse?
Sam: Exactly. The study used 826 sets of answers from the Eysenck Personality Questionnaire, a short quiz with yes/no questions about behavior and feelings. They fed those into five different LLMs with a prompt: "Pretend this is from a real person—now describe their full profile, like age, job, and beliefs, in a neat list." The personas matched the input traits well, acting outgoing if told to, for example. But demographics skewed WEIRD every time.
Alex: And when they pushed traits to extremes, like high Psychoticism—the tendency to be impulsive or tough—that's when stereotypes showed up more?
Sam: That's right. For max Psychoticism, non-binary identities jumped to around 89 percent in one top model, compared to 29 percent normally. LGBTQ+ labels hit over 94 percent there, while straight dropped near zero. The paper suggests the AIs link tough, impulsive traits to marginalized groups in their training data—like assuming extremes mean you're outside the norm. This pathologizes those groups.
Alex: So the promise of cheap fake populations... but with built-in traps. How did they confirm the traits really stuck despite those shifts?
Sam: They had the new personas retake the EPQR-A test. Scores lined up closely, with low error rates. They also matched well on the Big Five test for overlapping traits like extraversion. Internal checks showed reliable patterns too—the AI held onto the personality core despite skewed details.
Alex: Traits stick mechanically... but channel biases from test answers straight to demographics. With those swings, what does it say about the links the models make?
Sam: Even after non-binary jumped from under 3 percent to 97 percent in one model for max Psychoticism, scores stayed close. Low variation across runs meant consistent outputs. Maxing Neuroticism—the tendency to feel anxious or moody—boosted women and progressives, matching some data patterns. But high Psychoticism raised non-binary and LGBTQ+ shares dramatically—a mismatch that flags harmful stereotypes.
Alex: So the models draw from training data tying extreme traits to certain groups... amplifying WEIRD defaults and risks.
Sam: Precisely. The study suggests LLMs internalize personality-demographic ties from data but default to biased demographics—a meaningful limit for diverse synthetic humans. It notes its own bounds: it used AI-made quiz answers, not real ones, so human data checks are next; it stuck to English and this quiz.
Alex: Fair points. Overall, LLMs can mirror personality cores from quizzes, but baked-in demographic traps limit scaling to simulations. It urges measured use in social science.
Sam: That's the grounded takeaway—debiasing is needed for truly representative synthetic groups.
Alex: Thanks for breaking it down, Sam. Listeners, that's our look at how personality quizzes meet AI personas—and the biases in between. Thanks for listening to ResearchPod.