Alexis Carrillo, Enrique Taietta, Ali Aghazadeh Ardebili, Giuseppe Alessandro Veltri, Massimo Stella
6 min
Talk2AI is a groundbreaking dataset capturing 3,080 conversations (30,800 turns) between 770 Italian adults and large language models (LLMs) over four weekly sessions. Collected in Spring 2025, it focuses on how AI can persuade humans on real-world topics like climate change, math anxiety, and health misinformation. Unlike prior studies that measure attitude change in one-off interactions, Talk2AI tracks shifts longitudinally—revealing how repeated chats with AI influence beliefs, opinions, and behaviors over time.
Participants were randomly assigned to one of four LLMs: GPT-4o, Claude Sonnet 3.7, DeepSeek-chat V3, or Mistral Large. Each engaged in 10-turn discussions per topic per session, followed by feedback on opinion change, conviction stability, perceived AI humanness, and behavioral intentions. Rich metadata—including sociodemographics and psychometric profiles—links to every conversation, enabling analysis of who is most swayed by AI and why.
Most AI persuasion research treats change as a 'single endpoint' after brief exposure, missing the dynamics of ongoing dialogue. Talk2AI's within-subject, four-wave design (weekly sessions) captures temporal evolution: How does conviction waver? Does trust in AI grow? Do effects compound or fade?
This setup disentangles AI's role as a 'continuous persuasive agent,' influenced by model architecture, user traits, and context. For instance, early sessions might build rapport (boosting perceived humanness), while later ones test sustained influence on attitudes.
Core Components:
Processing: Data cleaned, translated (EN, ES, NL, DE, PT, FR), and structured for ML/NLP analysis. Figure 1 in the paper outlines the workflow: recruitment → profiling → interact → feedback → repeat.
AI is everywhere in info-seeking and debate. Talk2AI quantifies its persuasive power at scale, addressing gaps like:
Findings preview: LLMs induce measurable shifts, but effects vary by model (e.g., Claude vs. Mistral) and user (e.g., open personalities more receptive). This scales to digital ecosystems—think social media bots or chat advisors shaping public opinion.
At ~400MB, Talk2AI is public-ready for cognitive science, HCI, and AI safety. It proves AI isn't just a tool—it's a dialogue partner reshaping minds, one turn at a time.
Talk2AI is a large-scale longitudinal dataset of 3,080 conversations (totaling 30,800 turns) between human participants and Large Language Models (LLMs), designed to support research on persuasion, opinion change, and human-AI interaction. The corpus was collected from 770 profiled Italian adults across four weekly sessions in Spring 2025, using a within-subject design in which each participant conversed with a single model (GPT-4o, Claude Sonnet 3.7, DeepSeek-chat V3, or Mistral Large) on three socially relevant topics: climate change, math anxiety, and health misinformation. Each conversation is linked to rich contextual data, including sociodemographic characteristics and psychometric profiles. After each session, participants reported on opinion change, conviction stability, perceived humanness of the AI, and behavioral intentions, enabling fine-grained longitudinal analysis of how AI-mediated dialogue shapes beliefs and attitudes over time.
Alex: Okay, so repeat those mindset checks each time to track if they stay steady... then the chat itself. What kept the talks focused and deep?
Sam: Users got randomly assigned one AI model and one topic—like climate change or math worries—that stayed the same all four weeks, for ten back-and-forth turns each session. To spark real debate, the first user message had to be at least 50 words, and pop-up hints suggested ways to push back, like asking for proof or giving counterpoints. Hidden instructions told the AI to spot flaws in user arguments early on, then keep replies short, without letting topics drift. After ten turns, they rated conviction strength in their views, any opinion shift, how human the AI felt, and did a quick choice: split 100 euros between themselves and a related charity—the less they kept, the more persuaded they seemed.
Alex: That charity split as a persuasion check is clever... but with thousands signing up, how did they end up with reliable data from just 770 people?
Sam: They ran a strict cleaning process on the raw files—over 2,600 sign-ups at first. They tossed sessions missing the full 20 messages from glitches, cut users who gave identical answers across questions, and kept only those finishing all four weeks completely. This left 770 solid participants, with data split into JSON files for demographics, psychometrics and ratings, chats, plus a ready-to-analyze CSV linking traits to feedback scores.
Alex: Huh... so the filters ensured each person's full journey was tracked consistently, traits to persuasion shifts.
Sam: Yes. That rigor lets researchers link steady personality measures to how beliefs evolved over weeks, a meaningful advance for studying repeated AI influence.
Alex: So with those clean profiles linked to chats... what exactly did they ask right after each conversation to measure persuasion?
Sam: After the ten turns, participants answered five quick questions. The first asked how convinced they felt of their starting arguments, on a scale from one to 100. The second gauged opinion shift on the topic from zero meaning no change to 100 for a full flip. Third was how human-like the AI chat felt, one to 100. Fourth imagined splitting 100 euros between self and a related charity. Last was an open write-up of at least 50 words on their final thoughts.
Alex: Right, so beyond describing the data... what could this mean for building better AI coaches?
Sam: The paper suggests it could help create AI that tailors persuasion to a person's traits—like using thinking style to adjust arguments on climate action or misinformation. For instance, train models to predict if someone high in mental diligence shifts views faster, scaling talks for real therapy or public campaigns. But results depend on future tests; this dataset provides the foundation.
Alex: That sounds practical... though fixed ten-turn chats might cut off deeper talks, right? And starting in Italian could skew translations for other languages.
Sam: Exactly—those are noted limits. Shorter sessions keep data clean but miss free-flowing debates; Italian originals work for that language but need care for multilingual tools, as machine translations might lose nuances. Still, the within-subject tracking over weeks offers a clear step forward in understanding repeated AI influence.
Alex: Well put. That's our look at the Talk2AI dataset—a solid base for linking who you are to how chats reshape beliefs long-term. Thanks for listening to ResearchPod.