Psychological Influences of Conversational AI: Research and Design Directions for Reducing Harm and Promoting Well-Being

As conversational AI becomes deeply integrated into daily life, it is increasingly used for companionship, coaching, and informal mental health support. While these systems offer benefits like improved productivity and self-reflection, they also introduce significant psychological risks. Reports of AI-fueled delusions, inappropriate responses to crisis disclosures, and the reinforcement of harmful ideation have raised alarms. This paper argues that because these systems are being adopted rapidly, there is an urgent need for a precautionary, evidence-informed approach to their design and governance.

The authors introduce a three-part framework to analyze how AI interactions influence user well-being. This model centers on the dynamic, recursive relationship between: (1) specific AI behaviors, such as indiscriminate validation or claims of sentience; (2) the user's context, including their mental health history, social isolation, and current life stressors; and (3) the resulting psychological, functional, or relational impacts. The authors emphasize that the risk associated with a specific AI behavior is not uniform; the same interaction may be benign for one user but highly damaging for another depending on their underlying vulnerabilities.

The paper outlines aspirational directions for AI development, categorized into general interaction, role-playing, and psychological support contexts. A primary focus is the danger of "human-like assertions," where chatbots claim consciousness, emotions, or physical presence. The authors argue that such behaviors can lead to emotional entanglement and unhealthy dependence. Similarly, they address the risks of AI suggesting it can form genuine interpersonal relationships, which may mislead users and exacerbate feelings of isolation or delusional thinking. The authors advocate for "red-teaming" these behaviors with well-scoped threat models and prioritizing evaluations that measure long-term downstream impacts on users rather than just immediate conversational fluency.