Gianluca Barmina, Federico Torrielli, Sven Harms, Jacob Nielsen, Felix Mächtle, Stine Lyngsø Beltoft, Peter Schneider-Kamp, Thomas Eisenbarth, Lukas Galke Poech, Anne Lauscher
4 min
Abstract
Large language models (LLMs) routinely face requests that should be refused, creating a trade-off between helpfulness and harm prevention. However, refusals themselves can be helpful. In high-risk interactions involving crisis, coercion, or escalating intent, blunt non-compliance may prevent direct harm while still failing to support the needs of the person behind the request. We present PsychoSafe, a psychologically-informed refusal framework that reframes refusal as structured supportive communication grounded in evidence-based intervention strategies. To develop PsychoSafe, we construct a corpus of 8019 prompt-response pairs spanning five psychologically salient risk domains and apply prompting and parameter-efficient fine-tuning to Qwen 3.5 27B. On a balanced validation set of 500 prompts, evaluated with an LLM judge and validated through human ratings, PsychoSafe prompting improves overall refusal quality by 28.1% over a generic baseline, with particularly strong gains in external resource referral (+46.8%) and psychological grounding (+34.8%), while preserving downstream performance on non-refusal tasks. Fine-tuning achieves near-perfect refusal and resource-referral rates but reduces response relevance. Additional evaluations on SORRY-Bench and XSTest show strong in-domain robustness but limited out-of-domain generalization, suggesting that future work should diversify fine-tuning data to help models apply interventions selectively rather than schematically.
Sam: Precisely. And that matters because the AI keeps all its general abilities while getting meaningfully better at this specific, high-stakes task. The results support this — using their framework, the quality of the model's refusals improved by over twenty-eight percent compared to the standard approach. The improvement was even larger when it came to providing relevant external resources, like crisis hotlines, which the researchers consider a key measure of genuine helpfulness.
Alex: That's a meaningful gap. But I want to ask about the risks. Is there a danger in making an AI sound too empathetic?
Sam: The researchers take this seriously. Because the model generates language rather than following a fixed script, it can't guarantee perfect behavior every time. There's a real risk that a user might mistake it for a human therapist, or feel a false sense of security that discourages them from seeking actual professional help. The authors are explicit: this is not a substitute for clinical advice.
Alex: So the empathetic tone is a feature, but also a potential vulnerability.
Sam: Exactly. They also flag that the system currently only works in English and is calibrated for resources available in the U.S. and U.K. If someone in another country, speaking another language, encounters this system, it may not serve them well. And the model performs reliably on the five risk categories it was trained on, but it doesn't generalize cleanly to new types of situations it hasn't seen before.
Alex: So the honest picture is: a meaningful improvement over what exists, but with real boundaries.
Sam: That's a fair summary. The researchers point to several clear next steps — broadening the training data to cover more languages and cultural contexts, and testing the system under adversarial conditions, meaning situations where someone might deliberately try to manipulate the AI into behaving badly. The goal is to make the support robust, not just polished.
Alex: The core insight here seems worth sitting with — that a refusal is still an interaction, and that interaction can either help or harm the person on the other end.
Sam: It's a useful reframe. Building safer AI isn't only about what the system blocks. It's equally about what it offers in that moment instead.
Alex: Thanks for listening to ResearchPod.