AI systems fail silently far more often than they fail visibly. In a large-scale quantitative analysis of human-AI interactions from the WildChat dataset, we find that 78% of AI failures are invisible: something went wrong but the user gave no overt indication that there was a problem. These invisible failures cluster into eight archetypes that help us characterize where and how AI systems are failing to meet users' needs. In addition, the archetypes show systematic co-occurrence patterns indicating higher-level failure types. To address the question of whether these archetypes will remain relevant as AI systems become more capable, we also assess failures for whether they are primarily interactional or capability-driven, finding that 91% involve interactional dynamics, and we estimate that 94% of such failures would persist even with a more capable model. Finally, we illustrate how the archetypes help us to identify systematic and variable AI limitations across different usage domains. Overall, we argue that our invisible failure taxonomy can be a key component in reliable failure monitoring for product developers, scientists, and policy makers. Our code and data are available at https://github.com/bigspinai/bigspin-invisible-failure-archetypes
Alex: Welcome to another episode of ResearchPod.
Sam: The report is called "Invisible Failures in Human-AI Interactions," by Christopher Potts and Moritz Sudhof from Bigspin AI and Stanford. It looks at real conversations between people and AI systems like ChatGPT.
Alex: So this is about times when AI messes up, but the person doesn't notice or complain?
Sam: Exactly. The key finding is that in a big set of these chats, 78% of the AI's failures are invisible—meaning the AI didn't meet the user's goal, but the user gave no sign of being unhappy or correcting it. They analyzed over 196,000 English transcripts from the WildChat dataset, the largest collection of real, unfiltered ChatGPT conversations.
Alex: Okay, so most problems slip by without anyone flagging them. Why does that matter for building better AI?
Sam: Standard checks—like user satisfaction scores or how fast the AI responds—miss these silent slip-ups. The study shows these invisible failures aren't random; they group into eight common patterns, based on signals from the chats, like the AI drifting off-topic or confidently giving wrong info. When they checked with a newer AI called Claude Opus, 91% of these issues came from how the AI interacts—things like churning out smooth text instead of asking for clarification—rather than just lacking smarts.
Alex: Right, so even if AI gets brainier, these conversation glitches persist. Like the user asks for a 50-word summary but gets 100 words of unrelated fluff—and just accepts it?
Sam: That's a perfect example of one archetype, the silent mismatch. The report argues this taxonomy of eight patterns could help developers spot and fix these hidden problems systematically, since relying on complaints alone leaves most failures undetected.
Alex: Huh... that changes how we think about monitoring AI. So if complaints miss most issues, how did they actually figure out the quality of these chats? What counts as a good conversation versus one that's falling short?
Sam: They rated each chat on a simple four-level scale based on whether the AI hit the user's main goal. The top level means the task got done cleanly, like finishing homework right the first time. Next is when it works but with some fixable hiccups, like getting the answer but with a small error you can overlook. Lower levels are when the task fails badly or gives confidently wrong info that could trick someone. They labeled these as good, acceptable, poor, and critical.
Alex: Okay, so good is smooth success, acceptable has glitches but still okay. What did the numbers look like across thousands of chats?
Sam: Good and acceptable together covered about 84 percent. But even acceptable ones often hide real trouble—93 percent of them had at least one warning sign from their list of 28 tags.
Alex: Wait, so these acceptable chats seem fine on the surface, but they're loaded with signals? Give me an example of what that looks like.
Sam: Picture a developer asking for code, and the AI delivers something that runs but uses outdated commands no one should touch anymore. Or someone requests a quick business report with made-up references, and it hands over a longer version than asked—and the user just takes it. These slip through because the goal sort of gets met, but the AI's being sloppy in ways that add up.
Alex: Huh, so the acceptable tier is like a sneaky problem area—success masks the flaws. And how do these signals tie into overall quality?
Sam: Conversations rated good usually have zero or just one or two signals—90 percent have none at all. But as quality drops to poor or critical, signals pile up—almost all chats there have multiple negative ones, such as the AI drifting off track or acting overly sure when wrong. Signal count alone ends up being a solid clue to spot trouble.
Alex: So signal density flags the trouble spots pretty reliably. But with those eight archetypes for invisible failures, do they vary in how badly they tank quality?
Sam: Yes, they do. Most archetypes strongly link to poor or critical ratings—conversations that miss the mark badly or give misleading info. For instance, the silent mismatch never shows up in good chats and has the highest share of poor ones among invisible types; it's like the AI quietly delivers something off-base that sort of fits but doesn't really help.
Alex: Huh, so some patterns are sneakier failures. What about the common ones, like drift or confidence trap—do they predict low quality too?
Sam: They do, and they're especially telling because they're so frequent. Both heavily predict poor or critical outcomes, meaning when the AI veers to a related but wrong goal or states fiction with total assurance, the chat usually ends up subpar.
Alex: Interesting—users just bail when it's drifting nowhere. Do these archetypes ever team up, or do they happen solo?
Sam: They often overlap in telling ways. The researchers measured how much certain pairs stick together more than chance. They call this measure PPMI, for positive pointwise mutual information; darker spots on their chart highlight strong links, such as confidence trap with self-contradiction, or drift leading to walkaways.
Alex: Oh, so confident errors unravel into contradictions, or drifts make users quit. That points to bigger failure chains. Because models keep getting sharper, how did they test if these stick around?
Sam: They reran tags on a fresh sample using newer AIs like Claude Opus—distributions matched closely. Earlier, Claude Opus tagged 91% of failures as interaction-driven, not just smarts gaps, estimating 94% would linger in upgraded models. The paper suggests these dynamics are structural, evading fixes from capability alone.
Alex: So these patterns stick around because they're about conversation habits, not just raw brainpower. How exactly did they separate those—interaction issues from straight-up knowledge gaps?
Sam: They took over a thousand transcripts where the AI clearly missed the goal, then had Claude Opus check each one step by step. First, it confirmed if there really was a problem—95 percent passed that test. Then it asked: did the original AI lack the facts, logic, or skills to do the job? Or did the flop come from chat behavior, such as spitting out text instead of saying "I need more details"?
Alex: Okay, so capability is "can't do it," interaction is "won't ask or check right." What did Claude Opus decide overall?
Sam: In 91 percent of cases, interaction played a big role—58 percent mostly that, 33 percent a mix. Only 7 percent were mainly capability. And 94 percent of these would likely hang on even in smarter AIs.
Alex: Huh, so chatting style causes most trouble, and it won't vanish easily. Pulling it all together, though—what's the big picture here on why these patterns matter so much?
Sam: The core insight is that 91 percent of failures trace back to how AI handles conversations—persistent habits like over-generating or drifting—rather than just gaps in knowledge or smarts. These hold steady even as models improve, pointing to a need for fixes in interaction design over raw power alone. This shifts monitoring from end results to real-time tracking of archetype signals across chats.
Alex: Right, so conversation flow is the real weak spot. Does that mean developers could catch millions of these daily issues before users notice?
Sam: Exactly—this taxonomy applies to any set of human-AI chats, letting teams flag over 20 million invisible failures a day in active systems. It guides targeted changes, like UI tweaks to prompt clarifications or behavioral rules to curb drift.
Alex: Huh, that scales up monitoring without needing every user to report problems. But to keep it balanced, what are the limits to this approach?
Sam: A few key ones stand out. The signals were tuned for English text chats from WildChat, so voice, images, or other languages would need adjustments. Quality ratings rely on classifier judgments and team reviews, not direct user feedback. The data captures one snapshot of AI abilities, so exact rates may shift with newer models—though the patterns seem more lasting.
Alex: Fair points—they keep the findings grounded. So overall, this maps out a hidden layer of AI chats worth watching closely.
Sam: It does. By spotlighting these durable interaction pitfalls, the work offers a clear path to more reliable human-AI exchanges. That's the meaningful contribution here.
Alex: Thanks, Sam—solid breakdown. And that's our look at invisible failures in AI interactions. Thanks for listening to ResearchPod.