Available but Unclaimed: An Empirical Study of Human–AI Synergy

This study investigates the conditions under which human-AI teams achieve synergy—defined as performance exceeding both the human and the AI working alone. The researchers conducted a between-subjects experiment with 535 participants who completed a 40-item battery of reasoning tasks (matrix reasoning, mental rotation, syllogisms, and letter-string analogies). Participants worked either unaided or with one of four LLMs (GPT-5.6-Luna, Claude Opus 4.8, Gemini 3.6 Flash, or Kimi K3). To establish a rigorous baseline, the researchers measured each LLM's competence on the exact same items over 100 independent runs, allowing for a granular, item-level comparison between human performance, AI performance, and team performance.

• Synergy is Rare: On average, human-AI teams did not outperform the better of their two components. While assisted participants generally performed better than unaided ones, they rarely exceeded the accuracy of the AI alone.
• Pass-through Limitations: Assisted accuracy increases only about half as steeply as LLM competence. This suggests that even as LLMs become more capable, the team does not capture the full potential of that increased intelligence.
• Reliance Patterns: Participants' deference to AI advice varied by task and tracked item-level competence to some degree. However, participants frequently adopted incorrect advice, and their post-advice confidence was less effective at distinguishing correct from incorrect answers compared to unaided performance.
• Metacognitive Gaps: Stated confidence levels remained relatively stable even when the assistant's actual accuracy fluctuated significantly, indicating that users lack a reliable signal to judge when to trust the AI.

These findings challenge the assumption that simply providing a more capable AI will automatically lead to better team performance. The results suggest that the "jagged frontier" of AI reliability—where a model is highly accurate on one item but fails on a similar one—is a major hurdle for human-AI interaction. Because users often rely on task-level heuristics rather than item-level evaluation, they struggle to filter out incorrect advice. The study highlights the need for interface designs that support selective deference and help users calibrate their trust based on the specific item at hand, rather than relying on a general impression of the model's quality.