The Judgment-Consequence Gap: LLM Moral Reasoning in Healthcare Decisions

This study investigates whether Large Language Models (LLMs) mirror human moral reasoning when faced with high-stakes healthcare decisions, specifically the allocation of scarce medical resources. The authors focus on the 'judgment-consequence gap'—the potential disconnect between acknowledging a patient's responsibility for their illness and using that responsibility to inform treatment priority. Using clinical vignettes adapted from human studies on kidney, lung, and hip treatments, the researchers tested 12 LLMs across 19 configurations (including both reasoning and non-reasoning modes) to trace moral judgment from initial behavior to final allocation.

The researchers identified a consistent pattern: LLMs and humans largely agree that patients bear responsibility for health-harming behaviors (e.g., smoking or poor diet). However, while humans often use this responsibility to justify prioritizing less-culpable patients for scarce resources, LLMs overwhelmingly default to random allocation. The models explicitly reject behavior-based prioritization as unfair, even when they have already assigned high moral responsibility to the patient for their condition.

The study also found that LLMs are significantly more sensitive than humans to the patient's knowledge state. If a patient was unaware of the health risks associated with their behavior, LLMs drastically reduced their responsibility scores, adhering to a strict informed-consent framework that differs from human intuition. Furthermore, enabling 'thinking' or extended reasoning modes in LLMs actually widened the gap; while these modes made the models' responsibility assessments more similar to human judgments, they simultaneously pushed allocation decisions further toward randomization, suggesting that this behavior is a stable normative commitment rather than a simple processing error.