The Crisis of Judicial Cognition

As generative AI tools become increasingly capable of drafting legal filings and judicial opinions, the legal system faces a fundamental challenge to its legitimacy. The author argues that a legal opinion is not merely a product of information transmission, but a record of a decisionmaker's authentic intellectual and emotional engagement with a case. When judges outsource the drafting of these opinions to AI, they risk abandoning the non-delegable duty to think, which is essential to the rule of law. This duty ensures that the person who decides a case is the same person who has heard and considered the arguments presented.

Failure Modes of Generative AI

Large language models (LLMs) are fundamentally different from human decisionmakers. While they can produce fluent text, they lack a world model, intent, and the ability to experience the social and moral stakes of a legal dispute. The author highlights several failure modes, including the instability of LLM outputs, where small changes in prompts lead to divergent results, and the tendency of these models to hallucinate legal authorities. Furthermore, the opacity of these systems makes it impossible for the public or the parties involved to understand how a specific rationale was generated, thereby undermining the accountability required for judicial decisions.

The Necessity of Human Engagement

Drawing on the philosophy of language, the author distinguishes between the designative-instrumental view of language—which treats words as mere tools for representing facts—and the expressive-intrinsic view, which sees language as a way to articulate and refine human understanding. Judicial reasoning belongs to the latter category. The process of writing an opinion is a form of thinking; it is through the effort of articulation that a judge clarifies their reasoning and confronts the complexity of the law. By delegating this process to a machine, the judge loses the opportunity for the critical reflection that constitutes the essence of wise judgment.

Institutional Requirements for Legitimacy

Ultimately, the author contends that the legal system must explicitly recognize and preserve the duty of active mental consideration. This requires that judges not only review the final output but also personally grapple with the facts and arguments. While workload pressures are real, they do not justify an abdication of judicial responsibility. The paper calls for a legal culture that prioritizes thoughtful, human-centered adjudication over the efficiency of automated rationalization.

Key Terms and Definitions

• Non-delegable duty to think — The fundamental requirement that a legal decisionmaker must personally engage with and deliberate upon the facts and arguments of a case rather than outsourcing the reasoning process to an automated system.
• Stochastic parrot — A term used to describe the tendency of LLMs to generate fluent, plausible-sounding text based on statistical patterns without any actual understanding of the underlying meaning or reality.
• Hyperobject — A concept used to describe complex, distributed systems like LLMs that are impossible for humans to fully visualize or understand, leading to inherent opacity in their decision-making processes.
• Designative-instrumental theory — A view of language that treats it as a neutral tool for representing reality, which the author argues is insufficient for the complex, value-laden work of judicial reasoning.
• Expressive-intrinsic theory — A view of language that emphasizes its role in bringing human concerns to consciousness and creating shared meaning, which the author identifies as essential to legitimate legal judgment.