Haoxi Zhong, Chaojun Xiao, Cunchao Tu, Tianyang Zhang, Zhiyuan Liu, Maosong Sun
9 min
Legal Artificial Intelligence (LegalAI) aims to automate the time-consuming tasks inherent in legal work, such as reviewing documents, predicting case outcomes, and matching similar precedents. The field sits at an intersection between two distinct approaches: the traditional rule-based or symbol-based systems favored by legal professionals for their transparency, and the data-driven embedding methods (such as deep neural networks) preferred by NLP researchers for their performance. This paper surveys the current state of the field, highlighting the tension between these two methodologies and the ongoing efforts to integrate them.
The authors categorize LegalAI techniques into two primary buckets. Embedding-based methods use deep learning to represent discrete legal texts as continuous vectors, often utilizing Pretrained Language Models (PLMs). While powerful, these models are frequently "black boxes" that fail to account for the formal reasoning constraints defined by law. Conversely, symbol-based methods leverage extracted legal entities, events, and relationships. These offer higher interpretability—crucial for legal systems to remain fair and non-discriminatory—but often struggle to achieve the predictive accuracy of modern neural architectures.
The paper highlights three key domains of LegalAI:
To move from theoretical models to real-world deployment, the field must address the challenges of knowledge modeling, legal reasoning, and interpretability. The authors stress that the objective of LegalAI should not be to replace practitioners, but to act as a reliable assistant. Models must be designed to serve as a reference, leaving final, high-stakes decisions to humans while reducing the burden of repetitive administrative tasks.
Legal Artificial Intelligence (LegalAI) focuses on applying the technology of artificial intelligence, especially natural language processing, to benefit tasks in the legal domain.In recent years, LegalAI has drawn increasing attention rapidly from both AI researchers and legal professionals, as LegalAI is beneficial to the legal system for liberating legal professionals from a maze of paperwork.Legal professionals often think about how to solve tasks from rulebased and symbol-based methods, while NLP researchers concentrate more on data-driven and embedding methods.In this paper, we describe the history, the current state, and the future directions of research in LegalAI.We illustrate the tasks from the perspectives of legal professionals and NLP researchers and show several representative applications in LegalAI.We conduct experiments and provide an indepth analysis of the advantages and disadvantages of existing works to explore possible future directions.You can find the implementation of our work from https://github. com/thunlp/CLAIM.
Alex: Then the representation should preserve facts that matter legally, rather than just making similar paragraphs close together. But someone still has to detect those facts correctly.
Sam: That is a central difficulty. The authors test element extraction and find promising results, but say performance remains insufficient for the corresponding applications. Explicit structure is useful only if the system can recover it reliably.
Alex: Where do they show the approaches actually working together, rather than just arguing that they should?
Sam: In legal judgment prediction: predicting outcomes from case facts and relevant law. Their example is TopJudge. It uses a neural text encoder to represent the facts, alongside a graph describing dependencies among prediction tasks, such as identifying charges and determining penalties.
Alex: So legal structure shapes how the predictions relate to each other. What evidence supports that design?
Sam: They evaluate it on a Chinese criminal-case benchmark containing over 2.68 million legal documents. TopJudge improves on the plain text encoder in their comparison. The authors treat that as evidence that modelling relationships among legal tasks can help prediction.
Alex: Better prediction still leaves the explanation question open. Does a graph inside the model tell a lawyer why a particular case received its prediction?
Sam: That comparison does not establish a complete legal explanation. It supports the value of adding structure. My reading is that the review makes a stronger case for researching combined approaches than for treating any existing approach as ready for use.
Alex: What else goes wrong in judgment prediction, even with a dataset that large?
Sam: Rare charges and statutory provisions remain difficult. Scores that emphasize frequent labels look much better than scores that give each label equal weight. Long documents are another obstacle: the BERT language model’s input limit prevents it from reading some cases in full.
Alex: That means a strong average score can hide both missing categories and missing text. Do the retrieval experiments expose a different kind of weakness?
Sam: They do. Consider similar case matching, where the task is to identify which case is more like another. The authors use a benchmark with about nine thousand triples of legal documents. Each example asks the model to choose the closer match to a reference case.
Alex: That seems like a task where learned language representations should beat keyword overlap. Did they?
Sam: The best tested neural systems reached about seventy percent accuracy. The term-matching baseline reached about fifty-three percent. Learning semantic relationships helps, but the remaining gap is substantial.
Alex: What do the authors think those systems are missing? Is legal similarity different from describing roughly the same events?
Sam: In this dataset, legal professionals emphasize shared legal elements. Similar wording, or even similar general meaning, need not identify the legally relevant resemblance. The authors therefore propose representations based more explicitly on those elements, with legal knowledge incorporated into matching.
Alex: That proposal follows from their diagnosis, but they have not shown that an element-based matcher closes the gap.
Sam: Correct. It’s a research direction, not a demonstrated solution. The distinction matters because the review repeatedly moves between reporting experiments and recommending what researchers should try next.
Alex: Let’s turn to answering questions. That seems closer to the public-facing promise of affordable legal help. How demanding is their test?
Sam: They use a Chinese legal examination dataset containing roughly twenty-nine thousand questions. It includes questions about legal knowledge and questions requiring case analysis. On the overall evaluation, BERT achieved about twenty-two percent accuracy, compared with about eighty-one percent for skilled humans.
Alex: That is a large gap. What does a question require beyond finding a relevant passage?
Sam: The review gives an example involving counterfeit currency transported into China. Relevant material points toward both transportation and smuggling offenses. Additional legal principles explain how to handle overlapping offenses and select the more serious one.
Alex: So retrieving both offenses still does not answer the question. The system must connect them through another rule and compare their consequences.
Sam: That’s multi-hop reasoning: linking several pieces of evidence through successive steps. The authors identify it as a major difficulty, alongside understanding legal concepts. Their evidence is a warning against equating successful passage matching with competent legal advice.
Alex: How far should listeners generalize this assessment? These are particular models, tested on particular legal materials.
Sam: Keep that scope in view. This is a dated snapshot, and the experiments heavily feature Chinese legal datasets. They do not establish performance across legal systems, or show that improved scores guarantee fairness and reliable reasoning in practice.
Alex: Given those limits, who should read the full document, and where should they begin?
Sam: Researchers entering legal language processing should read the application sections, especially the experiments and analyses on similar case matching and legal question answering. They show why seemingly familiar language tasks demand different representations and reasoning. Then read the conclusion on ethics and professional oversight.
Alex: And what should everyone else carry away, particularly someone deciding whether to use a model’s answer?
Sam: The authors say model outputs should be references for legal professionals, not replacements for them. Even simpler cases handled with model assistance still need review. The line to remember is this: predicting the answer is not enough; legal assistance must support reasons and human responsibility.
Alex: Keep that distinction in view when you read the next benchmark.