WuYu-EnvLE-Bench: A Benchmark for Evaluating Large Language Models in Environmental Law Enforcement

Overview

Environmental law enforcement is a high-stakes, multi-stage process that requires integrating pollution facts, monitoring data, and legal standards into traceable administrative decisions. As large language models (LLMs) are increasingly considered for this domain, it is critical to understand their ability to handle the full lifecycle of enforcement—from initial risk discovery to final penalty reasoning. This paper introduces WuYu-EnvLE-Bench, a comprehensive benchmark designed to evaluate LLMs across 14 tasks and 12 pollution-medium subdomains, using 2,521 real-world enforcement instances.

Research Approach

The authors constructed the benchmark using a case-to-task pipeline derived from actual enforcement materials, including inspection records, inquiry transcripts, and legal documents. The evaluation framework organizes tasks into three stages: pre-enforcement (risk discovery), in-enforcement (inspection and evidence gathering), and post-enforcement (penalty decision-making). To assess model performance, the authors developed the Absolute Environmental Enforcement Score (AES) and the Intelligent Enforcement Index (IEI), which account for both task accuracy and resource efficiency. They utilized an LLM-as-a-Judge framework to evaluate open-ended responses based on factual consistency, legal reasoning, and procedural adherence.

Key Findings

The study reveals that while LLMs are proficient at rule-bounded tasks—such as those involving explicit facts or standardized outputs—they struggle significantly with the nuances of environmental enforcement. Specifically, models often fail in evidence-chain construction, identifying contradictions in multi-source data, and applying procedural logic correctly. A notable finding is the presence of diminishing returns regarding model scale: larger models do not reliably outperform medium-sized models in these complex reasoning tasks. This suggests that for practical deployment in resource-constrained enforcement environments, simply increasing model size is less effective than focusing on evidence-grounded and task-adaptive reasoning architectures.

This research provides a necessary reality check for the deployment of AI in public sector enforcement. By highlighting that current LLMs lack the reliability required for high-stakes administrative decisions, the study shifts the focus from raw parameter scaling to the development of specialized, evidence-aware models. The WuYu-EnvLE-Bench framework serves as a vital tool for developers and policymakers to assess whether an AI system is truly ready to support, rather than merely mimic, the complexities of environmental law enforcement.