WaterMoE: Expert-Routing-based Watermarking for High Fidelity and Efficiency

As Large Language Models (LLMs) become increasingly prevalent, the need for reliable provenance tracking has grown. However, existing watermarking methods often suffer from two major drawbacks: they degrade the quality of generated text—particularly in complex, constrained tasks like coding or reasoning—and they introduce significant computational overhead that hinders deployment in latency-sensitive production environments. This paper asks whether it is possible to design a watermarking mechanism that maintains high detection rates, preserves output fidelity, and operates with minimal efficiency costs.

The authors propose WaterMoE, a watermarking scheme tailored for Mixture-of-Experts (MoE) LLMs. Unlike traditional methods that modify token logits during the final sampling stage, WaterMoE injects a subtle, deterministic bias into the routing decisions at each MoE layer. By using a precomputed 'Green Expert Map,' the model is steered toward specific experts during inference. Because these experts are functionally similar, the model's output remains high-quality and task-compliant. The watermark is then detected by comparing the likelihood of the generated text under the biased routing configuration versus a reference unwatermarked configuration.

WaterMoE demonstrates superior performance compared to state-of-the-art baselines across a comprehensive benchmark of low- to high-complexity tasks. In highly constrained domains like competitive coding and instruction following, WaterMoE maintains generation quality nearly identical to unwatermarked models, whereas traditional methods often see significant performance drops. Furthermore, because the routing bias is applied as a simple element-wise addition during the forward pass, WaterMoE incurs only 1% additional inference latency, offering up to a 4x speedup in embedding efficiency compared to existing token-sampling-based watermarking techniques.