Rethinking Forgery Attacks on Semantic Watermarks in Black-Box Settings: A Geometric Distortion Perspective

As generative AI models become ubiquitous, semantic watermarking has emerged as a key tool for provenance and attribution. However, recent research suggests that adversaries can perform black-box forgery attacks—using a proxy model to invert a watermarked image and regenerate it with a different content while preserving the original watermark. This paper investigates the theoretical limits of these attacks and asks whether we can detect forged samples by analyzing the inherent distortions they introduce in the latent space.

Instead of relying on empirical trial-and-error, the authors model black-box forgery as a rate-distortion problem. They argue that when an adversary uses a proxy model to invert a target model's output, the structural mismatch between these models creates an irreducible distortion floor. This distortion is not mere stochastic noise; rather, it manifests as structured geometric deviations on the latent manifold. The authors characterize these deviations using two metrics: Spherical Angular Distortion (SAD), which measures global directional drift on the latent hypersphere, and Local SPD Geometric Inconsistency (LGI), which measures structural deformation on the manifold of Symmetric Positive Definite (SPD) matrices.

• The authors prove that model mismatch imposes a fundamental lower bound on the distortion of any forged latent, meaning high-fidelity forgery is theoretically impossible when the proxy and target models differ.
• Forged latents exhibit consistent geometric signatures—specifically global angular drift and local structural deformation—that are absent in genuine watermarked latents.
• A scheme-agnostic detection method based on these geometric metrics achieves near-perfect AUC scores across diverse black-box scenarios, effectively flagging forgeries without requiring modifications to existing watermarking schemes.