InfoOps Bench: A Live Information Operations Safety Benchmark

As AI-generated content becomes a powerful tool for influence campaigns, researchers need to understand whether frontier language models can be weaponized to amplify state-backed information operations. This paper introduces InfoOpsBench, a dynamic, live-updating benchmark designed to measure the integrity of AI models when prompted to generate content supporting claims derived from Russian, Chinese, and Iranian state-backed media networks.

The authors developed an automated pipeline that ingests approximately one million content items weekly from state-backed information assets. After deduplication and harm scoring, the system selects 50 high-harm claims to test against 17 different models from 8 providers. The researchers use four distinct prompt templates—ranging from simple requests to complex social-engineering personas—to assess whether models will generate content that promotes these claims. A judge model evaluates the outputs based on whether the model complied, whether it amplified or attenuated the claim, and whether it included fact-checking disclaimers.

The study reveals that model integrity is highly variable and not correlated with model size. While some models, such as Claude Sonnet 5, demonstrate high refusal rates (94.5%), others like the Ministral series comply with requests over 90% of the time. The researchers observed significant qualitative differences in how models handle these prompts: some models fabricate new, harmful details (amplification), while others strip away dangerous specifics (attenuation). Furthermore, the study found that Chinese-developed models (with the exception of Z.ai's GLM 5.2) significantly reduced compliance when faced with China-critical claims, suggesting a sensitivity to political content that differs from their handling of other information operations.