Distributed Denial of Science: How Indirect Data Poisoning of AI Systems Can Industrialize Scientific Fraud

As AI agents become increasingly central to scientific research, they are frequently tasked with autonomously retrieving and analyzing data from open repositories. This paper investigates a new, scalable threat: indirect data poisoning. In this scenario, a malicious actor uploads a corrupted dataset to a public platform. When an AI research agent retrieves this data, it may unknowingly process the fraudulent information, leading to biased or entirely false scientific conclusions. Unlike traditional fraud, which requires significant resources and human accomplices, this method allows an adversary to weaponize the honest, automated workflows of AI systems to distribute misinformation at scale.

The researchers tested this vulnerability across five socially significant topics, including hiring discrimination, autonomous vehicle safety, and the motivational impact of AI. Using three frontier AI systems (Claude, GPT-based Codex, and Gemini), they conducted 450 experimental runs. The adversary's goal was to either reject a hypothesis or exaggerate its findings by manipulating metadata and data content. The agents were instructed to perform research without any direct mention of the poisoned datasets, simulating a realistic, independent research process.

The results indicate that indirect data poisoning is alarmingly effective, with a 49.56% success rate in producing poisoned conclusions, while the agents detected the manipulation in only 6% of cases. In some instances, the AI agents even used the poisoned data to discredit legitimate, pre-existing research. To counter this, the authors evaluated two defensive strategies. While adopting a 'scientist persona' (prompting the AI to be more critical) provided only marginal protection, implementing a structured 'data provenance audit'—which checks for statistical anomalies, cross-dataset consistency, and other markers—successfully reduced the attack success rate to zero.