Taxonomy of Risks on Automated Fact-Checking Systems Considering its Propagation

Automated fact-checking systems, powered by large language models (LLMs) and artificial intelligence, are increasingly seen as essential tools for managing the deluge of misinformation on social media. However, these systems introduce unique risks. Beyond standard cybersecurity threats, they are susceptible to adversarial manipulation—where users iteratively refine disinformation until the system validates it as 'true'—and technical failures like hallucinations. Current risk assessment frameworks, such as STRIDE, were designed for traditional IT systems and struggle to address the nuances of generative AI and the societal impacts of automated verification.

To address this gap, the authors developed a taxonomy of 32 specific risks inherent to automated fact-checking. They adopted a three-stage propagation model: identifying risk factors (the source), hazardous situations (the mechanism), and harm (the outcome). By synthesizing insights from existing generative AI risk taxonomies and AI safety guidelines, the researchers categorized these risks into a structured fault tree. This approach allows developers to trace the logical relationships between a technical failure (e.g., model bias) and its ultimate societal consequence (e.g., defamation or the viral spread of state-sponsored disinformation).

The authors tested their taxonomy by applying it as a set of 'guide words' to the DEFAME automated fact-checking system. By performing a risk assessment on the system's data flow, they demonstrated that their framework successfully identified critical risks—such as the potential for system-endorsed misinformation—that were invisible to the traditional STRIDE methodology. This suggests that specialized taxonomies are necessary for the safe deployment of AI-driven tools in sensitive domains like truth verification.