Generative AI may backfire for counterspeech

Online hate speech remains a significant threat to societal cohesion and individual well-being. While counterspeech—direct, public replies intended to encourage users to reconsider hateful posts—is a promising, non-censorial intervention, it has traditionally been limited by a trade-off between manual, high-quality human intervention and scalable, generic, 'one-fits-all' automated messages. This study investigates whether generative AI, specifically large language models (LLMs), can bridge this gap by producing scalable, contextualized counterspeech that is more effective than generic alternatives.

The researchers conducted a large-scale, pre-registered field experiment on Twitter/X with 2,664 participants. They employed a 2x2 between-subjects design, comparing two types of counterspeech (contextualized LLM-generated vs. non-contextualized generic) across two strategies (empathy-based vs. warning-of-consequences). The study measured three primary outcomes: the rate of post deletion, the frequency of subsequent hateful posts, and the relative change in the toxicity of the user's language over a two-week period following the intervention.

The results indicate that non-contextualized counterspeech using a 'warning-of-consequences' strategy significantly reduces online hate speech. In contrast, contextualized counterspeech generated by Llama-3 70B proved ineffective. Furthermore, the study found evidence that LLM-generated counterspeech may backfire, potentially increasing the hostility of the target users. The authors suggest that users may perceive LLM-generated messages as less authentic or feel deceived, which undermines the persuasive intent of the intervention.

This research provides critical evidence that more advanced technology does not necessarily lead to better outcomes in social moderation. By demonstrating that LLM-generated, contextualized counterspeech can be counterproductive, the study highlights the 'uncanny valley' of automated moderation and warns against the uncritical deployment of generative AI in sensitive social interactions. It suggests that simple, clear, and non-AI-generated warnings remain a more reliable tool for curbing online hate than complex, AI-tailored responses.