It Matters How You Say It: Exploring Rhetorical Patterns for AI-Assisted Information Evaluation

Most AI-assisted information evaluation systems rely on directive rhetoric, where the system provides a confident verdict that users often passively accept. This study investigates whether alternative rhetorical patterns—drawn from linguistics and psychology—can induce deeper contemplation and improve user accuracy in fact-checking tasks. The authors tested eight distinct rhetorical patterns: Intentional Misleading, Interpretive Alternative, Scaffold Explanation, Triggering Distrust, Information Distortion, Alternative Framing, Socratic Questioning, and an Oracle baseline.

Using a within-subject study with 98 participants, the researchers evaluated how these patterns influenced performance on a hint-on-demand fact verification task. Participants were asked to evaluate the truthfulness of various claims, with the option to request AI advice structured according to one of the eight rhetorical styles. The study measured accuracy, confidence calibration, and user preference to determine the trade-offs between satisfaction and critical reasoning.

The study found that Scaffold Explanation, which guides users through logical deduction, yielded the highest accuracy gains and successfully encouraged deeper reflection. Interestingly, adversarial patterns like Triggering Distrust and Intentional Misleading also led to modest improvements in accuracy. The authors hypothesize that these adversarial styles force users to adopt a more skeptical, motivated stance, which inadvertently improves their ability to verify the underlying claims.

A significant tension emerged between user preference and objective performance. Participants expressed the highest preference for Alternative Framing, yet this style did not correlate with the highest accuracy gains. Conversely, Interpretive Alternative was the least preferred, largely due to the perceived time cost. This divergence suggests that user satisfaction is a poor proxy for the effectiveness of AI advisory systems, highlighting the need for designers to balance user experience with the cognitive demands required for accurate information evaluation.