ResearchPod Summary
Reality monitoring is the cognitive process of distinguishing between information that was externally perceived and information that was internally generated. In humans, failures in this process are linked to hallucinations and confabulation. As LLMs are increasingly deployed in autonomous, multi-turn roles, the ability to correctly attribute the source of information—whether it originated from a user or the model itself—becomes a critical safety requirement rather than just a performance metric.
The researchers adapted a classic cognitive psychology paradigm to test six different LLMs (Gemma 3, Llama 3.3, and Llama 4 variants). They used a word-pair task where models either reproduced an externally provided word or generated their own. The study manipulated the structure of conversational memory across three conditions:
In the Single-Turn condition, models achieved near-ceiling accuracy for internally generated items because the information remained in the active context window. However, when the researchers introduced episodic delay (the Episodic-Chain paradigm), this advantage vanished, and models struggled significantly more with internal items than external ones.
Furthermore, the study found that corrective feedback is a double-edged sword. While it can improve accuracy in some contexts, it often leads to a decoupling of confidence from correctness. In some models, feedback even caused a complete swap of internal and external judgments. These dissociations are often invisible to standard benchmarks, highlighting that current evaluation methods may overlook fundamental failures in how models represent their own knowledge.
This research demonstrates that evaluating what an AI knows is insufficient; we must also evaluate how it tracks the provenance of that knowledge. As models take on more autonomous roles, the inability to distinguish between their own outputs and user inputs creates a high risk of models treating their own errors as established facts. The findings suggest that future safety efforts should focus on the architecture of conversational memory and the model's intrinsic ability to perform source attribution.
AI-generated third-party summary by ResearchPod. Not official content or an endorsement by the paper authors or affiliated organizations.