Human-in-the-Loop User Feedback Affects Perceived Accuracy and Trust, but Task Subjectivity Matters

This research investigates how the act of providing feedback to an intelligent system—a process known as Human-in-the-Loop (HITL) interaction—influences a user's perception of the system's accuracy and their overall trust in it. While HITL approaches are often used to improve model performance, the authors explore whether the process of providing feedback itself introduces psychological biases that alter how users evaluate the system.

The authors conducted three controlled experiments comparing two distinct contexts: objective tasks (where there is a single correct answer, such as object detection in images) and subjective tasks (where the "correct" answer is a matter of opinion, such as identifying relevant words for a text topic). In the objective context, participants corrected bounding boxes; in the subjective context, they adjusted word relevance lists. The researchers measured user trust and perceived accuracy over time, while also controlling for the user's belief that their feedback was actually causing the system to update.

The study reveals a stark contrast based on task subjectivity. In objective tasks, providing feedback consistently led to lower trust and lower perceived accuracy, regardless of whether the system actually improved. Participants in these tasks tended to distrust the system over time. Conversely, in subjective tasks, this negative bias disappeared; participants did not experience the same decline in trust or perceived accuracy. The researchers suggest that when users provide feedback on subjective matters, they are essentially aligning the system with their own mental model, which mitigates the negative psychological effects observed in error-correction scenarios.

These findings suggest that designers of intelligent systems must account for the nature of the task when implementing feedback mechanisms. If a system is designed for objective tasks, designers should be aware that asking for user feedback might inadvertently cause users to lose confidence in the system's capabilities. Understanding these psychological dynamics is crucial for calibrating user trust and ensuring that feedback loops are designed to support, rather than undermine, the user-system relationship.