Your Mouse and Eyes Secretly Leak Your Preference: LLM Alignment using Implicit Feedback from Users

Traditional LLM alignment relies on explicit human feedback (e.g., thumbs-up/down), which is sparse, expensive to collect, and often fails to capture the nuance of user preferences. This paper investigates whether implicit signals—specifically mouse trajectories and webcam-based eye-gazing data—can serve as a scalable, high-quality alternative for training reward models and aligning LLMs.

The authors developed a custom crowdsourcing platform to collect the IFLLM dataset, which includes 1,336 multi-turn interactions from 59 participants. During these sessions, the system recorded mouse movements and eye-gazing patterns (via webcam) while users evaluated responses from various LLMs. The researchers then trained a Random Forest reward model using these implicit features alongside standard text-based features. Finally, they evaluated the impact of these reward models on LLM performance using Direct Preference Optimization (DPO).

The study reveals that user reading behavior is highly diverse and strongly influenced by response length and interface layout. Notably, mouse trajectories and eye-gazing data provide complementary signals: eye-gazing is more informative for short responses, while mouse movement becomes a critical indicator for longer responses that require scrolling. The inclusion of these implicit signals boosted the reward model's preference prediction accuracy from 55% to 64%. Furthermore, when used to guide DPO, these signals nearly tripled the relative improvement in response quality compared to baselines that only utilized text-based feedback.

This research demonstrates that implicit feedback is a powerful, underutilized resource for LLM alignment. By leveraging signals that are already available in real-world web environments, developers can move beyond the limitations of explicit feedback. This approach not only improves average model performance but also paves the way for personalized alignment, where models can adapt to individual user preferences in real-time without requiring constant manual input.