Supporting Reflection in LLM-based Exploratory Search

Exploratory search is a complex, iterative process involving the discovery of new information, the refinement of goals, and the synthesis of knowledge over time. While Large Language Models (LLMs) can accelerate information retrieval, they often prioritize providing immediate, direct answers. This can inadvertently discourage the critical sensemaking and metacognitive effort required to navigate unfamiliar domains, potentially leading users to accept biased or incomplete information without sufficient reflection.

To address these challenges, the authors developed TrailLM, a prototype interface designed to make the exploratory search process more intentional. The system shifts the focus from simple prompt-response interactions to a structured, history-aware workflow. By externalizing the user's search process, TrailLM helps users maintain awareness of their progress and the evolution of their information needs.

TrailLM incorporates several key affordances to support metacognition:

• Goal Alignment Monitoring: The system displays the current exploration goal and allows users to define and modify subgoals, ensuring that the AI's assistance remains aligned with the user's changing intent.
• Navigable Interaction History: The interface organizes past interactions into a 'trail' of tiles, enabling users to revisit previous steps, link them to specific subgoals, and filter their history to maintain focus.
• Emerging Path Discovery: By visualizing alternative directions and subgoals, the system encourages users to explore different facets of a topic rather than settling for the first response provided by the LLM.

As LLMs become standard tools for research and learning, there is a risk that users will become passive consumers of AI-generated content. TrailLM represents a shift toward 'alignment-centered' design, where the interface actively scaffolds the user's cognitive processes. By forcing a more deliberate interaction with the search history, the system aims to preserve the critical thinking skills necessary for deep learning and effective sensemaking in complex information spaces.