AgentTrails: Towards Trust and Reuse for Agentic Tasks

LLM-powered agents often execute complex tasks by calling tools and manipulating data artifacts. These actions are typically recorded as chronological logs, which are difficult to interpret because they obscure the underlying dataflow—the specific dependencies between an agent's actions and the artifacts they produce or consume. This lack of structure makes it challenging for developers to debug failures, compare different execution strategies, or identify reusable workflows.

AgentTrails addresses these challenges by converting raw, sequential agent logs into structured provenance graphs. The system operates in three stages:

1. Overview: It provides a tool-agnostic, UpSet-style visualization that allows users to filter and group traces based on metrics like cost, latency, and tool usage frequency.
2. Provenance Reconstruction: It parses raw logs to identify producer-consumer relationships. By analyzing arguments, responses, and artifact identifiers, it constructs a bipartite graph where tool calls are represented as 'activities' and inputs/outputs as 'entities.'
3. Multi-Trace Comparison: To handle stochastic agent behavior, AgentTrails uses a 'joined quotient graph.' It clusters similar activities across multiple traces into 'activity capsules,' allowing users to visualize shared workflow motifs, divergent branches, and anomalous paths across different runs.

By exposing the internal dependency structure of agentic tasks, AgentTrails moves beyond simple log inspection. It enables researchers and developers to validate scientific workflows, identify redundant tool calls, and isolate the specific branches that lead to successful or failed outcomes. This structured view is a critical step toward building more reliable, auditable, and reusable agentic systems.